JP7274324B2 - Image processing device and image processing method - Google Patents

Description

The present invention relates to an image processing device and an image processing method.

2. Description of the Related Art An image processing device is known that simultaneously displays an image of the outside of a vehicle and the situation of the vehicle on a display mounted on the vehicle. Patent Literature 1 discloses an image processing device that displays an image of the front of the vehicle and an icon or the like indicating the tilt angle of the vehicle in the left-right direction on a display mounted on the vehicle.

When the tilt angle of the vehicle is displayed using an icon or the like, it is considered that the greater the size of the icon or the like, the easier it is for the occupant of the vehicle to understand the tilt angle of the vehicle. However, if the size of the icon or the like indicating the tilt angle of the vehicle is increased, the display space for the image of the front of the vehicle or the like becomes smaller, so the visibility of the image of the front of the vehicle or the like is reduced. On the other hand, if the size of the icon or the like indicating the tilt angle of the vehicle is reduced, it may be difficult for the passengers of the vehicle to understand the tilt angle of the vehicle.

Therefore, it is preferable to allow the occupants of the vehicle to easily understand the inclination angle of the vehicle without reducing the visibility of the image of the exterior of the vehicle.

JP 2016-49868 A

The present invention provides an image processing apparatus and an image processing method for generating a composite image that allows passengers of a vehicle to easily understand the tilt angle of the vehicle without reducing the visibility of an image of the exterior of the vehicle. intended to

(First invention)
An image processing apparatus according to a first aspect of the present invention includes an inclination calculator that calculates a roll angle and a pitch angle of the vehicle from the output of a sensor that detects vehicle attitude information, and a camera image generated by a camera. an image acquisition unit for acquiring, a first image generation unit for generating a first image of a first predetermined shape having a display mode according to the roll angle, and a second predetermined shape having a display mode according to the pitch angle a second image generation unit that generates a second image; arranges the first image at an upper end or a lower end of the camera image acquired by the image acquisition unit; an image synthesizing unit that arranges the second image and generates a synthetic image.

In the first invention, an image processing device combines a camera image generated by a camera with a first image and a second image that display the inclination of the vehicle to generate a composite image. Since the first image has the first predetermined shape and the second image has the second predetermined shape, a wide display area for the camera image can be secured in the synthesized image. Therefore, the image processing apparatus according to the first aspect of the present invention can allow passengers of the vehicle to easily understand the inclination angle of the vehicle without degrading the visibility of the camera image of the exterior of the vehicle.

(Second invention)
In the second invention, the first image includes a first frame region corresponding to the first predetermined shape, a roll angle display region arranged inside the first frame region and indicating the roll angle, and the roll angle display region. a first background area that is arranged inside one frame area and that is a separate area from the roll angle display area, wherein the first image generation unit determines the roll angle display area based on the roll angle. a position is determined, pixel values in the first background area are determined to be pixel values different from pixel values in the roll angle display area, and the second image includes a second frame area corresponding to the second predetermined shape; a pitch angle display area arranged inside the second frame area and indicating the pitch angle; and a second background area arranged inside the second frame area and being a separate area from the pitch angle display area. wherein the second image generation unit determines a position of the pitch angle display area based on the pitch angle, and sets pixel values in the second background area to pixel values different from pixel values in the pitch angle display area. to decide.

In the second invention, the position of the roll angle display area is changed according to the roll angle, and the position of the pitch angle display area is changed according to the pitch angle. The pixel values of the roll angle display area are different from the pixel values of the first background area, and the pixel values of the pitch angle display area are different from the pixel values of the second background area. Therefore, the occupant of the vehicle can easily recognize the roll angle and the pitch angle.

(Third invention)
In the third invention, the first image generator determines a color of the roll angle display area based on the roll angle, and the second image generator determines the pitch angle display area based on the pitch angle. determine the color of

In the third invention, the color of the roll angle display area is changed according to the roll angle, and the color of the pitch angle display area is changed according to the pitch angle. Therefore, a vehicle occupant can easily understand the inclination angle of the vehicle from the color of the roll angle display area and the color of the pitch angle display area.

(Fourth Invention)
In the fourth invention, the first image is generated to extend in the horizontal direction, and the first image generation unit arranges the roll angle display area to the right of a reference line indicating a roll angle of 0 degrees. The second image is formed to extend in the vertical direction, and the second image generation unit adjusts the pitch angle display area from a reference line indicating a pitch angle of 0 degrees. It is determined based on the pitch angle whether or not to place the pitch angle above.

In the fourth invention, it is determined whether or not to arrange the roll angle display area to the right of the reference line based on the roll angle of the vehicle, and the pitch angle display area is arranged on the reference line based on the pitch angle of the vehicle. Decide whether to Therefore, the occupant of the vehicle can easily understand in which direction the vehicle is tilted in the left-right direction and in which direction in the front-rear direction the vehicle is tilted.

(Fifth Invention)
In the fifth invention, the first image is generated to extend in the horizontal direction, and the first background area includes a left background area arranged to the left of the roll angle display area and the roll angle display area. and a right background area arranged to the right of the roll angle display area, wherein the first image generation unit gradually converts the pixel values of the left background area and the right background area as the distance from the roll angle display area increases. The second image is generated to extend in the vertical direction, and the second background area includes an upper background area arranged above the pitch angle display area and an upper background area arranged below the pitch angle display area. and a lower background area, and the second image generation unit gradually changes pixel values of the upper background area and the right background area as the distance from the pitch angle display area increases.

In the fifth invention, the first image generator gradually changes the pixel values of the left background area and the right background area as the distance from the roll angle display area increases. Also, the second image generator gradually changes the pixel values of the upper background area and the right background area as the distance from the pitch angle display area increases. Therefore, the image processing apparatus according to the fifth aspect of the invention can easily make the vehicle occupant understand the inclination angle of the vehicle.

(Sixth Invention)
In the sixth invention, the first image is generated to extend in the horizontal direction, and the first image generating section determines the horizontal length of the roll angle display area based on the roll angle, and the It is determined based on the roll angle whether the roll angle display area is arranged to extend to the right or to the left of a reference line indicating a roll angle of 0 degrees, and the second image is a vertical image. The second image generator determines the length of the pitch angle display area in the vertical direction based on the pitch angle, and sets the pitch angle display area to a reference indicating a pitch angle of 0 degrees. Whether to extend upward or downward from the line is determined based on the pitch angle.

In the sixth invention, the roll angle display area extends in a direction based on the roll angle and has a length based on the roll angle. Also, the pitch angle display area extends in a direction based on the pitch angle and has a length based on the pitch angle. Therefore, a vehicle occupant can easily understand the inclination angle of the vehicle from the direction and length in which the roll angle display area and the pitch angle display area extend.

(Seventh Invention)
In the seventh invention, the roll angle display area has a constant length and is arranged at a position based on the roll angle, the pitch angle display area has a constant length, and the pitch angle is positioned based on the

In the seventh invention, the position of the roll angle display area changes based on the roll angle, and the position of the pitch angle display area changes based on the pitch angle. A vehicle occupant can easily understand the tilt angle of the vehicle based on the positions of the pitch angle display area and the roll angle display area.

(Eighth Invention)
In the eighth invention, the first image is generated to extend in the horizontal direction, and the first image generating section rotates the first image extending in the horizontal direction at an angle corresponding to the roll angle.

In the eighth invention, the inclination of the first image extending in the horizontal direction changes according to the roll angle. Therefore, the vehicle occupant can understand the inclination angle of the vehicle 1 in the left-right direction from the inclination of the first image.

(Ninth Invention)
In the ninth invention, the first image generation unit arranges characters representing the roll angle in the vicinity of the first frame region, and the second image generation unit arranges characters representing the pitch angle in the second direction. Place it near the frame area.

An image processing apparatus according to a ninth aspect arranges characters representing the roll angle of the vehicle in the vicinity of the first frame area, and arranges characters representing the pitch angle of the vehicle in the vicinity of the second frame area. A vehicle occupant can more easily understand the inclination angle of the vehicle.

(Tenth invention)
An image processing apparatus according to a tenth aspect of the present invention includes a tilt calculation unit that calculates the pitch angle of the vehicle from the output of a sensor that detects vehicle posture information, and an image acquisition unit that acquires a camera image generated by a camera. determining a relative position between a front end surface image corresponding to the front end surface of the vehicle and a rear end surface image corresponding to the rear end surface of the vehicle based on the pitch angle; A pitch angle image generation unit that generates a pitch angle display image including an image and a rear end surface image; and a synthesized image is generated by combining the camera image and the pitch angle display image acquired by the image acquisition unit. and an image synthesizing unit.

In the tenth invention, the relative positions of the front end surface image showing the front end surface of the vehicle and the rear end surface of the vehicle are changed based on the pitch angle. Therefore, the occupant of the vehicle can easily understand the tilt angle in the longitudinal direction of the vehicle.

(Eleventh Invention)
An image processing method according to an eleventh aspect includes a tilt calculation step of calculating a roll angle of the vehicle and a pitch angle of the vehicle from the output of a sensor that detects vehicle attitude information, and a camera image generated by a camera. a first image generating step of generating a first image having a first predetermined shape having a display mode corresponding to the roll angle; and a second predetermined shape having a display mode corresponding to the pitch angle. a second image generating step of generating a second image; arranging the first image at the upper end or lower end of the camera image obtained by the image obtaining step; and an image synthesizing step of arranging the second image to generate a synthesized image.

In the eleventh invention, the image processing method combines the camera image generated by the camera with the first and second images representing the tilt of the vehicle to generate a composite image. Since the first image has the first predetermined shape and the second image has the second predetermined shape, a wide display area for the camera image can be secured in the synthesized image. Therefore, the image processing method according to the eleventh aspect of the invention enables the vehicle occupants to easily understand the inclination angle of the vehicle without degrading the visibility of the camera image of the vehicle exterior.

The present invention provides an image processing apparatus and an image processing method for generating a composite image that allows passengers of a vehicle to easily understand the tilt angle of the vehicle without reducing the visibility of an image of the exterior of the vehicle. can do.

1 shows an example of a schematic diagram of a vehicle equipped with an image recording device according to Embodiment 1 of the present invention. 2 is a diagram for explaining directions in the vehicle shown in FIG. 1; FIG. 2 is a functional block diagram of the image processing apparatus shown in FIG. 1; FIG. 2 is a diagram for explaining a synthesized image synthesized by the image processing apparatus shown in FIG. 1; FIG. 2 is a diagram for explaining the operation of the image processing apparatus shown in FIG. 1; FIG. It is a figure explaining the 1st image which concerns on Embodiment 2 of this invention. It is a figure explaining the 2nd image which concerns on Embodiment 2 of this invention. It is a figure explaining the 1st image and 2nd image which concern on Embodiment 3 of this invention. It is a figure explaining the 1st image and 2nd image which concern on Embodiment 4 of this invention. It is a figure explaining the 1st image and 2nd image which concern on Embodiment 5 of this invention. FIG. 11 is a functional block diagram of an image processing apparatus according to Embodiment 6 of the present invention; 12A and 12B are diagrams for explaining a synthesized image synthesized by the image processing apparatus shown in FIG. 11; FIG. FIG. 10 is a first example of diagrams for explaining a first image and a second image according to a modification of the present invention; FIG. 11 is a second example of diagrams for explaining a first image and a second image according to a modification of the present invention; FIG. 11 is a third example of diagrams for explaining a first image and a second image according to a modification of the present invention; FIG. 11B is a fourth example of diagrams for explaining the first image and the second image according to the modified example of the present invention; It is a bus block diagram which connects CPUs etc. which concern on the modification of this invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to embodiment shown below. Moreover, these examples of implementation are merely illustrative, and the present invention can be implemented in various modified and improved forms based on the knowledge of those skilled in the art. In the present specification and drawings, constituent elements with the same reference numerals indicate the same or equivalent elements.

(Embodiment 1)
FIG. 1 shows a schematic diagram of a vehicle 1 equipped with an image processing device according to Embodiment 1 of the present invention. As shown in FIG. 1, a camera 11, an acceleration sensor 12, a display 13, and an image processing device 20 are mounted on a vehicle 1. FIG.

Camera 11 is a front camera mounted on the front of vehicle 1 . The camera 11 photographs the front of the vehicle 1 and generates a moving image including a plurality of frames. The camera 11 supplies the generated moving image to the image processing device 20 on a frame-by-frame basis. In the following description, the moving image frame generated by the camera 11 is referred to as "camera image 11A".

The acceleration sensor 12 is attached to the center of the vehicle 1 or near the center of the vehicle 1 and detects the acceleration of the vehicle 1 in three axial directions of the X-axis, Y-axis and Z-axis as posture information of the vehicle 1 . The X-axis, Y-axis and Z-axis of the vehicle 1 will be described later. The acceleration sensor 12 supplies the detected acceleration in the three axial directions to the image processing device 20 as acceleration information 12A, which is posture information of the vehicle 1 .

The image processing device 20 generates a first image indicating the tilt of the vehicle 1 in the left-right direction and a second image indicating the tilt of the vehicle 1 in the front-rear direction based on the acceleration information 12A. The image processing device 20 combines the first image and the second image with the camera image 11A to generate a composite image 25A. The image processing device 20 supplies the generated composite image 25A to the display 13 .

The display 13 receives the composite image 25A from the image processing device 20 and displays the received composite image 25A.

In FIG. 1, the camera 11, the acceleration sensor 12, the display 13, and the image processing device 20 are shown with solid lines for easy viewing, but they are actually arranged inside the vehicle 1. FIG.

FIG. 2 shows a three-dimensional orthogonal coordinate system defined by the vehicle 1. As shown in FIG. The X-axis, Y-axis and Z-axis are coordinate axes of a three-dimensional orthogonal coordinate system with the center of the vehicle 1 as the origin. In FIG. 2, the X-axis, Y-axis, and Z-axis comply with ISO (International Organization for Standardization) 8855, for example. The X-axis is an axis extending in the longitudinal direction of the vehicle 1 . The Y-axis is an axis extending in the left-right direction of the vehicle 1 . The Z-axis is an axis extending in the vertical direction of the vehicle 1 .

The positive direction of the X-axis is the direction from the driver's seat toward the steering wheel. The positive direction of the Y-axis is the left direction with respect to the front of the vehicle 1 . The positive direction of the Z-axis is above the vehicle 1 as viewed from the driver's seat.

<Configuration of image processing device 20>
FIG. 3 shows a functional block diagram showing the configuration of the image processing device 20. As shown in FIG. As shown in FIG. 3 , the image processing device 20 includes an image acquisition section 21 , an inclination calculation section 22 , an image correction section 28 , an inclination image generation section 26A, and an image synthesis section 25 .

The image acquisition unit 21 acquires the camera image 11A from the camera 11 . The image acquisition unit 21 supplies the acquired camera image 11A to the image correction unit 28 .

The tilt calculator 22 receives acceleration information 12A from the acceleration sensor 12, and calculates a roll angle 22A of the vehicle 1 and a pitch angle 22B of the vehicle 1 from the received acceleration information 12A. The roll angle 22A is the rotation angle of the vehicle 1 with the X axis as the rotation axis. That is, the roll angle 22A indicates the inclination of the vehicle 1 in the left-right direction. The positive direction of the roll angle is the clockwise direction when the X-axis is viewed from the origin in the positive direction. The pitch angle 22B is the rotation angle of the vehicle 1 with the Y axis as the rotation axis. That is, the pitch angle 22B indicates the inclination of the vehicle 1 in the front-rear direction. The positive direction of the pitch angle is the clockwise direction when the Y-axis is viewed from the origin in the positive direction. The tilt calculator 22 supplies the calculated roll angle 22A and pitch angle 22B to the tilt image generator 26A. The tilt calculator 22 also supplies the calculated roll angle 22A to the image corrector 28 .

The tilt image generator 26A receives the roll angle 22A and the pitch angle 22B from the tilt calculator 22 . The tilt image generator 26A uses the received roll angle 22A to generate the first image 100A, and uses the received pitch angle 22B to generate the second image 200A. The tilt image generator 26A includes a first image generator 23 and a second image generator 24 .

The first image generator 23 receives the roll angle 22A from the tilt calculator 22 . The first image generator 23 generates a first image 100A having a first predetermined shape having a display mode according to the received roll angle 22A. That is, the first image 100A is an image showing the tilt of the vehicle 1 in the left-right direction. Details of the first image 100A will be described later. The first image generating section 23 supplies the generated first image 100A to the image synthesizing section 25 .

The second image generator 24 receives the pitch angle 22B from the tilt calculator 22 . The second image generator 24 generates a second image 200A having a second predetermined shape having a display mode according to the received pitch angle 22B. That is, the second image 200A is an image showing the inclination of the vehicle 1 in the front-rear direction. Details of the second image 200A will be described later. The second image generating section 24 supplies the generated second image 200A to the image synthesizing section 25 .

The image correction unit 28 receives the camera image 11A from the image acquisition unit 21 and receives the roll angle 22A from the tilt calculation unit 22 . The image correction unit 28 corrects the lateral tilt of the received camera image 11A using the received roll angle 22A. The camera image 11A corrected for tilt in the horizontal direction corresponds to an image generated by the camera 11 when the roll angle is 0 degrees. The image correction unit 28 supplies the camera image 11A corrected for tilt in the horizontal direction to the image composition unit 25 as the camera image 11B.

The image synthesizing unit 25 receives the camera image 11B from the image correcting unit 28, the first image 100A from the first image generating unit 23, and the second image 200A from the second image generating unit 24. FIG. The image synthesizing unit 25 arranges the first image 100A at the upper end of the camera image 11B and arranges the second image 200A at the right end of the corrected camera image 11B to generate a synthesized image 25A. Details of the composite image 25A will be described later. The image composition unit 25 supplies the generated composite image 25A to the display 13 .

<Generation of synthetic image 25A>
FIG. 4 shows an example of a synthesized image 25A generated by the image synthesizing section 25. As shown in FIG. The composite image 25A has a camera image 11B, a first image 100A, and a second image 200A. In FIG. 4, the display of the subject included in the camera image 11B is omitted.

In the description of FIG. 4, it is assumed that the roll angle 22A is −10 degrees and the pitch angle 22B is 10 degrees.

The first image 100A is a bar-shaped image extending in the horizontal direction of the camera image 11B, and is arranged at the upper end of the camera image 11B. Specifically, the first image 100A is arranged along the upper side of the camera image 11B. The upper side of the camera image 11B is the upper end side of the camera image 11B. The second image 200A is a bar-shaped image extending in the vertical direction of the camera image 11B, and is arranged at the right end of the camera image 11B. Specifically, the second image 200A is arranged along the right side of the camera image 11B. The right side of the camera image 11B is the right end side of the camera image 11B. The first images 100A and 200A are placed over the camera image 11B.

The first image 100A includes a first frame area 101A, a roll angle display area 102A, a first background area 103A, and a first reference line 104A.

The first frame area 101A indicates the maximum area of the first image 100A. The first frame area 101A extends in the horizontal direction of the camera image 11B. That is, the horizontal size of the first frame region 101A is longer than the vertical size of the first frame region 101A. A roll angle display area 102A, a first background area 103A, and a first reference line 104A are arranged inside the first frame area 101A.

The first reference line 104A is a line arranged in the center of the first frame region 101A in the horizontal direction. The position of the first reference line 104A corresponds to the case where the roll angle 22A is 0 degrees.

The first image generator 23 determines the position and color of the roll angle display area 102A based on the roll angle 22A.

First, the position of the roll angle display area 102A will be described. When the roll angle 22A is positive, the first image generator 23 arranges the roll angle display area 102A to the right of the first reference line 104A. When the roll angle 22A is negative, the first image generator 23 arranges the roll angle display area 102A to the left of the first reference line 104A. When the roll angle is 0 degrees, the first image generator 23 does not arrange the roll angle display area 102A inside the first frame area 101A.

Next, the color of the roll angle display area 102A will be described. The first image generator 23 determines the color of the roll angle display area 102A based on the absolute value of the roll angle 22A. For example, when the absolute value of the roll angle 22A is greater than 0 degrees and less than 10 degrees, the first image generator 23 fills the roll angle display area 102A with yellow. When the absolute value of the roll angle 22A is 10 degrees or more and less than 20 degrees, the first image generator 23 fills the roll angle display area 102A with orange. When the absolute value of the roll angle 22A is 20 degrees or more, the first image generator 23 fills the roll angle display area 102A with red.

In the example shown in FIG. 4, the roll angle 22A is -10 degrees, so the roll angle display area 102A is arranged to the left of the first reference line 104A. Also, the roll angle display area 102A is filled with orange.

The first background area 103A is an area inside the first frame area 101A that is separate from the roll angle display area 102A. Specifically, the first background area 103A is arranged on the opposite side of the roll angle display area 102A with respect to the first reference line 104A. The pixel value of the first background area 103A indicates a color different from the color indicating the roll angle 22A. For example, the color of the first background area 103A is fixed and white.

The second image 200A includes a second frame area 201A, a pitch angle display area 202A, a second background area 203A, and a second reference line 204A.

A second frame area 201A indicates the maximum area of the second image 200A. The second frame area 201A extends in the vertical direction of the camera image 11B. That is, the vertical size of the second frame region 201A is longer than the horizontal size of the second frame region 201A. A pitch angle display area 202A, a second background area 203A, and a second reference line 204A are arranged inside the second frame area 201A.

The second reference line 204A is a line arranged in the center of the second frame region 201A in the vertical direction. The position of the second reference line 204A corresponds to the case where the pitch angle 22B is 0 degrees.

The second image generator 24 determines the position and color of the pitch angle display area 202A based on the pitch angle 22B.

First, the position of the pitch angle display area 202A will be described. When the pitch angle 22B is positive, the second image generator 24 arranges the pitch angle display area 202A above the second reference line 204A. When the pitch angle 22B is negative, the second image generator 24 arranges the pitch angle display area 202A below the second reference line 204A. When the pitch angle 22B is 0 degrees, the second image generator 24 does not arrange the pitch angle display area 202A inside the second frame area 202A.

Next, the color of the pitch angle display area 202A will be described. The second image generator 24 determines the color of the pitch angle display area 202A based on the pitch angle 22B. For example, when the absolute value of the pitch angle 22B exceeds 0 degrees and is less than 10 degrees, the second image generator 24 fills the pitch angle display area 202A with yellow. When the absolute value of the pitch angle 22B is 10 degrees or more and less than 20 degrees, the second image generator 24 fills the pitch angle display area 202A with orange. When the absolute value of the pitch angle 22B is 20 degrees or more, the second image generator 24 fills the pitch angle display area 202A with red.

In the example shown in FIG. 4, the pitch angle 22B is 10 degrees, so the pitch angle display area 202A is arranged above the second reference line 204A. The pitch angle display area 202A is filled with orange.

The second background area 203A is an area inside the second frame area 201A that is separate from the pitch angle display area 202A. The second background area 203A is arranged on the opposite side of the pitch angle display area 202A with respect to the second reference line 204A. The pixel value of the second background region 203A indicates a color different from the color indicating the pitch angle 22B. For example, the color of the second background area 203A is fixed and white.

<Operation of image processing device 20>
FIG. 5 shows a flowchart representing the operation of the image processing apparatus 20. As shown in FIG. The image processing method according to this embodiment includes each step shown in this flowchart. The image processing device 20 executes this flowchart when the ignition signal is turned ON. Also, the image processing device 20 periodically executes this flowchart until the ignition signal is turned off. For example, when the frame rate of the camera 11 is 30 fps (frames per second), the image processing device 20 executes this flowchart every 10 frames.

The tilt calculator 22 receives acceleration information 12A from the acceleration sensor 12 . The tilt calculator 22 calculates the roll angle 22A and the pitch angle 22B based on the received acceleration information 12A (step S101). Step S101 corresponds to the tilt calculation step in the image processing method according to this embodiment.

The image acquisition unit 21 acquires the camera image 11A generated by the camera 11 (step S102). Step S102 corresponds to the image acquisition step in the image processing method according to this embodiment. The image acquisition unit 21 supplies the acquired camera image 11A to the image correction unit 28 .

The image correction unit 28 corrects the camera image 11A acquired in step S102 based on the roll angle 22A calculated in step S101 (step S103). The image correction unit 28 supplies the camera image 11B generated in step S103 to the image synthesizing unit 25 .

The first image generator 23 generates the first image 100A based on the roll angle 22A calculated in step S101 (step S104). Step S104 corresponds to the first image generation step in the image processing method according to this embodiment. The first image generating section 23 supplies the generated first image 100A to the image synthesizing section 25 .

The second image generator 24 generates the second image 200A based on the pitch angle 22B calculated in step S101 (step S105). Step S105 corresponds to the second image generating step in the image processing method according to this embodiment. The second image generating section 24 supplies the generated second image 200A to the image synthesizing section 25 .

The image combining unit 25 generates a combined image 25A using the camera image 11B generated in step S103, the first image 100A generated in step S104, and the second image 200A generated in step S105. (Step S106). Step S106 corresponds to the image synthesizing step in the image processing method according to this embodiment.

As described above, the image processing device 20 synthesizes the camera image 11B with the bar-shaped first image 100A and the bar-shaped second image 200A that display the tilt of the vehicle to generate the synthesized image 25A. . Therefore, it is possible to widen the display area of the camera image 11B in the synthesized image 25A. The image processing device 20 allows passengers of the vehicle 1 to easily understand the inclination angle of the vehicle 1 without reducing the visibility of the camera image that captures the exterior of the vehicle 1 .

Furthermore, the image processing device 20 changes the position of the roll angle display area 102A according to the roll angle 22A, and changes the position of the pitch angle display area 202A according to the pitch angle 22B. Therefore, the occupants of the vehicle 1 can easily understand the inclination angle of the vehicle 1 by the colors of the roll angle display area 102A and the pitch angle display area 202A.

(Embodiment 2)
FIG. 6 shows a first image 100B generated by the image processing apparatus according to the second embodiment of the invention, and FIG. 7 shows a second image 200B generated by the image processing apparatus according to the second embodiment of the invention. The configuration of the image processing apparatus according to this embodiment is the same as the configuration of the image processing apparatus 20 according to the first embodiment. Therefore, the image processing apparatus according to the present embodiment will be referred to as "image processing apparatus 20", and detailed description of its configuration will be omitted.

The composite image generated by the image processing apparatus 20 according to this embodiment is the same as the composite image 25A according to the first embodiment except for the first image 100B and the second image 200B. Therefore, in the description of the present embodiment, description of portions other than the first image 100B and the second image 200B will be omitted.

The pixel value of the pixel in this embodiment includes not only the color of the pixel such as RGB but also the transmittance. In the present embodiment, the first image generator 23 adjusts the transmittance of the pixels of the background areas arranged to the left and right of the roll angle display area 102B in the first image 100B to change gradually according to In addition, in the second image 200B, the second image generation unit 24 gradually changes the transmittance of the pixels of the background regions arranged above and below the pitch angle display region 202B as the distance from the pitch angle display region 202B increases. Let A detailed description will be given below.

6 and 7, it is assumed that the roll angle 22A is -10 degrees and the pitch angle 22B is 10 degrees.

First, the first image 100B according to this embodiment will be described with reference to FIG. The first image generator 23 generates the first image 100B according to this embodiment. The first image 100B has a first frame area 101B, a roll angle display area 102B, background areas 103LB and 103RB, and a first reference line 104B.

The position of the roll angle display area 102B will be described. The roll angle display area 102B is arranged inside the first frame area 101B. The roll angle display area 102B has a constant lateral length. The horizontal length of the roll angle display area 102B is the length from the first reference line 104B to the left edge 105B of the first image 100B and the length from the first reference line 104B to the right edge 106B of the first image 100B. shorter than length.

The roll angle display area 102B indicates the roll angle 22A of the vehicle 1 depending on the position where it is arranged. Specifically, the roll angle display area 102B indicates the direction and angle of the lateral tilt of the vehicle 1 depending on the position where it is arranged.

When the roll angle 22A is negative, the first image generator 23 arranges the roll angle display area 102B to the left of the first reference line 104B. Further, the first image generator 23 increases the distance between the roll angle display area 102B and the reference line 104B as the absolute value of the roll angle 22A increases. Further, when the roll angle 22A is positive, the first image generator 23 arranges the roll angle display area 102B to the right of the first reference line 104B. Further, the first image generator 23 increases the distance between the roll angle display area 102B and the reference line 104B as the absolute value of the roll angle 22A increases. When the roll angle 22A is 0 degrees, the first image generator 23 arranges the roll angle display area 102B so as to overlap the first reference line 104B.

In the description of FIG. 6, the roll angle 22A of the vehicle 1 is -10 degrees. Therefore, the roll angle display area 102B is located on the left side of the first reference line 104B, at a position corresponding to 10 degrees.

The color of the roll angle display area 102B will be described. The color of the roll angle display area 102B is blue, for example. The color of the roll angle display area 102B does not change with the roll angle 22A.

The background area 103LB is a left background area arranged to the left of the roll angle display area 102B. The color of the background area 103LB is the same as the color of the roll angle display area 102B. The first image generator 23 gradually changes the pixel values of the pixels included in the background area 103LB as the distance from the roll angle display area 102B increases. For example, the first image generation unit 23 sets the transmittance of the pixels included in the background area 103LB to 0% in the portion in contact with the roll angle display area 102B, and to 62% in the left end 105B of the first image 100B. set.

The background area 103RB is a right background area arranged to the right of the roll angle display area 102B. The color of the background area 103RB is the same as the color of the roll angle display area 102B. The first image generator 23 gradually changes the pixel values of the pixels included in the background region 103RB as the distance from the roll angle display region 102B increases. For example, the first image generation unit 23 sets the transmittance of the pixels of the background region 103RB to 0% in the portion in contact with the roll angle display region 102B, and sets it to 100% in the right end 106B of the first image 100B. .

A second image 200B according to the present embodiment will be described with reference to FIG. The second image generator 24 generates a second image 200B according to this embodiment. The second image 200B has a second frame area 201B, a pitch angle display area 202B, background areas 203TB and 203BB, and a second reference line 204B.

The pitch angle display area 202B is arranged inside the second frame area 201B. The pitch angle display area 202B has a constant vertical length. The vertical length of the pitch angle display area 202B is the length from the second reference line 204B to the upper end 205B of the second image 200B and the length from the second reference line 204B to the lower end 206B of the second image 200B. short.

The pitch angle display area 202B indicates the pitch angle 22B of the vehicle 1 depending on the position where it is arranged. Specifically, the pitch angle display area 202B indicates the direction and angle of inclination in the vertical direction of the vehicle 1 depending on the position where it is arranged.

The second image generator 24 arranges the pitch angle display area 202B below the second reference line 204B when the pitch angle 22B is negative. Furthermore, the second image generator 24 increases the distance between the pitch angle display area 202B and the second reference line 204B as the absolute value of the pitch angle 22B increases. The second image generator 24 arranges the pitch angle display area 202B above the second reference line 204B when the pitch angle 22B is positive. Furthermore, the second image generator 24 increases the distance between the pitch angle display area 202B and the second reference line 204B as the absolute value of the pitch angle 22B increases. When the pitch angle 22B is 0 degree, the second image generator 24 arranges the pitch angle display area 202B so as to overlap the second reference line 204B.

In the description of FIG. 7, the pitch angle 22B of the vehicle 1 is 10 degrees. Therefore, the pitch angle display area 202B is located above the second reference line 204B at a position corresponding to 10 degrees.

The color of the pitch angle display area 202B will be described. The color of the pitch angle display area 202B is blue, for example. The color of the pitch angle display area 202B does not change with the pitch angle 22B.

The background area 203TB is an upper background area arranged above the pitch angle display area 202B. The second image generator 24 gradually changes the pixel values of the pixels included in the background area 203TB as the distance from the pitch angle display area 202B increases. For example, the second image generator 24 sets the transmittance of the pixels of the background region 203TB to 0% in the portion in contact with the pitch angle display region 202B, and sets it to 62% in the upper end 205B of the second image 200B.

The background area 23BB is a lower background area arranged below the pitch angle display area 202 . The second image generation unit 24 gradually changes the pixel values of the pixels included in the background area 203BB as they move away from the pitch angle display area 202B. For example, the second image generator 24 sets the transmittance of the pixels of the background region 203BB to 0% in the portion contacting the pitch angle display region 202B, and sets it to 100% in the lower end 206B of the second image 200B.

In the present embodiment, the first image generator 23 gradually changes the transmittance of the background areas 103LB and 103RB as the distance from the roll angle display area 102B increases. In addition, the second image generator 24 gradually changes the transmittance of the background areas 203TB and 203BB as the distance from the pitch angle display area 202B increases. Therefore, the roll angle display area 102B indicating the roll angle 22A is more conspicuous than the background areas 103LB and 103RB, and the pitch angle display area 202B indicating the pitch angle 22B is more conspicuous than the background areas 203TB and 23BB. As a result, the image processing device 20 according to the present embodiment can allow passengers of the vehicle 1 to easily understand the inclination angle of the vehicle 1 .

In the present embodiment, the first image generator 23 keeps the length of the roll angle display area 102B constant, and arranges the roll angle display area 102B at a position based on the roll angle 22A. Further, the second image generator 24 keeps the length of the pitch angle display area 202B constant, and arranges the pitch angle display area 202B at a position based on the pitch angle 22B. Therefore, the roll angle display area 102B functions as a cursor indicating the roll angle 22A, and the pitch angle display area 202B functions as a cursor indicating the pitch angle 22B. Therefore, the image processing device 20 according to the present embodiment can allow passengers of the vehicle 1 to easily understand the inclination angle of the vehicle 1 .

(Embodiment 3)
FIG. 8 shows a composite image 25C generated by the image processing apparatus according to Embodiment 3 of the present invention. The configuration of the image processing apparatus according to this embodiment is the same as the configuration of the image processing apparatus 20 according to the first embodiment. Therefore, the image processing apparatus according to the present embodiment will be referred to as "image processing apparatus 20", and detailed description of its configuration will be omitted.

A synthesized image 25C generated by the image processing apparatus 20 according to the present embodiment is the same as the synthesized image 25A according to the first embodiment except for the first image 100C and the second image 200C. Therefore, in the description of the present embodiment, description of portions other than the first image 100C and the second image 200C will be omitted.

The first image 100C has a first frame area 101C, a roll angle display area 102C, background areas 103LC and 103RC, and a first reference line 104C.

The position and length of the roll angle display area 102C will be described. In the present embodiment, the first image generator 23 changes the horizontal length of the roll angle display area 102C based on the absolute value of the roll angle 22 . A detailed description will be given below.

The first image generator 23 determines whether to extend the roll angle display area 102C rightward or leftward from the first reference line 104C based on whether the roll angle 22A is positive or negative. When the roll angle 22A is positive, the first image generator 23 aligns the left side of the roll angle display area 102C with the first reference line 104C, and moves the roll angle display area 102C to the right of the first reference line 104C. Decide to extend. When the roll angle 22A is negative, the first image generator 23 aligns the right side of the roll angle display area 102C with the first reference line 104C, and moves the roll angle display area 102C to the left of the first reference line 104C. Decide to extend. When the roll angle 22A is 0 degrees, the first image generator 23 determines not to arrange the roll angle display area 102C inside the first frame area 101C.

The first image generator 23 determines such that the horizontal size of the roll angle display area 102C increases as the absolute value of the roll angle 22A increases. In FIG. 8, the length D11 corresponds to the lateral size when the absolute value of the roll angle 22A is 20 degrees. The length D12 corresponds to the lateral size when the absolute value of the roll angle 22A is 15 degrees. The length D13 corresponds to the lateral size when the absolute value of the roll angle 22A is 10 degrees. Length D11 is longer than length D12. Length D12 is longer than length D13.

The color of the roll angle display area 102C will be described. The color of the roll angle display area 102C is blue, for example, and does not change with the roll angle 22A.

The background area 103LC is a left background area arranged to the left of the roll angle display area 102C. Also, the background area 103RC is a right background area arranged to the right of the roll angle display area 102C. The colors of the background areas 103LC and 103RC are different from the color of the roll angle display area 102C. The color of the background regions 103LC and 103RC is constant regardless of the roll angle 22A, eg white.

The position and length of the pitch angle display area 202C will be described. The second image generator 24 sets the vertical length of the pitch angle display area 202C with reference to the second reference line 204C to a length based on the absolute value of the pitch angle 22B in the second image 200C. The second image generator 24 determines whether to extend the pitch angle display area 202C upward or downward from the second reference line 204C based on whether the pitch angle 22B is positive or negative. When the pitch angle 22B is positive, the second image generator 24 aligns the lower side of the pitch angle display area 202C with the second reference line 204C, and extends the pitch angle display area 202C above the second reference line 204C. and decide. When the pitch angle 22B is negative, the second image generator 24 aligns the upper side of the pitch angle display area 202C with the second reference line 204C, and extends the pitch angle display area 202C below the second reference line 204C. and decide. When the pitch angle 22B is 0 degrees, the second image generator 24 determines not to arrange the pitch angle display area 202C inside the second frame area 201C.

The second image generator 24 determines such that the vertical size of the pitch angle display area 202C increases as the absolute value of the pitch angle 22B increases. In FIG. 8, the length D21 corresponds to the vertical size when the absolute value of the pitch angle 23A is 20 degrees. The length D22 corresponds to the vertical size when the absolute value of the pitch angle 23A is 15 degrees. The length D23 corresponds to the vertical size when the absolute value of the pitch angle 22B is 10 degrees. Length D21 is longer than length D22. Length D22 is longer than length D23.

The background area 203TC is an upper background area arranged above the pitch angle display area 202C. Also, the background area 203BC is a lower background area arranged below the pitch angle display area 202C. The colors of the background areas 203TC and 203BC are different from the color of the pitch angle display area 202C. The color of the background regions 203TC and 203BC is constant regardless of the pitch angle 22B, eg white.

The background area 103LC is a left background area arranged to the left of the roll angle display area 102C. Also, the background area 103RC is a right background area arranged to the right of the roll angle display area 102C. The colors of the background areas 103LC and 103RC are different from the color of the roll angle display area 102C. The color of the background regions 103LC and 103RC is constant regardless of the roll angle, eg white.

As described above, in the present embodiment, the first image generator 23 extends the roll angle display area 102C from the first reference line 104C in the direction based on the roll angle 22A, and the length of the roll angle display area 102C is set to a length based on the absolute value of the roll angle 22A. Further, the second image generator 24 extends the pitch angle display area 202C from the second reference line 204C in the direction based on the pitch angle 22B, and extends the length of the pitch angle display area 202C based on the absolute value of the pitch angle 22B. length. Therefore, a passenger of the vehicle 1 can easily understand the roll angle 22A and the pitch angle 22B from the extending direction and length of the roll angle display area 102C and the pitch angle display area 202C. Therefore, the image processing device 20 according to the present embodiment can allow passengers of the vehicle 1 to easily understand the inclination angle of the vehicle 1 .

(Embodiment 4)
FIG. 9 shows a synthesized image 25D generated by the image processing apparatus according to the fourth embodiment of the invention. The configuration of the image processing apparatus according to this embodiment is the same as the configuration of the image processing apparatus 20 according to the first embodiment. Therefore, the image processing apparatus according to the present embodiment will be referred to as "image processing apparatus 20", and detailed description of its configuration will be omitted.

Synthetic image 25D has camera image 11B, first image 100D, and second image 200D.

The first image 100D has a first frame area 101D, a roll angle display area 102D, background areas 103LD and 103RD, a first reference line 104D, and a first character area 107D. The first frame region 101D, the roll angle display region 102D, and the first reference line 104D are the same as the first frame region 101B, the roll angle display region 102B, and the first reference line 104B shown in FIG. Therefore, description of the first image 100D excluding the background areas 103LD and 103RD and the first character area 107D is omitted.

The second image 200D has a second frame area 201D, a pitch angle display area 202D, background areas 203TD and 203BD, a second reference line 204D, and a second character area 207D. The second frame region 201D, the pitch angle display region 202D, and the second reference line 204D are the same as the second frame region 201B, the pitch angle display region 202B, and the second reference line 204B shown in FIG. Therefore, description of the second image 200D excluding the background areas 203TD and 203BD and the second character area 207D is omitted.

9, the roll angle 22A is assumed to be −10 degrees and the pitch angle 22B is assumed to be 10 degrees.

The first image generator 23 arranges a first character area 107D having characters representing the received roll angle 22A outside and near the first frame area 101D. The shape of the first character area 107D is arbitrary. Since the roll angle 22A in the description of FIG. 9 is -10 degrees, the first image generator 23 places the numerical value "-10" in the first character area 107D.

The transmittance and color of the pixels in the background areas 103LD and 103RD are constant regardless of both the roll angle and the distance from the roll angle display area 102D.

The second image generator 24 arranges a second character area 207D having characters representing the received pitch angle 22B outside and near the second frame area 201D. The shape of the second character area 207D is arbitrary. The pitch angle 22B in the description of FIG. 9 is 10 degrees. Therefore, the second image generator 24 arranges the numerical value “10” in the second character area 207 .

The colors of the background areas 103LD and 103RD are constant regardless of both the roll angle and the distance from the roll angle display area 102D.

In this embodiment, the first image generator 23 arranges the first character area 107D indicating the value of the roll angle 22A near the roll angle display area 102D. Also, the second image generation unit 24 arranges a second character area 207 indicating the value of the pitch angle 22B near the pitch angle display area 202D. Therefore, the passenger of the vehicle 1 can grasp the values of the roll angle 22A and the pitch angle 22B in letters. As a result, the image processing device 20 according to the present embodiment can allow passengers of the vehicle 1 to easily understand the inclination angle of the vehicle 1 .

Further, the image processing apparatus 20 according to the present embodiment simultaneously uses the roll angle display area 102D and the first character area 107D to represent the roll angle 22A, and simultaneously uses the pitch angle display area 202D and the second character area 207 to represent the pitch. It represents corner 22B. Therefore, the image processing device 20 according to the present embodiment allows passengers of the vehicle 1 to understand the inclination angle of the vehicle 1 more easily than when the roll angle display area 102D and the pitch angle display area 202D are used. be able to.

(Embodiment 5)
FIG. 10 shows a synthesized image 25E generated by the image processing apparatus according to Embodiment 5 of the present invention. The configuration of the image processing apparatus according to this embodiment is the same as the configuration of the image processing apparatus 20 according to the first embodiment. Therefore, the image processing apparatus according to the present embodiment will be referred to as "image processing apparatus 20", and detailed description of its configuration will be omitted.

Synthetic image 25E has first image 100E and second image 200A. Synthetic image 25E includes first image 100E instead of first image 100A included in synthetic image 25A. Therefore, in the present embodiment, explanations other than the first image 100E are omitted.

The first image generator 23 rotates the first image 100E having the first predetermined shape at an angle corresponding to the roll angle 22A. Also, part of the first image 100E is not included in the composite image 25E. A detailed description will be given below.

The first image generator 23 first generates a bar-shaped first image 100E extending in the horizontal direction of the camera image 11B. The initial position of the first image is a position where the upper side of the first image 100 contacts the upper side of the camera image 100B. First image generator 23 arranges generated first image 100E at a position where first image 100A shown in FIG. 4 is arranged. The first image 100E has a first frame area 101E, a roll angle display area 102E, and a first reference line 104E. The range of the roll angle display area 102E matches the range of the first frame area 101E. At the time of generating the first image 100E, the position of the first reference line 104E is the same as the position of the first reference line 101A shown in FIG. The color of the roll angle display area 102E is constant regardless of the roll angle 22A. The color of the roll angle display area 102E is yellow, for example.

The first image generator 23 rotates the generated first image 100E around the point 108E according to the roll angle 22A. Point 108E is any point that lies on first reference line 104E. In the example shown in FIG. 10, point 108E is the upper end of first reference line 104E.

The first image 100E is divided into a display area 100E1 and a non-display area 100E2 by being rotated. The display area 100E1 is an area arranged above the camera image 11B in the first image 100E. The non-display area 100E2 is an area arranged above the camera image 11B in the first image 100E. The ratio of the display area 100E1 and the non-display area 100E2 in the first image 100E is determined by the rotation angle of the first image 100E. As shown in FIG. 10, composite image 25E includes display area 100E1 but does not include non-display area 100E2.

In this embodiment, the tilt of the first image 100E changes according to the roll angle 22E. Therefore, a passenger of the vehicle 1 can easily understand the tilt angle of the vehicle 1 from the tilt of the first image 100E. Also, the synthesized image 25E includes the display area 100E1 of the first image 100E, but does not include the non-display area 100E2. Therefore, the passengers of the vehicle 1 can easily understand that the synthesized image 25E does not include the non-display area 100E2. Therefore, the image processing device 20 according to the present embodiment can allow passengers of the vehicle 1 to easily understand the inclination angle of the vehicle 1 .

(Embodiment 6)
FIG. 11 shows a functional block diagram showing the configuration of the image processing apparatus 20F according to this embodiment.

<Configuration of image processing device 20F>
As shown in FIG. 11, the image processing device 20F includes an image acquiring section 21, a tilt calculating section 22, an image correcting section 28, a tilt image generating section 26F, and an image synthesizing section 25. The tilt image generator 26</b>F includes a first image generator 23 and a pitch angle image generator 27 . The image processing device 20</b>F has a configuration in which the second image generation section 24 provided in the image processing device 20 according to the first embodiment is replaced with a pitch angle image generation section 27 . Therefore, the description of the configuration other than the pitch angle image generator 27 is omitted.

The pitch angle image generator 27 receives the pitch angle 22B from the tilt calculator 22 . The pitch angle image generator 27 uses the received pitch angle 22B to generate a pitch angle display image 300F. A pitch angle display image 300F indicates the pitch angle of the vehicle 1 . The pitch angle display image 300F will be described later.

Image synthesizing unit 25 receives camera image 11B from image correcting unit 28 , first image 100F from first image generating unit 23 , and pitch angle display image 300F from pitch angle image generating unit 27 . The image composition unit 25 generates a composite image 25F using the camera image 11B, the first image 100F, and the pitch angle display image 300F.

<Synthetic image 25F>
FIG. 12 shows an example of a synthesized image 25F generated by the image synthesizing section 25F. The composite image 25F has the camera image 11B, the first image 100F, and the pitch angle display image 300F. The first image 100F shown in FIG. 12 is the same as the first image 100A shown in FIG. Therefore, description of the first image 100F in FIG. 12 is omitted.

In the description of FIG. 12, it is assumed that the pitch angle 22B at time T1 is 10 degrees and the pitch angle 22B at time T2 is 0 degrees.

The pitch angle display image 300F has a front end surface image 301F and a rear end surface image 302F. A front end face image 301F corresponds to the front end face of the vehicle 1 . A rear end surface image 302F corresponds to the rear end surface of the vehicle 1 . The rear end surface image 302F is fixed in the pitch angle display image 300F. When the vehicle 1 is viewed from the rear, the rear end surface of the vehicle 1 appears larger than the front end surface. Therefore, the front end surface image 301F and the rear end surface image 302F are made larger than the front end surface image 301F so that the passenger can intuitively understand which is the front end surface image.

The pitch angle image generator 27 receives the pitch angle 22B from the tilt calculator 22 . Based on the received pitch angle 22B, the pitch angle image generator 27 determines the vertical relative positions of the front end surface image 301F and the rear end surface image 302F. Specifically, as the absolute value of the pitch angle 22B increases, the pitch angle image generation unit 27 increases the distance between the vertical center position of the front end surface image 301F and the vertical center position of the rear end surface image 302F. Decide to be longer in the vertical direction. The pitch angle image generator 27 determines whether the front end surface image 301F is positioned above or below the rear end surface image 302F based on the pitch angle 22B.

In the present embodiment, the rear surface image 302F is fixed and the front surface image 301F is moved with reference to the position of the rear surface image 302F. 301F may be used as a reference.

The pitch angle image generator 27 determines that the front end surface image 301F is located above the rear end surface image 302F when the pitch angle 22B is positive. The pitch angle image generator 27 determines that the front end surface image 301F is located below the rear end surface image 302F when the pitch angle 22B is negative.

The pitch angle image generator 27 arranges the rear end surface image 302F at a predetermined position in the pitch angle display image 300F. The predetermined position is the vertical center of the pitch angle display image 300F. The pitch angle image generator 27 arranges the rear end surface image 302F so that the vertical center of the rear end surface image 302F is positioned at the vertical center position of the pitch angle display image 300F.

The pitch angle image generator 27 arranges the front end face image 301F in the pitch angle display image 300F based on the determined relative position. Thereby, a pitch angle display image 300F is generated. The pitch angle image generator 27 supplies the generated pitch angle display image 300F to the image synthesizer 25 .

The image synthesizer 25 arranges the received pitch angle display image 300F at the right end of the camera image 11B to generate a synthesized image 25F. The image synthesizing unit 25 arranges the pitch angle display image 300F at a position where the vertical center of the pitch angle display image 300F coincides with the vertical center of the synthetic image 25F.

In the description of FIG. 12, the pitch angle 22B at time T1 is 10 degrees, and the pitch angle 22B at time T2 is 0 degrees. Therefore, in the synthesized image 25F, the front end image 301F at time T1 is arranged above the rear end image 302F. Further, the front end surface image 301F at time T2 is arranged so as to overlap the rear end surface image 302F.

In this embodiment, the pitch angle image generator 27 changes the relative positions of the front end surface image 301F and the rear end surface image 302F based on the pitch angle 22A. Therefore, a passenger of the vehicle 1 can understand the tilt angle of the vehicle 1 in the front-rear direction from the front end image 301F and the rear end image 302F. The image processing device 20</b>F according to the present embodiment can allow passengers of the vehicle 1 to easily understand the inclination angle of the vehicle 1 .

(Modification 1)
In the above-described embodiment, in the pitch angle display image 300F arranged at the right end of the camera image 11B, the position of the rear end surface image 302F is fixed and the front end surface image 301F is moved. is not limited to Both the leading edge image and the trailing edge image may be moved relative to each other. For example, like the pitch angle display image 300G shown in FIG. 13, the front end surface image 301G and the rear end surface image 302G may move relatively. A pitch angle display image 300G shown in FIG. 13 indicates that the pitch angle 22B at time T1 is 15 degrees and the pitch angle 22B at time T2 is 0 degrees.

In FIG. 13, the image synthesizing unit 25 synthesizes the pitch angle display image 300G so that the vertical center of the pitch angle display image 300G is aligned with the vertical center of the camera image 11B in the synthetic image 25G. Deploy. In addition, the pitch angle image generator 27 arranges the front end surface image 301G and the rear end surface image 302G symmetrically with respect to the vertical center of the pitch angle display image 300G. Since the pitch angle 22B at time T1 is 15 degrees, the pitch angle image generator 27 arranges the front end surface image 301G and the rear end surface image 302G at the positions when the pitch angle 22B is 15 degrees. Also, since the pitch angle 22B at time T2 is 0 degree, the pitch angle image generator 27 arranges the front end surface image 301G and the rear end surface image 302G at the center of the pitch angle display image 300G in the vertical direction.

In this modification, the image processing device 20F displays the pitch angle 22B using a front end surface image 301G and a rear end surface image 302G whose relative positions change according to the pitch angle 22B. Therefore, this modified example allows passengers of the vehicle 1 to easily understand the inclination angle of the vehicle 1 .

(Modification 2)
In the above embodiment, the pitch angle display image 300F is arranged at the right end of the camera image 11B and the first image 100F is arranged at the upper end of the camera image 11B, but the present invention is not limited to this. As shown in FIG. 13, the pitch angle display image 300G may be arranged at the right end of the camera image 11B, and the roll angle display image 400G may be arranged at the upper end of the camera image 11B. In this case, the roll angle display image 400G includes a left end surface image 401G that indicates the roll angle of the left end surface that is the left end surface of the vehicle 1, and a right end surface that indicates the roll angle of the right end surface that is the right end surface of the vehicle 1. and an image 402G.

The first image generator 23 determines the lateral position of the right end surface image 402G to be the lateral center of the roll angle display image 400G. Based on the roll angle 22A, the first image generator 23 determines whether the left edge image 401G is positioned to the left or right of the right edge image 402G. When the roll angle 22A is positive, the first image generator 23 determines that the left edge image 401G is positioned to the right of the right edge image 402G. When the roll angle 22A is negative, the first image generator 23 determines that the left edge image 401G is positioned to the left of the right edge image 402G.

The first image generation unit 23 generates the roll angle display image 400G by arranging the right end surface image 402G and the left end surface image 401G at the determined positions. The image synthesizing unit 25 arranges the roll angle display image 400G and the camera image 11B so that the horizontal center of the roll angle display image 400G and the horizontal center of the camera image 11B are aligned, and creates a combined image. Generate 25G.

In this modification, the roll angle 22A is displayed using a right end image 402G and a left end image 401G whose relative positions change according to the roll angle 22A. Therefore, this modified example allows passengers of the vehicle 1 to easily understand the inclination angle of the vehicle 1 .

(Modification 3)
In the above embodiment, an example in which the bar-shaped roll angle display area 102A is arranged at the upper end of the camera image 11B and the bar-shaped pitch angle display area 202A is arranged at the right end of the camera image 11B to generate the combined image 25A. , the invention is not limited to this. As shown in FIG. 14, a triangular pitch angle display area 202H may be arranged at the right end of the camera image 11B to generate a synthesized image 25H. In the case shown in FIG. 14, the pitch angle display area 202H represents the inclination of the vehicle 1 in the front-rear direction. In this case, the vehicle 1 is tilted forward and the pitch angle 22B is positive. In other words, the vehicle 1 is tilted toward the acute angle of the pitch angle display area 202H.

In this modification, a pitch angle display area 202H having an acute angle is used to represent the pitch angle 22B. For this reason, this modification can make the passengers of the vehicle 1 easily understand the inclination angle of the vehicle 1 .

(Modification 4)
In the above embodiment, an example in which the bar-shaped first image 100A is arranged at the upper end of the camera image 11B and the bar-shaped second image 200A is arranged at the right end of the camera image 11B to generate the combined image 25A will be described. However, the invention is not limited to this. As shown in FIG. 15, bar-shaped images may be displayed on all of the upper end, right end, lower end, and left end of the camera image 11B.

In the case shown in FIG. 15, a bar-shaped first image 100J is arranged at the upper end of the camera image 11B, a bar-shaped second image 200J is arranged at the right end of the camera image 11B, and a bar-shaped image 200J is arranged at the lower end of the camera image 11B. A shape third image 500J is arranged, and a bar-shaped fourth image 600J is arranged at the left end of the camera image 11B to generate a composite image 25J. In this case, for example, the image arranged in the direction in which the vehicle 1 is tilted is filled with a color corresponding to the tilt, and the colors of the images in other directions are transparent or white.

In this modified example, the inclination angle of the vehicle 1 is represented using a first image 100J, a second image 200J, a third image 500J, and a fourth image 600J. For this reason, this modification can make the passengers of the vehicle 1 easily understand the inclination angle of the vehicle 1 .

(Modification 5)
In the above embodiment, the transmittance is used as the pixel value of the background region to be changed, and the transmittance of the background region is gradually changed as the distance from the roll angle display region 102B or the pitch angle display region 202B is increased. The invention is not so limited. Color may be used as the pixel value of the background region to be changed. In this case, the color of the left background area and the color of the right background area are gradually changed with increasing distance from the roll angle display area. In addition, the color of the upper background area and the color of the lower background area are gradually changed with increasing distance from the pitch angle display area. Also, luminance may be used as the pixel value of the background area to be changed.

(Modification 6)
In the above embodiment, the image processing device 20 acquires the camera image 11B from the camera 11 mounted on the vehicle 1, and displays the generated composite image on the display 13 mounted on the vehicle 1. The invention is not so limited. The image processing device 20 may, for example, acquire a camera image from a camera of a device such as a smart phone that can be carried out from the vehicle 1, and display a synthesized image on the display of the device such as the smart phone. In addition, the image processing device 20 acquires a camera image from a camera of a device such as a smart phone that can be carried out from the vehicle 1, and displays a composite image on both the display of the device such as the smart phone and the display 13 mounted on the vehicle 1. may be displayed.

In this modification, the image processing device 20 acquires an image from a camera of a device such as a smart phone, and displays a composite image on a display of the device such as a smart phone. Therefore, the synthesized image can be displayed on the familiar screen of the passenger. Therefore, passengers of the vehicle 1 can easily understand the inclination angle of the vehicle 1 .

Also, in this modification, the image processing device 20 displays the composite image on both the display of the device such as the smartphone and the display 13 mounted on the vehicle 1 . Therefore, the image processing device 20 can display the composite image at a position that is easy for the passenger to see, compared to the case where the composite image is displayed only on the display 13 mounted on the vehicle 1 .

(Modification 7)
In the above embodiment, the first character area 107D is arranged outside the first frame area 101D and near the first frame area 101D, and the second character area 207D is arranged outside the second frame area 201D and inside the second frame area 201D. Although an example of arranging them close to each other has been described, the present invention is not limited to this. The first character area 107D may be arranged inside the first frame area 101D. Also, the second frame region 201D may be arranged inside the second frame region 201D.

(Modification 8)
In the above embodiment, an example in which the synthesized image includes the bar-shaped first image and the bar-shaped second image has been described, but the present invention is not limited to this. The shapes of the first image and the second image included in the synthesized image are arbitrary. For example, as shown in FIG. 16, the synthesized image 25K includes a first image including a left triangle image 401K and a right triangle image 402K, and a first image including a front triangle image 301K and a rear triangle image 302K. 2 images. Further, the first image may include the first character area 107K and the second image may include the second character area 207K.

In the case shown in FIG. 16, the first image generator 23 colors the left end image 401K or the right end image 402K according to the sign of the roll angle 22A. For example, when the roll angle 22A is -10 degrees, the first image generator 23 colors the left end image 401K. The color at the time of coloring is a color according to the absolute value of the roll angle 22A. The first image generation unit 23 colors yellow when the absolute value of the roll angle 22A is greater than 0 degrees and less than 10 degrees, and orange when the absolute value of the roll angle 22A is 10 degrees or more and less than 20 degrees. , and when the absolute value of the roll angle 22A is 20 degrees or more, it is colored red. The first image generation unit 23 sets the color of the uncolored side of the left end image 401K or the right end image 402K to white.

Further, the second image generator 24 colors the front end surface image 301K or the rear end surface image 302K according to the sign of the pitch angle 22B. For example, when the pitch angle 22B is 10 degrees, the second image generator 24 colors the upper end surface image 301K. The color when coloring is a color according to the absolute value of the pitch angle 22B. The second image generation unit 24 colors yellow when the absolute value of the pitch angle 22B is greater than 0 degree and less than 10 degrees, and orange when the absolute value of the pitch angle 22B is 10 degrees or more and less than 20 degrees. , and when the absolute value of the pitch angle 22B is 20 degrees or more, it is colored red. The second image generator 24 sets the color of the uncolored side of the front end surface image 301K or the rear end surface image 302K to white.

The first image generator 23 arranges the first character area 107K near the colored side of the left end image 401K or the right end image 402K. The first character area 107K has characters indicating the roll angle 22A.

The second image generator 24 arranges the second character area 207K near the colored side of the front end face image 301K or the rear end face image 302K. The second character area 207K has characters indicating the pitch angle 22B.

In this modification, a triangle left edge image 401K, a triangle right edge image 402K, a triangle front edge image 301K, a triangle rear edge image 302K, a first character area 107K, and a second character area 207K. is used to represent the tilt angle of the vehicle 1 . For this reason, this modification can make the passengers of the vehicle 1 easily understand the inclination angle of the vehicle 1 .

(Modification 9)
Further, in the above embodiment, the image processing device 20 may be individually integrated into one chip by a semiconductor device such as LSI (Large Scale Integration), or may be integrated into one chip so as to include part or all of it. good. Although LSI is used here, it may also be called IC (Integrated Circuit), system LSI, super LSI, or ultra LSI, depending on the degree of integration.

The method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure connections and settings of circuit cells inside the LSI may be used.

Also, part or all of the processing executed by the image processing device 20 may be implemented by a program. Part or all of the processing of each functional block in each of the above embodiments is performed by a central processing unit (CPU) in a computer. A program for performing each process is stored in a storage device such as a hard disk or ROM, and is read from the ROM or RAM and executed.

Further, each process of the above embodiment may be realized by hardware, or may be realized by software (including cases where it is realized together with an OS (operating system), middleware, or a predetermined library). . Furthermore, it may be realized by mixed processing of software and hardware.

For example, when the image processing apparatus 20 is implemented by software, the hardware configuration shown in FIG. , each functional unit may be realized by software processing.

Also, the execution order of the processing methods in the above embodiments is not limited to the description of the above embodiments, and the execution order may be changed without departing from the spirit of the invention.

A computer program that causes a computer to execute the method described above and a computer-readable recording medium that records the program are included in the scope of the present invention. Examples of computer-readable recording media include flexible disks, hard disks, CD-ROMs, MOs, DVDs, DVD-ROMs, DVD-RAMs, large-capacity DVDs, next-generation DVDs, and semiconductor memories. .

Although the embodiments of the present invention have been described above, the above-described embodiments are merely examples for carrying out the present invention. Therefore, the present invention is not limited to the above-described embodiment, and can be implemented by appropriately modifying the above-described embodiment without departing from the scope of the invention.

11: Camera 12: Acceleration sensor 12A: Acceleration information 13: Display 20: Image processing device 21: Image acquisition unit 22: Tilt calculation unit 22A: Roll angle 22B: Pitch angle 23: First image generation unit 24: Second image generation Unit 25: Image synthesizing unit 25A: Synthetic image 26A: Tilt image generating unit 27: Pitch angle image generating unit 28: Image correcting unit

Claims (11)

a tilt calculation unit that calculates a roll angle and a pitch angle of the vehicle from outputs of sensors that detect posture information of the vehicle;
an image acquisition unit that acquires a camera image generated by a camera;
a first image generation unit that generates a first image having a first predetermined shape having a display mode according to the roll angle;
a second image generating unit that generates a second image of a second predetermined shape having a display mode according to the pitch angle;
An image generated by arranging the first image at the upper end or the lower end of the camera image acquired by the image acquisition unit and arranging the second image at the left end or right end of the camera image to generate a composite image. a synthesizer;
with
The first image is
a first frame region corresponding to the first predetermined shape;
a roll angle display area arranged inside the first frame area and indicating the roll angle;
a first background area that is arranged inside the first frame area and that is a separate area from the roll angle display area;
The first image generation unit determines the position of the roll angle display area based on the roll angle, determines the pixel value in the first background area to be a pixel value different from the pixel value in the roll angle display area,
The second image is
a second frame region corresponding to the second predetermined shape;
a pitch angle display area arranged inside the second frame area and indicating the pitch angle;
a second background area that is arranged inside the second frame area and is a separate area from the pitch angle display area;
The second image generation unit determines the position of the pitch angle display area based on the pitch angle, and determines the pixel value in the second background area to be a pixel value different from the pixel value in the pitch angle display area.
Image processing device. The first image generation unit determines a color of the roll angle display area based on the roll angle,
The second image generation unit determines a color of the pitch angle display area based on the pitch angle.
The image processing apparatus according to claim 1 . the first image is generated extending in a horizontal direction;
The first image generation unit determines whether or not to arrange the roll angle display area to the right of a reference line indicating the roll angle of 0 degree based on the roll angle,
The second image is formed extending in the vertical direction,
The second image generation unit determines whether or not to place the pitch angle display area above a reference line indicating that the pitch angle is 0 degrees, based on the pitch angle.
The image processing apparatus according to claim 2 . the first image is generated extending in a horizontal direction;
The first background area is
a left background area arranged to the left of the roll angle display area;
a right background area arranged to the right of the roll angle display area;
including
The first image generation unit gradually changes the pixel values of the left background area and the right background area as the distance from the roll angle display area increases,
the second image is generated extending in the vertical direction;
The second background area is
an upper background area arranged above the pitch angle display area;
a lower background area arranged below the pitch angle display area;
including
The second image generation unit gradually changes the pixel values of the upper background area and the right background area as the distance from the pitch angle display area increases.
The image processing apparatus according to claim 1 . the first image is generated extending in a horizontal direction;
The first image generator determines a horizontal length of the roll angle display area based on the roll angle, and extends the roll angle display area rightward from a reference line indicating the roll angle of 0 degree. determining whether to arrange the camera so as to extend to the left based on the roll angle;
The second image is formed extending in the vertical direction,
The second image generation unit determines a vertical length of the pitch angle display area based on the pitch angle, and extends the pitch angle display area upward from a reference line indicating the pitch angle of 0 degrees. Based on the pitch angle, determine whether to place the
The image processing apparatus according to claim 1 . The roll angle display area has a certain length and is arranged at a position based on the roll angle,
The pitch angle display area has a certain length and is arranged at a position based on the pitch angle.
The image processing apparatus according to claim 1 . The first image generation unit arranges characters representing the roll angle in the vicinity of the first frame area,
The second image generation unit arranges characters representing the pitch angle in the vicinity of the second frame area.
The image processing apparatus according to any one of claims 1 to 6 . a tilt calculation unit that calculates a roll angle and a pitch angle of the vehicle from outputs of sensors that detect posture information of the vehicle;
an image acquisition unit that acquires a camera image generated by a camera;
a first image generation unit that generates a first image having a first predetermined shape having a display mode according to the roll angle;
a second image generating unit that generates a second image of a second predetermined shape having a display mode according to the pitch angle;
An image generated by arranging the first image at the upper end or the lower end of the camera image acquired by the image acquisition unit and arranging the second image at the left end or right end of the camera image to generate a composite image. a synthesizer;
with
the first image is generated extending in a horizontal direction;
The first image generator rotates the first image extending in the horizontal direction at an angle corresponding to the roll angle .
Image processing device. a tilt calculation unit that calculates a pitch angle of the vehicle from an output of a sensor that detects posture information of the vehicle;
an image acquisition unit that acquires a camera image generated by a camera;
a relative position between a front end surface image corresponding to the front end surface of the vehicle and a rear end surface image corresponding to the rear end surface of the vehicle is determined based on the pitch angle, and the front end surface image arranged at the determined relative position; a pitch angle image generation unit that generates a pitch angle display image including the rear end surface image;
an image synthesizing unit for synthesizing the camera image acquired by the image acquiring unit and the pitch angle display image to generate a synthetic image;
with
The rear end surface image is set to be displayed larger than the front end surface image,
Image processing device. an inclination calculation step of calculating a roll angle of the vehicle and a pitch angle of the vehicle from an output of a sensor that detects posture information of the vehicle;
an image acquisition step of acquiring a camera image produced by a camera;
a first image generating step of generating a first image having a first predetermined shape having a display mode according to the roll angle;
a second image generating step of generating a second image having a second predetermined shape having a display mode according to the pitch angle;
An image generated by arranging the first image at the upper end or the lower end of the camera image acquired by the image acquisition step and arranging the second image at the left end or right end of the camera image to generate a composite image. a synthetic process;
with
The first image is
a first frame region corresponding to the first predetermined shape;
a roll angle display area arranged inside the first frame area and indicating the roll angle;
a first background area that is arranged inside the first frame area and that is a separate area from the roll angle display area;
In the first image generating step, the position of the roll angle display area is determined based on the roll angle, the pixel value in the first background area is determined to be a pixel value different from the pixel value in the roll angle display area,
The second image is
a second frame region corresponding to the second predetermined shape;
a pitch angle display area arranged inside the second frame area and indicating the pitch angle;
a second background area that is arranged inside the second frame area and is a separate area from the pitch angle display area;
In the second image generating step, the position of the pitch angle display area is determined based on the pitch angle, and the pixel value in the second background area is determined to be a pixel value different from the pixel value in the pitch angle display area.
Image processing method. an inclination calculation step of calculating a roll angle of the vehicle and a pitch angle of the vehicle from an output of a sensor that detects posture information of the vehicle;
an image acquisition step of acquiring a camera image produced by a camera;
a first image generating step of generating a first image having a first predetermined shape having a display mode according to the roll angle;
a second image generating step of generating a second image having a second predetermined shape having a display mode according to the pitch angle;
An image generated by arranging the first image at the upper end or the lower end of the camera image acquired by the image acquisition step and arranging the second image at the left end or right end of the camera image to generate a composite image. a synthetic process;
with
the first image is generated extending in a horizontal direction;
In the first image generating step, the first image extending in the horizontal direction is rotated at an angle corresponding to the roll angle.
Image processing method.

Images