WO2017158829A1

WO2017158829A1 - Display control device and display control method

Info

Publication number: WO2017158829A1
Application number: PCT/JP2016/058749
Authority: WO
Inventors: 聖崇加藤; 篤史前田
Original assignee: 三菱電機株式会社
Priority date: 2016-03-18
Filing date: 2016-03-18
Publication date: 2017-09-21

Abstract

The present invention is provided with: a distance image generation unit (5) for using a captured image in which a blind spot is captured where the field of vision of a driver is obstructed by a structure of a vehicle to generate a distance image the pixel value of which is represented by a value of the distance from the position of the viewpoint of the driver to an object located in the blind spot; a coordinate conversion unit (6) for using the generated distance image to set a plurality of virtual projection spheres in an area around the vehicle that are defined as the outer circumferential surfaces of a globe centering around the position of the viewpoint of the driver, and using the plurality of virtual projection spheres to convert coordinates on a display screen for displaying the captured image into coordinates on the image-capture surface of the captured image; and a display image generation unit (7) for using the converted coordinates on the image-capture surface to generate an image displayed on the display screen.

Description

Display control apparatus and display control method

The present invention relates to a technique for displaying an image of an area where a driver's view is blocked.

Conventionally, by displaying an image obtained by capturing an area where the driver's view is blocked by the vehicle structure on a display or the like installed in the vehicle structure, the situation of the area where the driver's view is blocked is displayed. Has been proposed to assist in the visual confirmation. Also, a technique has been proposed in which when a captured image is displayed on a display or the like, the captured image is displayed to some extent continuously with the actual scenery outside the vehicle viewed from the viewpoint of the driver. For example, a blind spot image display device for a vehicle described in Patent Document 1 defines a virtual projection spherical surface having a radius from a driver's viewpoint position to a subject as a radius, and each pixel of the display using the driver's viewpoint position as a viewpoint. The intersection of the straight line passing through the virtual projection sphere and the virtual projection spherical surface is obtained, the image of the image captured by the camera corresponding to the obtained intersection is specified, and an image to be displayed on the display is generated.

JP 2007-104538 A

In the device described in Patent Document 1, since the radius of the virtual projection spherical surface is defined using the distance from the driver's viewpoint position to one subject, the area in which the driver's field of view is blocked by the vehicle structure unit When there are a plurality of subjects, there exists a subject whose radius from the defined virtual projection spherical surface does not match the distance from the viewpoint position of the driver. An image obtained by capturing an object in which the radius of the virtual projection spherical surface and the distance from the driver's viewpoint position do not match has a problem that the continuity with the actual scenery outside the vehicle viewed from the driver's viewpoint position is reduced.

The present invention was made to solve the above-described problems, and even when there are a plurality of subjects in an area where the driver's field of view is blocked, an image of the area where the driver's field of view is blocked is captured. The purpose is to display the image continuously with the scenery outside the vehicle viewed from the viewpoint position of the driver.

The display control apparatus according to the present invention uses a captured image obtained by capturing a blind spot area in which a driver's field of view is blocked by a structure of the vehicle in a peripheral area of the vehicle, from a driver's viewpoint position to a blind spot. A distance image generation unit that generates a distance image using a value of a distance to an object located in the region as a pixel value, and a driver's viewpoint position in a region around the vehicle using the distance image generated by the distance image generation unit A plurality of virtual projection spheres defined as outer peripheral surfaces of a sphere centered on the image are set, and the coordinates on the display screen for displaying the captured image are displayed on the imaging surface of the captured image using the plurality of virtual projection spheres. And a display image generation unit that generates an image to be displayed on the display screen using the coordinates on the imaging surface converted by the coordinate conversion unit.

According to the present invention, an image obtained by capturing an area where the driver's view is blocked can be displayed continuously with the scenery outside the vehicle viewed from the viewpoint position of the driver.

1 is a block diagram illustrating a configuration of a display control device according to a first embodiment. 2 is a diagram illustrating a hardware configuration of a display control apparatus according to Embodiment 1. FIG. It is the figure which looked at the vehicle provided with the display control apparatus which concerns on Embodiment 1 from upper direction. It is the figure which looked at the part in the vehicle provided with the display control apparatus which concerns on Embodiment 1 from the rear seat. It is a figure which shows the scenery which a driver | operator visually recognizes when looking directly outside the vehicle from a driver | operator's viewpoint position. 3 is a flowchart showing an operation of the display control apparatus according to the first embodiment. 6 is a diagram illustrating an example of a captured image received by an image input unit of the display control apparatus according to Embodiment 1. FIG. 6 is a diagram illustrating an example of a distance image generated by a distance image generation unit of the display control apparatus according to Embodiment 1. FIG. 6 is a diagram illustrating a display example of a display image generated by a display image generation unit of the display control apparatus according to Embodiment 1. FIG. 6 is a diagram illustrating a display example of a display image generated by a display image generation unit of the display control apparatus according to Embodiment 1. FIG. 6 is a diagram illustrating a display example of a display image generated by a display image generation unit of the display control apparatus according to Embodiment 1. FIG. 4 is a flowchart illustrating an operation of a coordinate conversion unit of the display control device according to the first embodiment. FIG. 6 is a diagram schematically illustrating coordinate conversion processing of a coordinate conversion unit of the display control device according to the first embodiment. 4 is a flowchart illustrating an operation of a display image generation unit of the display control apparatus according to the first embodiment. 6 is a diagram showing a display image generated by a display image generation unit of the display control apparatus according to Embodiment 1. FIG. 6 is a diagram showing a display result of the display control apparatus according to Embodiment 1. FIG. 6 is a block diagram illustrating a configuration of a display control device according to Embodiment 2. FIG. 10 is a flowchart illustrating an operation of a display image generation unit of the display control apparatus according to the second embodiment. FIG. 10 is a diagram illustrating an example of setting conditions of the display control device according to the second embodiment. 20A and 20B are diagrams illustrating an example of setting conditions of the display control device according to the second embodiment.

Hereinafter, in order to explain the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram illustrating a configuration of a display control apparatus 10 according to the first embodiment.
The display control device 10 includes an image input unit 1, a distance information acquisition unit 2, a viewpoint information acquisition unit 3, an image processing unit 4, and a display control unit 8. Further, the image processing unit 4 includes a distance image generation unit 5, a coordinate conversion unit 6, and a display image generation unit 7. The display control device 10 is mounted on, for example, a vehicle 20 described later.

The image input unit 1 receives an input of a captured image obtained by imaging an area around the vehicle 20 by an imaging unit such as a camera. The captured image is an image obtained by capturing an area where the driver's field of view is blocked by at least a structure of the vehicle 20 such as a pillar (hereinafter referred to as a blind spot area) among the areas around the vehicle 20.
The distance information acquisition unit 2 acquires distance information that is a result of measuring a distance from the sensor to an object in the area by scanning at least a blind spot area by a sensor or the like mounted on the vehicle 20.

The viewpoint information acquisition unit 3 acquires viewpoint information indicating the driver's viewpoint position. The viewpoint position of the driver is, for example, the position of the driver's eyes or the position of the head.
The captured image received by the image input unit 1, the information acquired by the distance information acquisition unit 2 and the viewpoint information acquisition unit 3 are output to the image processing unit 4.

In the image processing unit 4, position information indicating the arrangement position of the imaging means, position information indicating the arrangement position of the distance measuring means, and position information indicating the arrangement position of the display described later are set in advance.
The distance image generation unit 5 of the image processing unit 4 uses at least the blind spot from the driver's viewpoint position using the distance information acquired by the distance information acquisition unit 2 and the driver's viewpoint information acquired by the viewpoint information acquisition unit 3. The distance to the object located in the area is calculated. The distance image generation unit 5 further refers to the position information of the image pickup unit and the position information of the distance measurement unit, for each pixel of the picked-up image received by the image input unit 1 from the viewpoint position of the driver. A distance image is generated in which the distance values up to (hereinafter referred to as distance values) are associated as pixel values.

The distance image generation unit 5 performs a process of generating a single same target area by combining the areas having the same distance from the viewpoint position of the driver among the areas where the distance is measured, and the distance of the same target area The subject area and the background area are set according to the above. For example, when the distance value of the generated same target area is less than the threshold, the distance image generation unit 5 sets the same target area as the subject area, and when the generated distance value of the same target area is greater than or equal to the threshold The same target area is set as the background area.

The coordinate conversion unit 6 refers to the same target region generated by the distance image generation unit 5 and the distance information of the distance image, the subject region set from the driver's viewpoint position with the driver's viewpoint position as the center. A virtual projection spherical surface defined as the outer peripheral surface of the sphere with the radius up to is set. Similarly, the coordinate conversion unit 6 sets a virtual projection spherical surface defined as an outer peripheral surface of a sphere centered on the driver's viewpoint position and having a radius from the driver's viewpoint position to the set background area as a radius. To do.
The coordinate conversion unit 6 converts coordinates on the display (display screen) in the vehicle viewed from the viewpoint position of the driver into coordinates on the imaging surface of the captured image using the set virtual projection spherical surface. Thereby, the image data displayed on the display in the vehicle is specified by the coordinates of the imaging surface, that is, the pixels. Here, the display in the vehicle is a display arranged on a structure of the vehicle 20 such as a pillar that generates a blind spot area, and is a display for displaying an image of the blind spot area.
Details of the processing of the coordinate conversion unit 6 will be described later.

The display image generation unit 7 generates image data for each virtual projection spherical surface set by the coordinate conversion unit 6 using the coordinates converted by the coordinate conversion unit 6, that is, the coordinates on the imaging surface of the captured image.
The display image generation unit 7 refers to the distance value of the distance image generated by the distance image generation unit 5 and selects image data of the virtual projection spherical surface used for each of the subject region and the background region.
The display image generation unit 7 generates display image data from the selected image data based on a preset display position and display size.
Further, the display image generation unit 7 may have functions such as processing, synthesis, and graphics superimposition of the generated display image data. For example, the display image generation unit 7 may convert the display image data so that a menu screen or an alert screen is superimposed on the display image.

The display control unit 8 generates display control information for displaying an image based on the display image data generated by the display image generation unit 7 on a display or the like, and outputs the display control information to the display or the like.

Next, a hardware configuration example of the display control apparatus 10 will be described.
FIG. 2 is a diagram illustrating a hardware configuration example of the display control apparatus 10.
The image input unit 1, the distance information acquisition unit 2, the viewpoint information acquisition unit 3, the distance image generation unit 5, the coordinate conversion unit 6, the display image generation unit 7, and the display control unit 8 in the display control device 10 are realized by a processing circuit. The That is, the distance image generation unit 5, the coordinate conversion unit 6, the display image generation unit 7, and the display control unit 8 perform image processing of the captured image using the input captured image, distance information, and viewpoint information, and are generated. A processing circuit for performing display control of the displayed image.
When the processing circuit is dedicated hardware, the processing circuit is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-programmable Gate Array). ) Or a combination of these. The functions of the image input unit 1, distance information acquisition unit 2, viewpoint information acquisition unit 3, distance image generation unit 5, coordinate conversion unit 6, display image generation unit 7 and display control unit 8 are realized by processing circuits. Alternatively, the functions of the respective units may be combined and realized by a processing circuit.

When the processing circuit is a CPU (Central Processing Unit), the processing circuit is the CPU 12 that executes a program stored in the memory 13 shown in FIG.
The functions of the image input unit 1, distance information acquisition unit 2, viewpoint information acquisition unit 3, distance image generation unit 5, coordinate conversion unit 6, display image generation unit 7 and display control unit 8 are software, firmware, or software and firmware. It is realized by the combination. Software or firmware is described as a program and stored in the memory 13. The CPU 12 reads out and executes a program stored in the memory 13 to thereby execute an image input unit 1, a distance information acquisition unit 2, a viewpoint information acquisition unit 3, a distance image generation unit 5, a coordinate conversion unit 6, and a display image generation unit. 7 and the display control unit 8 are realized. That is, when the distance image generation unit 5, the coordinate conversion unit 6, the display image generation unit 7, and the display control unit 8 are executed by the CPU 12, each step shown in FIG. A memory 13 for storing the program. In addition, these programs include procedures or methods of the image input unit 1, the distance information acquisition unit 2, the viewpoint information acquisition unit 3, the distance image generation unit 5, the coordinate conversion unit 6, the display image generation unit 7, and the display control unit 8. It can also be said to be executed by a computer.

Here, the CPU 12 is, for example, a central processing unit, a processing unit, an arithmetic unit, a processor, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor).
The memory 13 may be, for example, a nonvolatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically EPROM). Further, it may be a magnetic disk such as a hard disk or a flexible disk, or an optical disk such as a mini disk, CD (Compact Disc), or DVD (Digital Versatile Disc).

Next, information input to the image input unit 1, the distance information acquisition unit 2 and the viewpoint information acquisition unit 3, and information output from the display control unit 8 will be described with reference to the explanatory diagrams of FIGS. 3 and 4. .
FIG. 3 is a view of the vehicle 20 including the display control device 10 according to the first embodiment as viewed from above. FIG. 4 is a view of a part of the vehicle 20 provided with the display control device 10 according to the first embodiment as seen from the rear seat.
The driver A of the vehicle 20 is blocked from view by a front pillar (so-called A pillar) 21 on the left side of the vehicle 20, for example. Therefore, when the driver A visually recognizes the front pillar 21 side from the seating position, the driver A cannot visually recognize the area located behind the front pillar 21 when viewed from the driver A, and the blind spot area 101 is generated. .
In this case, information input to the display control device 10 will be described.

The viewpoint information acquired by the viewpoint information acquisition unit 3 is calculated using a captured image obtained by capturing the interior of the vehicle by the in-vehicle camera 33 or the like mounted on the vehicle 20. In the example of FIGS. 3 and 4, the in-vehicle camera 33 arranged in front of the vehicle 20 captures the inside of the vehicle to acquire the captured image, and a calculation unit (not illustrated) calculates the face image of the driver A included in the captured image. The viewpoint position 105 is calculated by analysis. The viewpoint information can be acquired by using a known detection technique such as triangulation using captured images captured by a plurality of cameras or TOF (Time of Flight) using a monocular camera. The viewpoint information may be calculated based on the physical information of the driver A registered in advance, or may be calculated and acquired based on the seat position of the driver's seat, the angle of the rearview mirror, and the angle of the side mirror. Also good.
The viewpoint position 105 indicated by the viewpoint information is set as a reference position for calculating the distance from the driver A to the object.

The captured image input to the image input unit 1 is captured by the vehicle exterior camera 31 a mounted on the vehicle 20. The vehicle outside camera 31a is an imaging unit that images at least a blind spot area in which the driver's field of view is blocked by a structure of the vehicle 20, such as a pillar, in the peripheral area of the vehicle 20.
In FIG. 3, the outside camera 31 a is installed toward the left front of the vehicle 20, for example, near the side mirror 22. The vehicle exterior camera 31a images an object or the like existing in the imaging range 102 that is an area around the vehicle 20 including at least the blind spot area 101. As long as at least the blind spot area 101 can be imaged, the vehicle exterior camera 31a may be installed on the root portion of the front pillar 21 (not shown), the roof (not shown), the vehicle interior window (not shown), etc. in addition to the vicinity of the side mirror 22. The installation position is not limited. Further, a plurality of outside cameras 31a may be arranged. Furthermore, the vehicle exterior camera 31a may be configured by a single camera having a wide-angle lens, and the vehicle exterior camera 31a may be used for imaging both the left and right blind spot areas of the vehicle 20.

The vehicle exterior camera 31a is a camera that captures a color image, and includes an optical mechanism including a lens and a mirror, and an image sensor such as a CCD (Charge-Coupled Device) or CMOS (Complementary-Metal-Oxide Semiconductor) image sensor. Yes. In addition, the outside camera 31a may include an infrared sensor or a light emitting element in order to enable night imaging. The raw image data generated by the image sensor of the outside camera 31a is subjected to preprocessing such as color conversion processing, format conversion processing, filtering processing, and the like as necessary, and then the captured image To the image input unit 1.

The distance information acquired by the distance information acquisition unit 2 is generated by the distance measuring sensor 32a mounted on the vehicle 20. In FIG. 3, the distance measuring sensor 32 a is disposed, for example, near the side mirror 22 toward the left front of the vehicle 20. In the example of FIG. 3, the distance measurement sensor 32 a measures the distance 103 to the object B and the distance 104 to the object C. In practice, the distance measuring sensor 32a is not limited to the distance to the object B and the object C, but has a resolution range in which the distance to all the objects existing in the blind spot area 101 and each part constituting the object can be obtained. The distance information is generated.
As the distance measuring sensor 32a, for example, a well-known distance measuring technique for a vehicle such as a millimeter wave radar, a laser radar, or an ultrasonic sensor can be applied. Further, the outside camera 31a may be used as a distance measuring unit. In this case, the distance information is generated from a distance measured using a technique such as triangulation using a plurality of cameras or TOF using a monocular camera. The distance information acquired from the distance measuring sensor 32a is updated at a constant cycle such as 10 to 240 Hz.

The display control information generated by the display control unit 8 is output to the display 34a. The display 34 a is arranged on the surface of the front pillar 21 of the vehicle 20 that is visible to the driver A so as to overlap the blind spot area 101 that is a blind spot from the viewpoint position 105 of the driver A. The display 34a is configured by using various display devices such as a liquid crystal display (LCD), an organic electro luminescence (OLE) display, and a projector. Further, the display 34a may be a display in which a plurality of small displays are arranged side by side. The display 34a may include an illuminance sensor or the like for adjusting the brightness, and may display an image adjusted according to the amount of solar radiation in the vehicle interior.
The display image generation unit 7 has information on the arrangement position of the display 34a and the display size of the display 34a.

Although detailed description is omitted, the field of view of the driver A is similarly obstructed by the front pillar 23 (so-called A pillar) even when the vehicle 20 is placed on the right side of the vehicle 20, and a blind spot area is generated. Therefore, as shown in FIG. 3, the vehicle exterior camera 31 b and the distance measuring sensor 32 b can be arranged in front of the right side of the vehicle 20 to acquire a captured image and distance information of the blind spot area. A display image that displays an image of the blind spot area in front of the right side of the vehicle 20 is generated by the display control device 10 and is displayed on the display 34 b disposed in front of the front pillar 23. Similarly, for other pillars other than the front pillars 21 and 23 (so-called B pillars, C pillars, etc.), the display control device 10 generates a display image from an image obtained by imaging the blind spot area, and is provided in the pillar. Can be displayed on the display.

Next, the operation of the display control apparatus 10 will be described with reference to the explanatory diagram of FIG. 5 and the flowchart of FIG.
The arrangement positions and functions of the outside camera 31a, the ranging sensor 32a, the in-vehicle camera 33, the display 34a, and the like mounted on the vehicle 20 are the same as those shown in FIGS. . FIG. 5 shows what the driver A of the vehicle 20 looks in the state shown in FIGS. 3 and 4 when he / she sees an area where the two objects B and C as persons are present from the viewpoint position 105. It is a figure.
As shown in FIG. 5, when the driver A sees an area where the objects B and C exist, the front pillar 21 exists between the front window 24 of the vehicle 20 and the left side window 25 of the vehicle 20. . The field of view of the driver A is blocked by the front pillar 21, and a part of the object B and a part of the object C cannot be visually recognized. A region where the field of view is blocked by the front pillar 21 is a blind spot region 101. The display control device 10 generates a display image of the blind spot area 101 that cannot be visually recognized by the driver A, and performs display control for displaying the display image on the display 34a.

FIG. 6 is a flowchart showing the operation of the display control apparatus 10 according to the first embodiment.
First, the image input unit 1, the distance information acquisition unit 2, and the viewpoint information acquisition unit 3 acquire various types of information (step ST1). Specifically, the image input unit 1 receives an input of a captured image captured by the outside camera 31a. Further, the distance information acquisition unit 2 acquires distance information from the distance measuring sensor 32a to an object existing in the blind spot area 101. Further, the viewpoint information acquisition unit 3 acquires viewpoint information indicating the viewpoint position 105 of the driver A.

The distance image generation unit 5 includes the distance information acquired by the distance information acquisition unit 2 in step ST1, the position information of the distance measuring unit preset in the image processing unit 4, and the viewpoint acquired by the viewpoint information acquisition unit 3 in step ST1. From the information, the distance from the viewpoint position of the driver to each object is calculated (step ST2). In the example of FIG. 3, the distance image generation unit 5, for example, the distance 103 from the distance measurement sensor 32 a to the object B, the position information of the distance measurement sensor 32 a set in advance in the image processing unit 4, and the driver A's A distance value 201d from the viewpoint position 105 to the object B is calculated from the viewpoint position 105. Similarly, the distance image generation unit 5 includes the distance 104 from the distance measurement sensor 32a to the object C, the position information of the distance measurement sensor 32a preset in the image processing unit 4, and the viewpoint position 105 of the driver A. The distance value 202d from the viewpoint position 105 to the object C is calculated. Here, for each of the objects B and C, one distance is shown as a representative, but distance information is also generated for at least the distances of the parts constituting the object existing in the blind spot area.

The distance image generation unit 5 refers to the distance from the viewpoint position of the driver calculated in step ST2 to each object, sets the subject area and the background area by dividing the area where the distance is measured by the distance measuring sensor 32a. (Step ST3). In the example of FIG. 3, first, the distance image generation unit 5 collectively sets adjacent areas having the same distance from the viewpoint position 105 of the driver A as one identical target area. Here, the distance is the same, the distance values do not need to be exactly the same, and the two distance values are approximate values (for example, 5 m ± 30 cm) and are distance values obtained by measuring the same object. If it can be determined, the distances are considered to be the same.

Further, if the generated distance value of the same target area is less than the threshold value, the distance image generation unit 5 determines that the area is an object located near the vehicle 20 and sets it as a subject area. The threshold is set to “30 m”, for example. On the other hand, when the generated distance value of the same target area is equal to or greater than the threshold value, the distance image generation unit 5 determines that the area is an object located at a point away from the vehicle 20 and sets it as a background area. .
The subject area is not limited to one, and a plurality of subject areas are set as long as the condition is satisfied. In addition, the distance image generation unit 5 is configured such that, for example, the size of a certain same target area is less than a certain value even if the distance value of the generated same target area is less than a threshold, and the same target area is small enough to be ignored. If it is determined, it may be additionally determined that it is the background area.

The distance image generation unit 5 associates a distance value with the set subject area and background area. The distance image generation unit 5 associates an average value or a median value of the distance values of the subject area as the distance value of the subject area.
In addition, the distance image generation unit 5 uses a distance value farther than the distance value associated with the subject area as the distance value of the background area, and the distance associated with the two subject areas when there are two subject areas. The median value or the same distance value as one subject area is associated.

The distance image generation unit 5 considers the difference between the arrangement positions of the outside camera 31a and the distance measuring sensor 32a based on the position information of the outside camera 31a and the position information of the distance measuring sensor 32a set in the image processing unit 4 in advance. A distance image is generated by associating the distance value between the subject area and the background area set in step ST3 as a pixel value for each pixel of the captured image received by the image input unit 1 in step ST1 (step ST4). ).
FIG. 7 is a diagram illustrating a captured image received by the image input unit 1, and an object B and an object C that are persons exist in the captured image. FIG. 8 is a diagram illustrating a distance image generated by the distance image generation unit 5, and includes a subject area Ba, a subject area Ca, and a background area D. In the example of FIG. 8, the subject area Ba is an area associated with a distance value 201d, the subject area Ca is an area associated with a distance value 202d, and the background area D is associated with a distance value 203d. Indicates that this is an area.

The coordinate conversion unit 6 refers to the distance image generated in step ST4, and in the area around the vehicle 20, the driver's viewpoint position is the center, and the distance value set in each subject area is the radius. A projection spherical surface is set (step ST5). Similarly, with reference to the distance image generated in step ST4, the coordinate conversion unit 6 uses the driver's viewpoint position as the center in the area around the vehicle 20 and sets the distance value set in the background area as the radius. A virtual projection spherical surface is set (step ST6). In the example of FIGS. 3 and 8, the coordinate conversion unit 6 performs the step ST5 and the step ST6, the virtual projection spherical surface 201 having the radius of the distance value 201d set in the subject area Ba and the distance value set in the subject area Ca. A virtual projection spherical surface 202 having a radius of 202d and a virtual projection spherical surface 203 having a radius of the distance value 203d set in the background region D are set. The setting of the virtual projection spherical surface is virtually performed in the calculation processing place in the display control apparatus 10. The virtual projection

spherical surfaces

201, 202, and 203 in FIG. 3 are merely the virtual spherical surfaces.

The coordinate conversion unit 6 converts the coordinate value on the display viewed from the viewpoint position into the coordinate value on the imaging surface of the imaging means using the virtual projection spherical surface set in step ST5 and step ST6 (step ST7). In the example of FIG. 3, the coordinate conversion unit 6 uses the virtual projection

spherical surfaces

201, 202, 203 in step ST 7, and the coordinate values on the surface of the display 34 a viewed from the viewpoint position 105 are not shown in the camera 31 a outside the vehicle. Convert to coordinate values on the imaging surface. Details of the coordinate conversion processing of the coordinate conversion unit 6 will be described later.

The display image generation unit 7 generates a display image for each virtual projection spherical surface using the coordinate values on the imaging surface obtained by converting the coordinate values on the display surface of the display in step ST7 (step ST8). . In the example of FIG. 3, the display image generation unit 7 uses the virtual projection

spherical surfaces

201, 202, and 203 to convert the coordinates on the surface of the display 34 a to display images for display from the coordinate values on the imaging surface. Generate.

The display image generation unit 7 generates a display image to be displayed on the display using each display image generated in step ST8 (step ST9). In the example of FIGS. 3 and 8, the display image generation unit 7 converts the display image of the subject area Ba into the display area generated by coordinate conversion using the virtual projection spherical surface 201 and the subject area information indicating the object B. Similarly, the display image of the subject area Ca is generated from the display image generated using the virtual projection spherical surface 202 and the information of the subject area indicating the object C, and is used for displaying the background area D. An image is generated from the display image generated using the virtual projection spherical surface 203 and the subject area information indicating the background. The display image generation unit 7 generates an overall display image to be displayed on the display by integrating the display images. The display control unit 8 performs display control for displaying the display image generated in step ST9 on the display (step ST10), and returns to the process of step ST1.

9 to 11 are diagrams showing the respective display images generated by the display image generation unit 7.
FIG. 9 shows a display image configured using coordinate values on the imaging screen converted using the virtual projection spherical surface 201, and the size of the subject region Bb is the size when the object B is viewed from the viewpoint position 105. Match.
FIG. 10 shows a display image configured using coordinate values on the imaging screen converted using the virtual projection spherical surface 202, and the size of the subject region Cb is the size when the object C is viewed from the viewpoint position 105. Match.
FIG. 11 shows a display image configured using coordinate values on the imaging screen converted using the virtual projection spherical surface 203. Both the subject area Bb and the subject area Cb do not match the size when the object B and the object C are viewed from the viewpoint position 105.
In contrast to these display images, the display image generated in step ST9 is an image in which the object B, the object C, and the background coincide with the size when viewed from the viewpoint position 105 as shown in FIG. .

Next, the process of the coordinate conversion unit 6 shown in step ST7 of the flowchart of FIG. 6 will be described in more detail with reference to the flowchart of FIG. 12 and the explanatory diagram of FIG.
FIG. 12 is a flowchart showing the operation of the coordinate conversion unit 6 of the display control apparatus 10 according to the first embodiment.
FIG. 13 is a diagram schematically illustrating a coordinate conversion process of the coordinate conversion unit 6 of the display control apparatus 10 according to the first embodiment.
The vehicle exterior camera 31a, the display 34a, the blind spot area 101, the imaging range 102, the viewpoint position 105, the distance values 201d, 202d, and 203d, and the virtual projection

spherical surfaces

201, 202, and 203 shown in FIG. 13 are the same as those shown in FIG. Each corresponds. Further, the subject areas Ba and Ca and the background area D shown in FIG. 13 correspond to the respective areas on the distance image shown in FIG. Moreover, in FIG. 13, the display surface 301 of the display 34a and the imaging surface 302 of the vehicle exterior camera 31a are shown.

The coordinate conversion unit 6 calculates the coordinate in of the intersection of the half line passing the coordinate pn on the display surface 301 of the display 34a from the viewpoint position 105 and the virtual projection spherical surface N (step ST21). The coordinate pn is a coordinate in a coordinate system set so that the position of each point on the display surface 301 can be specified. The coordinates on the virtual projection spherical surface N may be any as long as the position on the virtual projection spherical surface N can be specified. The coordinate conversion unit 6 calculates a coordinate cn indicating an intersection of the straight line connecting the coordinate in calculated in step ST21 and the camera 31a outside the vehicle and the imaging surface 302 as a coordinate after conversion (step ST22). The coordinates on the imaging surface 302 are coordinates in a coordinate system set so that the position of each point on the imaging surface 302 can be specified. The coordinate conversion unit 6 determines whether all necessary coordinates on the display surface 301 have been converted (step ST23). If all necessary coordinates on the display surface 301 have not been converted (step ST23; NO), the coordinate conversion unit 6 sets the next coordinates on the display surface 301 (step ST24), and returns to the processing of step ST21. . The necessary coordinates are, for example, coordinate systems respectively corresponding to all the pixels on the display 34a.

On the other hand, when the coordinates of all the points on the display surface 301 are converted (step ST23; YES), the coordinate conversion unit 6 determines whether the coordinate conversion has been performed for all the virtual projection spherical surfaces (step ST25). When coordinates have not been converted for all virtual projection spheres (step ST25; NO), the coordinate conversion unit 6 sets the next virtual projection sphere (step ST26) and returns to the process of step ST21. On the other hand, when coordinates are converted for all virtual projection spherical surfaces (step ST25; YES), the process proceeds to step ST8 of the flowchart of FIG.

Next, the specific example shown in FIG. 13 will be described along the flowchart of FIG.
In the example of FIG. 13, the coordinate conversion unit 6 performs the processing from step ST21 to step ST26.
It carries out in order with respect to the virtual projection spherical surface 201, the virtual projection spherical surface 202, and the virtual projection spherical surface 203. Further, the coordinate conversion unit 6 repeats the processing from step ST21 to step ST24 for each virtual projection

spherical surface

201, 202, 203.

With respect to the virtual projection spherical surface 201 shown in FIG. 13, the coordinate conversion unit 6 calculates the coordinate i1 of the intersection point on the virtual projection spherical surface 201 and the half line passing through the viewpoint position 105 and the coordinate p1 on the display surface 301 in step ST21. To do. Next, as step ST 22, the coordinate conversion unit 6 calculates the coordinate c 1 of the intersection of the straight line connecting the coordinate i 1 and the vehicle exterior camera 31 a and the imaging surface 302. Thereafter, through the processing of step ST23 and step ST24, as step ST21, the coordinate conversion unit 6 uses the half line passing through the viewpoint position 105 and the coordinate p2 on the display surface 301 and the coordinate i2 of the intersection point on the virtual projection spherical surface 201. calculate. Next, as step ST 22, the coordinate conversion unit 6 calculates the coordinate c 2 of the intersection of the straight line connecting the coordinate i 2 and the outside camera 31 a and the imaging surface 302. In the example of FIG. 13, only the coordinates p1 and p2 on the display surface 301 are shown for the coordinate conversion using the virtual projection spherical surface 201. However, the coordinate conversion unit 6 has all necessary coordinates on the display surface 301. Perform coordinate transformation for.

Similarly, for the virtual projection spherical surface 202, the coordinate conversion unit 6 calculates the coordinate i5 of the intersection point on the virtual projection spherical surface 202 and the half line passing the viewpoint position 105 and the coordinate p5 on the display surface 301 in step ST21. Next, as step ST 22, the coordinate conversion unit 6 calculates the coordinate c 5 of the intersection point between the imaging surface 302 and the straight line connecting the coordinate i 5 and the vehicle exterior camera 31 a. Thereafter, through the processing of step ST23 and step ST24, as step ST21, the coordinate conversion unit 6 uses the half line passing through the viewpoint position 105 and the coordinate p6 on the display surface 301 and the coordinate i6 of the intersection point on the virtual projection spherical surface 202. calculate. Next, as step ST 22, the coordinate conversion unit 6 calculates the coordinate c 6 of the intersection point between the imaging surface 302 and the straight line connecting the coordinate i 6 and the vehicle exterior camera 31 a. In the example of FIG. 13, only the coordinates p5 and p6 on the display surface 301 are shown for the coordinate conversion using the virtual projection spherical surface 202. However, the coordinate conversion unit 6 has all necessary coordinates on the display surface 301. Perform coordinate transformation for.

Similarly, with respect to the virtual projection spherical surface 203, the coordinate conversion unit 6 calculates, as step ST21, the half line passing through the viewpoint position 105 and the coordinate p3 on the display surface 301 and the coordinate i3 of the intersection point on the virtual projection spherical surface 203. Next, as step ST 22, the coordinate conversion unit 6 calculates the coordinate c 3 of the intersection between the straight line connecting the coordinate i 3 and the outside camera 31 a and the imaging surface 302. Thereafter, through the processing of step ST23 and step ST24, as step ST21, the coordinate conversion unit 6 uses the half line passing through the viewpoint position 105 and the coordinate p4 on the display surface 301 and the coordinate i4 of the intersection point on the virtual projection spherical surface 203. calculate. Next, as step ST 22, the coordinate conversion unit 6 calculates a coordinate c 4 of an intersection point between the imaging surface 302 and a straight line connecting the coordinate i 4 and the vehicle exterior camera 31 a. In the example of FIG. 13, only the coordinates p3 and p4 on the display surface 301 are shown for the coordinate conversion using the virtual projection spherical surface 203, but the coordinate conversion unit 6 does not need all the necessary coordinates on the display surface 301. Perform coordinate transformation for.
When the processing is performed for all the coordinates on the display surface 301 for all the virtual projection

spherical surfaces

201, 202, and 203, the process proceeds to step ST8 of the flowchart shown in FIG.

Next, the process of the display image generation unit 7 shown in step ST9 of the flowchart of FIG. 6 will be described in more detail with reference to the flowchart of FIG. 14 and the explanatory diagram of FIG.
FIG. 14 is a flowchart showing the operation of the display image generation unit 7 of the display control apparatus 10 according to the first embodiment.

The display image generation unit 7 sets the image area of the display image displayed on the display from the position information such as the installation position of the display (step ST31). The display image generation unit 7 acquires the distance value at each pixel position of the distance image generated in step ST4 for the image region set in step ST31 (step ST32). The display image generation unit 7 selects a display image generated using the virtual projection spherical surface corresponding to the distance value acquired in step ST32 from the display image generated in step ST8 (step ST33). The display image generation unit 7 determines whether display images have been selected at all pixel positions of the distance image (step ST34). When the display image is not selected for all pixel positions (step ST34; NO), the process returns to step ST33. On the other hand, when display images are selected for all pixel positions (step ST34; YES), the display image generation unit 7 integrates all display images selected in step ST33 to generate a display image. (Step ST35). The display image generation unit 7 outputs the display image generated in step ST35 to the display control unit 8 (step ST36). Thereafter, the process proceeds to step ST10 shown in the flowchart of FIG.

Next, the specific example shown in FIG. 15 will be described along the flowchart of FIG.
FIG. 15 is a diagram illustrating a display image generated by the display image generation unit 7 of the display control apparatus 10 according to the first embodiment.
First, as step ST31, the display image generation unit 7 sets the size of the image area 401 of the display image that can be displayed on the display 34a shown in FIG. Next, as step ST32, the display image generation unit 7 sets, for example, the distance value 201d at the pixel position 402, the distance value 202d at the pixel position 403, and the distance value 203d at the pixel position 404 for the image area 401 whose size is set. Set. In the example of FIG. 15, only the pixel positions 402, 403, and 040 are shown, but distance values are set at all the pixel positions in the image area 401.

As step ST33, the display image generation unit 7 selects a display image generated using the virtual projection spherical surface 201 corresponding to the distance value 201d acquired at the pixel position 402, for example. Similarly, as step ST33, the display image generation unit 7 selects a display image generated using the virtual projection spherical surface 202 corresponding to the distance value 202d acquired at the pixel position 403. In step ST33, the display image generation unit 7 selects a display image generated using the virtual projection spherical surface 203 corresponding to the distance value 203d acquired at the pixel position 404.
When the display image generation unit 7 selects display images for all the pixel positions in the image area 401 in step ST34 (step ST34; YES), the image area 401 shown in FIG. A display image as shown in FIG. As step ST 36, the display image generation unit 7 outputs data indicating the generated display image to the display control unit 8.

Then, the display control part 8 performs display control of the input display image as step ST10.
FIG. 16 is a diagram illustrating a display result of the display control apparatus 10 according to the first embodiment.
As shown in FIG. 16, the display control unit 8 performs display control for displaying the display image indicated by the image area 401 in FIG. 15 on the display 34 a arranged in front of the front pillar 21.
According to the display control of the display control unit 8, the display 34a displays the display image indicated by the image area 401, so that the driver A actually moves from the viewpoint position 105 through the front window 24 and the side window 25 as shown in FIG. The scenery outside the vehicle seen in FIG. 5 and the display image of the blind spot area displayed on the display 34a on the front pillar 21 are continuously displayed. Thereby, the driver A does not feel discomfort in the continuity between the scenery of the vehicle actually viewed from the viewpoint position 105 and the display image on the display 34a.

As described above, according to the first embodiment, the driver's viewpoint is obtained by using a captured image in which a blind spot area in which the driver's field of view is blocked by the vehicle structure is imaged. A distance image generation unit 5 that generates a distance image using a value of a distance from a position to an object located in a blind spot area as a pixel value, and using the distance image, the driver's viewpoint position is centered in a peripheral area of the vehicle. A plurality of virtual projection spheres defined as the outer peripheral surface of the sphere are set, and coordinates on the display screen for displaying the captured image are converted into coordinates on the imaging surface using the plurality of virtual projection spheres set. Since it is configured to include the coordinate conversion unit 6 and the display image generation unit 7 that generates an image to be displayed on the display screen using the converted coordinates on the imaging screen, Outside the actual vehicle as seen from the viewpoint position It can be displayed by continuously color.

In the first embodiment described above, the distance image generation unit 5 sets the distance image having three target regions based on the three distance values. However, the distance value to be used is three. It is not limited to. For example, the distance image generation unit 5 may generate the same target area in units of pixels. In that case, the coordinate conversion unit 6 sets a virtual projection spherical surface in units of pixels, and performs coordinate conversion using the virtual projection spherical surface of each pixel. Further, the distance image generation unit 5 may be configured to set conditions for generation of the distance image and reduce the number of virtual projection spherical surfaces set by the coordinate conversion unit 6.

Various conditions can be applied for setting the distance image.
For example, the distance image generation unit 5 generates a distance image based on the condition that the number of virtual projection spherical surfaces is three. Further, the distance image generation unit 5 generates a distance image based on the condition that three objects in order from the smallest distance value are subjects for generating the same target region. Further, the distance image generation unit 5 generates a distance image based on the condition that the distance value of ± 50 cm is regarded as the same distance value. In addition, the distance image generation unit 5 generates a distance image based on the condition that all objects at a position having a distance value of 30 m or more are regarded as the background. Further, the distance image generation unit 5 refers to the appearance frequency of the distance value, and generates a distance image based on a condition that a certain number of distance images are generated in descending order of appearance frequency.

Embodiment 2. FIG.
The second embodiment shows a configuration for generating a display image in consideration of vehicle information.
FIG. 17 is a block diagram illustrating a configuration of the display control apparatus 10a according to the second embodiment.
The display control apparatus 10a according to the second embodiment is provided with a vehicle information acquisition unit 9 in addition to the display control apparatus 10 shown in the first embodiment, and includes an image processing unit 4a instead of the image processing unit 4. The image processing unit 4a includes a distance image generation unit 5, a coordinate conversion unit 6a, and a display image generation unit 7a.
In the following, the same or corresponding parts as the components of the display control apparatus 10 according to the first embodiment are denoted by the same reference numerals as those used in the first embodiment, and the description thereof is omitted or simplified.

The vehicle information acquisition unit 9 acquires vehicle information of a vehicle on which the display control device 10 is mounted via an in-vehicle network (not shown). The vehicle information is information indicating, for example, own vehicle position information, traveling direction, vehicle speed, acceleration, steering angle, and the like. Information acquired by the vehicle information acquisition unit 9 is input to the image processing unit 4.

The coordinate conversion unit 6a of the image processing unit 4a determines the number of virtual projection spherical surfaces to be set according to the distance image generated by the distance image generation unit 5 and the vehicle information acquired by the vehicle information acquisition unit 9. For example, when the coordinate conversion unit 6a refers to the vehicle speed of the vehicle information, when it is determined that the vehicle is traveling at a high speed from the vehicle speed acquired by the vehicle information acquisition unit 9, the number of virtual projection spherical surfaces is reduced. By changing the set number of virtual projection spherical surfaces according to the vehicle information by the coordinate conversion unit 6a, the processing load of image processing can be suppressed.

The display image generation unit 7a changes the image data of the display image to be generated according to the vehicle information acquired by the vehicle information acquisition unit 9. Here, the case where the display image generation unit 7a refers to the vehicle speed of the vehicle information will be described.
The processing amount that can be processed by the display image generating unit 7a is calculated by the data amount (size) of one frame of the display image to be generated and the number of frames to be displayed per second.
When the processing capability of the display image generation unit 7a is constant and the vehicle is traveling at a high speed, the update speed of the display image is determined from the definition of one frame of the display image. Therefore, when the vehicle is traveling at a high speed, the display image generation unit 7a reduces the resolution of the display image to be generated, and conversely improves the update speed of the display image. Thereby, the display image generation unit 7a can suppress the processing load of the image processing.

On the other hand, when the processing capability of the display image generation unit 7a is constant and the vehicle is traveling at a low speed, improvement in the definition of one frame of the display image is required rather than the update speed of the display image. Therefore, when the vehicle is traveling at a low speed, the display image generation unit 7a decreases the update speed of the display image to be generated and improves the definition of the display image. Thereby, the display image generation unit 7a can generate a display image with improved visibility.

The display image generation unit 7a is configured to perform a process of changing the number of colors of the display image according to the vehicle information acquired by the vehicle information acquisition unit 9, and to adjust the processing amount and the image quality of the display image. May be.
Moreover, the process of the coordinate conversion part 6a mentioned above and the process of the display image generation part 7a may be performed simultaneously, and only one process may be performed.

Next, a hardware configuration example of the display control apparatus 10a will be described. Note that the description of the same configuration as that of Embodiment 1 is omitted.
The vehicle information acquisition unit 9 in the display control device 10a is realized by the input device 11 in FIG. 2 that inputs information from the outside. Further, the coordinate conversion unit 6a and the display image generation unit 7a in the display control device 10a are realized by a processing circuit. In the processing circuit, functions may be executed by dedicated hardware or software. When the function is executed by software, the processing circuit is the CPU 12 that executes a program stored in the memory 13 shown in FIG.

Next, the operation of the display control apparatus 10a according to Embodiment 2 will be described.
First, the processing of the coordinate conversion unit 6a will be described with reference to FIGS.
FIG. 18 is a flowchart showing the operation of the coordinate conversion unit 6a of the display control apparatus 10a according to the second embodiment. In FIG. 18, the same steps as those in the flowchart of the first embodiment shown in FIG.
FIG. 19 is a diagram illustrating an example of data referred to by the coordinate conversion unit 6a of the display control apparatus 10a according to the second embodiment.
When the distance image is generated in step ST4, the coordinate conversion unit 6a refers to the vehicle information acquired by the vehicle information acquisition unit 9, and sets the number of virtual projection spherical surfaces set according to the vehicle information (step ST41). . For example, the coordinate conversion unit 6a refers to a database (not shown) in which the conditions shown in FIG. 19 are stored, and sets the number of virtual projection spherical surfaces according to the vehicle speed of the vehicle information, for example. When the vehicle speed of the vehicle is medium speed and the number of virtual projection spherical surfaces is set to 3, the coordinate conversion unit 6a performs the processing of step ST5 and step ST6 so that three virtual projection spherical surfaces are set. Note that which distance value the coordinate conversion unit 6a uses to set the virtual projection spherical surface is determined so that, for example, distance values having a high distribution of appearance frequencies are used in order as described in the first embodiment. .

Next, the processing of the display image generation unit 7a will be described with reference to FIGS. 20A and 20B.
20A and 20B are diagrams illustrating an example of data referred to by the display image generation unit 7a of the display control apparatus 10a according to Embodiment 2.
The display image generation unit 7a refers to a database (not shown) in which the setting conditions shown in FIG. 20A or 20B are stored, and determines the setting conditions for the display image according to the vehicle information.
The setting conditions of FIGS. 20A and 20B divide the vehicle speed into three stages of low speed driving, medium speed driving, and high speed driving, and the resolution (definition) of the display image set by the display image generating unit 7a according to each driving speed. The frame rate (update speed) of the display image and the number of colors of the display image are shown.
FIG. 20A shows a case where the number of colors of the display image is a fixed value.

The display image generation unit 7a refers to the setting conditions of FIG. 20A. When the vehicle is traveling at a low speed, the display image generation unit 7a generates a display image based on the conditions of resolution 1920 × 960, frame rate 30 fps, color number RGB 24 bits. Generate.
Further, when the vehicle is traveling at a medium speed, the display image generation unit 7a generates a display image based on the conditions of resolution 1280 × 720, frame rate 60 fps, color number RGB 24 bits.
Further, when the vehicle is traveling at a high speed, the display image generation unit 7a generates a display image based on the conditions of resolution 960 × 480, frame rate 120 fps, color number RGB 24 bits.

FIG. 20B shows a case where the resolution of the display image is a fixed value.
The display image generation unit 7a refers to the database of FIG. 20B, and generates a display image based on the conditions of resolution 1280 × 720, frame rate 30 fps, color number RGB 48 bits when the vehicle is traveling at a low speed. To do.
Further, when the vehicle is traveling at a medium speed, the display image generation unit 7a generates a display image based on the conditions of resolution 1280 × 720, frame rate 60 fps, color number RGB 24 bits.
In addition, when the vehicle is traveling at high speed, the display image generation unit 7a generates a display image based on the conditions of resolution 1280 × 720, frame rate 120 fps, color number YUV 16 bits.

In the above description, the coordinate conversion unit 6a and the display image generation unit 7a are configured to refer to the vehicle speed of the vehicle information. The number of projection spheres can be set, the definition of the display image, the update speed of the display image, and the like can be determined.

As described above, according to the second embodiment, the vehicle information acquisition unit 9 that acquires the vehicle information indicating the traveling state of the vehicle is provided, and the coordinate conversion unit 6a refers to the vehicle information according to the vehicle information. Since the configuration is such that the number of virtual projection spheres to be generated is set, it is possible to suppress the load of coordinate conversion processing.

Moreover, according to this Embodiment 2, it has the vehicle information acquisition part 9 which acquires the vehicle information which shows the driving state of a vehicle, and the display image generation part 7a refers to vehicle information, according to the driving state of a vehicle. Since it is configured to determine at least one of the definition of the display image to be generated, the update speed of the display image, or the number of colors of the display image, the load when generating the display image can be suppressed. it can.

In the first and second embodiments described above, the distance from the distance measuring sensor 32a to the object in the area, the position information of the outside camera 31a, the position information of the distance measuring sensor 32a, the driver's viewpoint information, and the display The description has been made on the assumption that the coordinates on 34a are information expressed using three-dimensional spatial coordinates.

In addition to the above, within the scope of the present invention, the present invention can be freely combined with each embodiment, modified any component of each embodiment, or omitted any component in each embodiment. Is possible.

The display control device according to the present invention is provided in a structure of a vehicle because an image obtained by imaging a region that becomes a blind spot for a driver can be displayed continuously with an actual scene viewed from the viewpoint position of the driver. The present invention can be applied to display an image on a display and can be used to improve driver visibility.

1 image input unit, 2 distance information acquisition unit, 3 viewpoint information acquisition unit, 4, 4a image processing unit, 5 distance image generation unit, 6, 6a coordinate conversion unit, 7, 7a display image generation unit, 8 display control unit, 9 Vehicle information acquisition unit, 10, 10a Display control device.

Claims

Using a captured image obtained by capturing a blind spot area in which the driver's field of view is blocked by a structure of the vehicle, the area from the driver's viewpoint position to an object located in the blind spot area. A distance image generation unit that generates a distance image with a distance value as a pixel value;
Using the distance image generated by the distance image generation unit, a plurality of virtual projection spheres defined as outer peripheral surfaces of a sphere centered on the driver's viewpoint position are set in an area around the vehicle, A coordinate conversion unit that converts coordinates on the display screen for displaying the captured image into coordinates on the imaging surface of the captured image using a plurality of virtual projection spherical surfaces;
A display control apparatus comprising: a display image generation unit configured to generate an image to be displayed on the display screen using the coordinates on the imaging surface converted by the coordinate conversion unit.
The distance image generation unit sets a plurality of identical target areas in the distance image based on the distance value,
The coordinate conversion unit, for each of the plurality of the same target region set by the distance image generation unit, a radius value of the distance from the driver's viewpoint position to the same target region, the driver's viewpoint position The display control apparatus according to claim 1, wherein the virtual projection spherical surface at the center is set.
The distance image generation unit generates the distance image with the pixel value as a fixed value when a distance from the driver's viewpoint position to an object located in the blind spot area is a threshold value or more. The display control apparatus according to claim 1.
The display control apparatus according to claim 1, wherein the coordinate conversion unit changes the number of virtual projection spherical surfaces to be set according to vehicle information of the vehicle.
The display image generation unit changes any one of a resolution of an image to be displayed on the display screen, a frame rate of the image, or a number of colors of the image according to vehicle information of the vehicle. The display control apparatus according to claim 1.
The range image generation unit uses the captured image obtained by capturing a blind spot area in which the driver's field of view is blocked by a structure of the vehicle in a peripheral area of the vehicle, from the viewpoint position of the driver to the blind spot area. Generating a distance image having a pixel value as a distance value to an object located at a position;
A coordinate conversion unit sets a plurality of virtual projection spheres defined as outer peripheral surfaces of a sphere centered on the driver's viewpoint position in a region around the vehicle using the distance image, and Transforming coordinates on the display screen for displaying the captured image into coordinates on the imaging surface of the captured image using a projection spherical surface;
A display image generation unit including a step of generating an image to be displayed on the display screen using coordinates on the imaging surface.