JP2019135816A - Display control device, display control method, and camera monitoring system - Google Patents

Abstract

To reduce feeling discomfort about a virtual image visually recognized in a camera monitoring system.SOLUTION: A display control device connected to an imaging apparatus installed in a vehicle comprises: a conversion unit that converts the coordinates of each pixel of image data captured by the imaging apparatus; and an output unit that outputs display image data generated by the conversion of the coordinates by the conversion unit to display units. The conversion unit converts the coordinates of each pixel of the image data captured by the imaging apparatus, on the basis of a positional relationship between a position where the display image data displayed in the display units is visually recognized as a virtual image via an optical member and a position where the imaging apparatus is installed.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display control device, a display control method, and a camera monitoring system.

  2. Description of the Related Art Conventionally, there is known a display control technique in which a camera monitoring system is mounted on a vehicle and a captured image behind the vehicle is displayed on a display device or the like in the vehicle to function as an electronic mirror.

  For example, Patent Document 1 below discloses a camera monitoring system that functions as an electronic side mirror by installing an imaging device at a position of a side mirror of a vehicle and displaying a captured image on a display device. Further, Patent Document 1 below further increases the focal length from the driver by causing the driver to visually recognize the captured image displayed on the display device as a virtual image through the concave mirror. A configuration for reducing the above is disclosed.

JP 2005-329768 A

  However, in the case of the camera monitoring system, the position of the virtual image visually recognized by the driver does not match the position where the imaging device is installed. For this reason, it is conceivable that the driver feels uncomfortable with the visually recognized virtual image.

  The present invention has been made in view of the above problems, and an object thereof is to reduce a sense of incongruity with a virtual image visually recognized in a camera monitoring system.

According to one aspect, the display control device has the following configuration. That is,
A display control device connected to an imaging device installed in a vehicle,
A conversion unit that converts coordinates of each pixel of image data captured by the imaging device;
An output unit that outputs display image data generated by converting coordinates by the conversion unit to the display unit,
The converter is
The image captured by the imaging device based on the positional relationship between the position where the display image data displayed on the display unit is visually recognized as a virtual image via an optical member and the position where the imaging device is installed It is characterized by converting the coordinates of each pixel of data.

  It is possible to reduce a sense of incongruity with a virtual image visually recognized in the camera monitoring system.

It is a figure which shows an example of the system configuration | structure of the camera monitoring system in 1st Embodiment. It is a figure which shows the position where an imaging device is installed, the imaging | photography range and imaging | photography direction of an imaging device. It is a figure for demonstrating the function of an optical member. It is a figure which shows the positional relationship of the position where an imaging device is installed, and the position of the virtual image visually recognized by the driver | operator. It is a figure which shows an example of conversion information. It is a figure which shows an example of the hardware constitutions of a display control apparatus. It is a figure which shows an example of a function structure of the viewpoint conversion part of the display control apparatus in 1st Embodiment. It is a flowchart which shows the flow of the viewpoint conversion process by the viewpoint conversion part of the display control apparatus in 1st Embodiment. It is a figure which shows an example of a function structure of the viewpoint conversion part of the display control apparatus in 2nd Embodiment. It is a flowchart which shows the flow of the viewpoint conversion process by the viewpoint conversion part of the display control apparatus in 2nd Embodiment. It is a figure which shows an example of the system configuration | structure of the camera monitoring system in 3rd Embodiment. It is a figure which shows the position where an imaging device is installed, and the imaging range of an imaging device in the camera monitoring system in 3rd Embodiment.

  Each embodiment will be described below with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, the duplicate description is abbreviate | omitted by attaching | subjecting the same code | symbol.

[First Embodiment]
<System configuration of camera monitoring system>
First, the system configuration of the camera monitoring system mounted on the vehicle will be described. FIG. 1 is a diagram illustrating an example of a system configuration of a camera monitoring system according to the first embodiment. As shown in FIG. 1, the camera monitoring system 100 includes imaging devices 110 and 111, a display control device 120, display devices 130 and 131, and optical members 140 and 141. In the present embodiment, the camera monitoring system 100 starts when the ignition switch of the vehicle is turned on and stops when the vehicle is turned off. However, the camera monitoring system 100 may be configured to start and stop based on an instruction from the driver of the vehicle.

  The imaging devices 110 and 111 are installed at the positions of the side mirrors of the vehicle and photograph the rear of the vehicle. The imaging devices 110 and 111 transmit image data (left image data and right image data) obtained by reversing the captured image to the display control device 120.

  A viewpoint conversion program is installed in the display control device 120, and the display control device 120 functions as the viewpoint conversion unit 121 by executing the viewpoint conversion program. The viewpoint conversion unit 121 reads the conversion information from the conversion information storage unit 122 and converts the left image data and the right image data to generate left display image data and right display image data. Further, the viewpoint conversion unit 121 outputs the generated left display image data and right display image data to the display devices 130 and 131, respectively.

  The display devices 130 and 131 display left display image data or right display image data, respectively. The left display image data displayed by the display device 130 is visually recognized by the vehicle driver as a virtual image via the optical member 140. The right display image data displayed by the display device 131 is visually recognized by the vehicle driver as a virtual image through the optical member 141.

<Installation example of imaging device>
Next, an installation example of the imaging devices 110 and 111 (a position where the imaging devices 110 and 111 are installed, a shooting range and a shooting direction of the imaging devices 110 and 111) will be described. FIG. 2 is a diagram illustrating a position where the imaging device is installed, a shooting range and a shooting direction of the imaging device.

  As shown in FIG. 2, the imaging device 110 is installed at or near the position where the left side mirror of the vehicle 200 is originally attached. The imaging device 110 performs imaging using the shaded area 210 behind the vehicle 200 as an imaging range. The shooting direction at this time is as shown by an arrow 220. Similarly, the imaging device 111 is installed at or near the position where the right side mirror of the vehicle 200 is originally attached. The imaging device 111 performs imaging using the shaded area 211 behind the vehicle 200 as an imaging range. The shooting direction at this time is as shown by an arrow 221.

<Function of optical member>
Next, functions of the optical members 140 and 141 for causing the driver of the vehicle 200 to visually recognize the left display image data or the right display image data displayed on the display devices 130 and 131 as a virtual image will be described. Since the function of the optical member 140 and the function of the optical member 141 are basically the same, the function of the optical member 141 will be described here.

  FIG. 3 is a diagram for explaining the function of the optical member. As shown in FIG. 3, the optical member 141 includes a concave mirror 301 and a concave mirror 302. The concave mirror 301 reflects the emitted light of the right display image data emitted from the display device 131. The concave mirror 302 further reflects the emitted light of the right display image data reflected by the concave mirror 301. The emitted light of the right-side display image data reflected by the concave mirror 302 is captured by the vision of the driver 300 of the vehicle 200.

  Here, the concave mirror 301 is installed at a position separated from the position where the display device 131 is installed by a distance L1. For this reason, the emitted light of the right display image data reflected by the concave mirror 301 is visually recognized as a virtual image 311 located at a distance L1 behind the concave mirror 301 at a position on the reflective side of the concave mirror 301.

  Similarly, the concave mirror 302 is installed at a position separated from the position of the virtual image 311 by a distance L2. For this reason, the emitted light of the right display image data reflected by the concave mirror 302 is a virtual image 312 at a distance L2 behind the concave mirror 302 at a position on the reflective side of the concave mirror 302 (that is, by the driver 300 of the vehicle 200). Will be visually recognized.

<Positional relationship between the position where the imaging device is installed and the position of the virtual image>
Next, the positional relationship between the position where the imaging devices 110 and 111 are installed and the position of the virtual image visually recognized by the driver 300 of the vehicle 200 will be described. FIG. 4 is a diagram illustrating a positional relationship between the position where the imaging device is installed and the position of the virtual image visually recognized by the driver.

  As described above, the left display image data displayed by the display device 130 is visually recognized as a virtual image by the driver 300 via the optical member 140. The right display image data displayed by the display device 131 is visually recognized as a virtual image by the driver 300 through the optical member 141.

  Here, the optical member 140 is configured so that the virtual image 400 is visually recognized at a position slightly away from the left side surface of the vehicle 200. Similarly, the optical member 141 is configured so that the virtual image 401 is visually recognized at a position slightly away from the right side surface of the vehicle 200. This is because the mirror surface portion of the side mirror in a general vehicle is in a position slightly away from the left side surface (or right side surface) of the vehicle.

  Assuming that the mirror surface portion of a side mirror in a general vehicle is at the position of the virtual image 400, the direction and region when the vehicle driver 300 visually recognizes the rear of the vehicle through the side mirror is an arrow. The direction indicated by 420 and the area indicated by the shaded area 410 are provided. In other words, the left display image data visually recognized as a virtual image 400 by the driver 300 of the vehicle 200 is captured with the imaging device 110 installed at the position of the virtual image 400, the arrow 420 as the shooting direction, and the shaded area 410 as the shooting range. It is desirable that

  However, actually, as described with reference to FIG. 2, the imaging device 110 is installed on the left side surface of the vehicle 200, and the arrow 220 is the shooting direction and the shaded area 210 is the shooting range. That is, in a general vehicle, the position of the virtual image 400 does not match the position where the imaging device 110 is installed. For this reason, the driver 300 of the vehicle 200 feels uncomfortable in the direction and the area visually recognized as the virtual image 400. Therefore, the viewpoint conversion unit 121 converts the left image data so as to reduce the uncomfortable feeling, and generates left display image data.

  Similarly, if the mirror surface portion of the side mirror in a general vehicle is at the position of the virtual image 401, the direction and region when the vehicle rear is visually recognized by the driver 300 of the vehicle through the side mirror is as follows: A direction indicated by an arrow 421 and an area indicated by a shaded area 411 are provided. In other words, right-side display image data visually recognized as a virtual image 401 by the driver 300 of the vehicle 200 is captured with the imaging device 111 installed at the position of the virtual image 401, the arrow 421 as the imaging direction, and the shaded area 411 as the imaging range. It is desirable that

  However, actually, as described with reference to FIG. 2, the imaging device 111 is installed on the right side surface of the vehicle 200, and the arrow 221 is the shooting direction and the shaded area 211 is the shooting range. That is, in a general vehicle, the position of the virtual image 401 and the position where the imaging device 110 is installed do not match. For this reason, the driver 300 of the vehicle 200 feels uncomfortable in the direction and area to be visually recognized as the virtual image 401. Therefore, the viewpoint conversion unit 121 converts the right image data so as to reduce the uncomfortable feeling, and generates right display image data.

<Details of conversion information>
Next, the conversion information in the conversion information storage unit 122 that is referred to when the viewpoint conversion unit 121 converts the left image data and the right image data into the left display image data and the right display image data will be described. FIG. 5 is a diagram illustrating an example of conversion information.

  Among these, FIG. 5A is a diagram illustrating an example of the conversion information 510 that is referred to when the viewpoint conversion unit 121 converts the left image data into the left display image data. As shown in FIG. 5A, the conversion information 510 includes, as information items, “viewpoint position”, “viewpoint direction”, “image coordinates” of the imaging apparatus, “viewpoint position”, and “viewpoint direction” of the virtual image. , And "image coordinates".

In the conversion information 510, the “viewpoint position” of the imaging device stores coordinates (“coordinate P _LC ”) indicating the position where the imaging device 110 is installed. In addition, in the “viewpoint direction” of the imaging device, a six-axis lens parameter (“lens parameter D _LC ”) indicating the shooting direction of the imaging device 110 is stored. Furthermore, the “image coordinates” of the imaging device stores the coordinates of each pixel of the left image data captured by the imaging device 110 at the corresponding viewpoint position and viewpoint direction.

Further, in the conversion information 510, coordinates (“coordinate P _LI ”) indicating the position of the virtual image 400 are stored in the “viewpoint position” of the virtual image. In the “viewpoint direction” of the imaging device, a six-axis lens parameter (“lens parameter D _LC ”) indicating the imaging direction when the imaging device 110 is installed at the position of the virtual image 400 is stored. Furthermore, the “image coordinates” of the virtual image stores the coordinates of each pixel of the image data captured by the imaging device 110 at the corresponding viewpoint position and viewpoint direction.

In FIG. 5A, “image coordinates” = “x _L1 , y _L1 ”,..., “X _Ln , y _Lm ” of the imaging device, and “image coordinates” = “X _L1 , Y _L1 ” of the virtual image, ... "X _Ln , Y _Lm " are associated with each other. The “image coordinates” of the imaging device and the “image coordinates” of the virtual image are associated with each other, for example, in the case of “x _L1 , y _L1 ” and “X _L1 , Y _L1 ” It is in.
When the imaging apparatus 110 captures the rear of the vehicle 200 with the viewpoint position = “coordinate P _LC ” and the viewpoint direction = “lens parameter D _LC ” on the image data of the subject A behind the vehicle 200 The coordinates are “x _L1 , y _L1 ”.
When the imaging apparatus 110 captures the rear of the vehicle 200 with the viewpoint position = “coordinate P _LI ” and the viewpoint direction = “lens parameter D _LI ” on the image data of the subject A behind the vehicle 200 The coordinates are “X _L1 , Y _L1 ”.

  That is, in the conversion information 510, the “image coordinates” of the imaging device and the “image coordinates” of the virtual image store the coordinates of corresponding points on the image data when the same subject is photographed at different viewpoint positions and viewpoint directions. The

  Similarly, FIG. 5B is a diagram illustrating an example of conversion information 520 that is referred to when the right image data is converted into right display image data. As shown in FIG. 5B, the conversion information 520 includes, as information items, “viewpoint position”, “viewpoint direction”, “image coordinates” of the imaging apparatus, “viewpoint position”, and “viewpoint direction” of the virtual image. , And "image coordinates".

In the conversion information 520, coordinates (“coordinate P _RC ”) indicating the position where the imaging device 111 is installed are stored in “viewpoint position” of the imaging device. In addition, in the “viewpoint direction” of the imaging apparatus, a six-axis lens parameter (“lens parameter D _RC ”) indicating the imaging direction of the imaging apparatus 111 is stored. Further, the “image coordinates” of the imaging device stores the coordinates of each pixel of the right image data captured by the imaging device 111 at the corresponding viewpoint position and viewpoint direction.

Further, in the conversion information 520, coordinates (“coordinate P _RI ”) indicating the position of the virtual image 401 are stored in the “viewpoint position” of the virtual image. In the “viewpoint direction” of the imaging device, a six-axis lens parameter (“lens parameter D _RC ”) indicating the imaging direction when the imaging device 111 is installed at the position of the virtual image 401 is stored. Furthermore, the “image coordinates” of the virtual image stores the coordinates of each pixel of the image data captured by the imaging device 111 at the corresponding viewpoint position and viewpoint direction.

In FIG. 5B, “image coordinates” = “x _R1 , y _R1 ”,..., “X _Rn , y _Rm ” of the imaging device, and “image coordinates” = “X _R1 , Y _R1 ” of the virtual image, ... "X _Rn , Y _Rm " are associated with each other. The “image coordinates” of the imaging device and the “image coordinates” of the virtual image are associated with each other, for example, in the case of “x _R1 , y _R1 ” and “X _R1 , Y _R1 ”, as follows: It is in.
When the imaging apparatus 111 captures the rear of the vehicle 200 with the viewpoint position = “coordinate P _RC ” and the viewpoint direction = “lens parameter D _RC ” on the image data of the subject B behind the vehicle 200 The coordinates are “x _R1 , y _R1 ”.
When the imaging apparatus 111 captures the rear of the vehicle 200 with the viewpoint position = “coordinate P _RI ” and the viewpoint direction = “lens parameter D _RI ” on the image data of the subject B behind the vehicle 200 The coordinates are “X _R1 , Y _R1 ”.

  That is, in the conversion information 520, the coordinates of the corresponding points on the image data when the same subject is photographed at different viewpoint positions and viewpoints are stored in the “image coordinates” of the imaging device and the “image coordinates” of the virtual image. The

<Hardware configuration of display control device>
Next, the hardware configuration of the display control device 120 will be described. FIG. 6 is a diagram illustrating an example of a hardware configuration of the display control apparatus. As illustrated in FIG. 6, the display control device 120 includes a CPU (Central Processing Unit) 601, a ROM (Read Only Memory) 602, and a RAM (Random Access Memory) 603. Further, the display control device 120 includes a GPU (Graphic Processing Unit) 604 and an I / F (Interface) device 605. Each unit of the display control device 120 is connected to each other via a bus 606.

  The CPU 601 is a device that executes various programs (for example, a viewpoint conversion program) installed in the ROM 602. A ROM 602 is a nonvolatile memory and functions as a main storage device that stores various programs executed by the CPU 601. The RAM 603 is a volatile memory such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). The RAM 603 functions as a main storage device that provides a work area that is expanded when various programs installed in the ROM 602 are executed by the CPU 601.

  The GPU 604 is a dedicated integrated circuit that performs image processing. In the present embodiment, the left display image data and the right display image data displayed on the display devices 130 and 131 based on an instruction from the CPU 601 that executes the viewpoint conversion program. And so on. The I / F device 605 is a connection device for connecting the imaging devices 110 and 111 and the display devices 130 and 131 to the display control device 120.

<Functional configuration of display control device>
Next, the functional configuration of the display control device 120 will be described. FIG. 7 is a diagram illustrating an example of a functional configuration of the viewpoint conversion unit of the display control apparatus according to the first embodiment. As described above, the viewpoint conversion program is installed in the display control apparatus 120, and when the camera monitoring system 100 is activated, the display control apparatus 120 executes the viewpoint conversion program. Accordingly, the display control device 120 includes a left image data acquisition unit 701, a right image data acquisition unit 711, a left image data conversion unit 702, a right image data conversion unit 712, a left display image data generation unit 703, and a right display image data generation. It functions as the unit 713.

  The left image data acquisition unit 701 and the right image data acquisition unit 711 acquire the left image data and the right image data transmitted from the imaging devices 110 and 111. The left image data acquisition unit 701 and the right image data acquisition unit 711 transmit the acquired left image data and right image data to the left image data conversion unit 702 and the right image data conversion unit 712 in units of frames.

  The left image data conversion unit 702 and the right image data conversion unit 712 are an example of a conversion unit. Based on the conversion information 510 and 520 stored in the conversion information storage unit 122, each pixel of the left image data and the right image data. Convert the coordinates of. Note that the conversion of the left image data and the right image data is obtained by calculation using a general conversion method (for example, coordinate conversion of the projection plane, affine conversion, etc.) instead of referring to the conversion information 510 and 520. Also good.

  The left display image data generation unit 703 and the right display image data generation unit 713 are examples of output units. The left display image data generation unit 703 and the right display image data generation unit 713 extract a region suitable for display on the display devices 131 and 132 from the left image data and the right image data obtained by converting the coordinates of each pixel. The left display image data and the right display image data are generated. The left display image data generation unit 703 and the right display image data generation unit 713 output the generated left display image data and right display image data to the display device 130 and the display device 131, respectively.

<Flow of viewpoint conversion processing by viewpoint converter>
Next, a flow of viewpoint conversion processing by the viewpoint conversion unit 121 of the display control device 120 will be described. FIG. 8 is a flowchart illustrating a flow of viewpoint conversion processing by the viewpoint conversion unit of the display control apparatus according to the first embodiment.

  Among these, FIG. 8A is a flowchart showing the flow of viewpoint conversion processing for the left image data. In step S <b> 801, the left image data acquisition unit 701 reads conversion information 510 from the conversion information storage unit 122. In step S <b> 802, the left image data acquisition unit 701 acquires left image data from the imaging device 110.

  In step S803, the left image data conversion unit 702 converts the coordinates of each pixel of the left image data based on the conversion information 510. In step S804, the left display image data generation unit 703 generates left display image data based on the left image data obtained by converting the coordinates of each pixel, and outputs the left display image data to the display device 130.

  In step S805, the left image data acquisition unit 701 determines whether or not to end the viewpoint conversion process. If it is determined that the viewpoint conversion process is not to be ended (No in step S805), the process returns to step S802. On the other hand, if it is determined in step S805 that the process is to be ended (Yes in step S805), the left image data acquisition unit 701 ends the viewpoint conversion process for the left image data.

  On the other hand, FIG. 8B is a flowchart showing the flow of viewpoint conversion processing for the right image data. In step S 811, the right image data acquisition unit 711 reads the conversion information 520 from the conversion information storage unit 122. In step S812, the right image data acquisition unit 711 acquires right image data from the imaging device 111.

  In step S813, the right image data conversion unit 712 converts the coordinates of each pixel of the right image data based on the conversion information 520. In step S814, the right display image data generation unit 713 generates right display image data based on the right image data in which the coordinates of each pixel are converted, and outputs the right display image data to the display device 131.

  In step S815, the right image data acquisition unit 711 determines whether or not to end the viewpoint conversion process. If it is determined not to end (in the case of No in step S815), the process returns to step S812. On the other hand, when it is determined in step S815 that the processing is to be ended (in the case of Yes in step S815), the right image data acquisition unit 711 ends the viewpoint conversion processing for the right image data.

<Summary>
As is apparent from the above description, the display control apparatus in the first embodiment is
The coordinates of each pixel of the image data photographed by the imaging device are converted and displayed on the display device (display unit).
-At that time, based on the positional relationship (viewpoint position, viewpoint direction) between the position where the display image data displayed by the display device is visually recognized as a virtual image via the optical member and the position where the imaging device is installed, The coordinates of each pixel of the image data are converted.

  Thereby, according to the display control apparatus in 1st Embodiment, the difference between the position of the virtual image visually recognized by the driver | operator and the position where the imaging device was set can be absorbed. As a result, in the camera monitoring system that functions as an electronic side mirror, the driver's discomfort with respect to the virtual image can be reduced.

[Second Embodiment]
In the first embodiment, the viewpoint direction of the virtual image is fixed. On the other hand, in the second embodiment, a case will be described in which the viewpoint direction of the virtual image is variable based on an instruction from the driver of the vehicle. This is because the viewpoint direction of the mirror surface portion of the side mirror in a general vehicle can be changed based on an instruction from the driver of the vehicle. Hereinafter, the second embodiment will be described focusing on differences from the first embodiment.

<Functional configuration of display control device>
FIG. 9 is a diagram illustrating an example of a functional configuration of the viewpoint conversion unit of the display control apparatus according to the second embodiment. The difference from FIG. 7 is that a left instruction receiving unit 901, a left image data converting unit 902, a right instruction receiving unit 911, and a right image data converting unit 912 are provided.

  The left instruction receiving unit 901 receives an instruction to change the viewpoint direction for the left display image data from the driver 300 of the vehicle 200. Further, the left instruction receiving unit 901 notifies the received instruction to the left image data converting unit 902.

  The left image data conversion unit 902 reads the conversion information 510 from the conversion information storage unit 122, and converts the coordinates of each pixel of the left image data based on the read conversion information 510. The conversion information storage unit 122 stores a plurality of pieces of conversion information having different “viewpoint directions” of virtual images, and the left image data conversion unit 702 responds to an instruction notified from the left instruction receiving unit 901. Read conversion information having the “viewpoint direction” of the virtual image.

  The right instruction receiving unit 911 receives an instruction to change the viewpoint direction for the right display image data from the driver 300 of the vehicle 200. Further, the right instruction receiving unit 911 notifies the right image data converting unit 912 of the received instruction.

  The right image data conversion unit 912 reads the conversion information 520 from the conversion information storage unit 122, and converts the coordinates of each pixel of the right image data based on the read conversion information 520. The conversion information storage unit 122 stores a plurality of pieces of conversion information having different “viewpoint directions” of the virtual image, and the right image data conversion unit 912 responds to the instruction notified from the right instruction reception unit 911. Read conversion information having the “viewpoint direction” of the virtual image.

<Flow of viewpoint conversion processing by viewpoint converter>
Next, a flow of viewpoint conversion processing by the viewpoint conversion unit 121 of the display control device 120 will be described. FIG. 10 is a flowchart illustrating a flow of viewpoint conversion processing by the viewpoint conversion unit of the display control apparatus according to the second embodiment.

  Among these, FIG. 10A is a flowchart showing the flow of viewpoint conversion processing for the left image data, and the difference from FIG. 8A is step S1001. In step S <b> 1001, the left instruction receiving unit 901 determines whether or not there is an instruction from the driver 300 of the vehicle 200 to change the left viewpoint direction. If it is determined in step S1001 that there is no instruction (No in step S1001), the process proceeds to step S805.

  On the other hand, if it is determined in step S1001 that there is an instruction (in the case of Yes in step S1001), the instruction is accepted, notified to the left image data conversion unit 902, and the process returns to step S801. Thereby, the left image data conversion unit 902 reads the conversion information having the “viewpoint direction” of the virtual image from the conversion information storage unit 122 in accordance with the notified instruction. As a result, the left image data conversion unit 902 can realize conversion according to an instruction from the driver 300 of the vehicle 200 for the left image data.

  FIG. 10B is a flowchart showing the flow of viewpoint conversion processing for the right image data, and the difference from FIG. 8B is step S1011. In step S <b> 1011, the right instruction receiving unit 911 determines whether or not there is an instruction from the driver 300 of the vehicle 200 to change the right viewpoint direction. If it is determined in step S1011 that there is no instruction (in the case of No in step S1011), the process proceeds to step S815.

  On the other hand, if it is determined in step S1011 that there is an instruction (in the case of Yes in step S1011), the instruction is accepted, notified to the right image data conversion unit 912, and the process returns to step S811. As a result, the right image data conversion unit 912 reads conversion information having the “viewpoint direction” of the virtual image from the conversion information storage unit 122 in accordance with the notified instruction. As a result, the right image data conversion unit 912 can realize conversion according to an instruction from the driver 300 of the vehicle 200 for the right image data.

<Summary>
As is clear from the above description, the display control apparatus in the second embodiment is
The display image data displayed by the display device is based on the positional relationship (viewpoint position, viewpoint direction) between the position where the image device is visually recognized through the optical member and the position where the imaging device is installed. The coordinates of each pixel are converted.
At that time, the coordinates of each pixel of the image data are converted based on the positional relationship (viewpoint direction) according to the instruction of the driver of the vehicle.

  Thereby, according to the display control apparatus in 2nd Embodiment, it becomes possible to implement | achieve the process similar to operating the mirror surface part of a side mirror in a general vehicle in an electronic side mirror.

[Third Embodiment]
In the first and second embodiments, the camera monitoring system in which two imaging devices are installed in the vehicle 200 has been described. However, the system configuration of the camera monitoring system is not limited to this. Hereinafter, the system configuration of another camera monitoring system will be described with reference to FIGS. 11 and 12.

  FIG. 11 is a diagram illustrating an example of a system configuration of a camera monitoring system according to the third embodiment. FIG. 12 is a diagram illustrating a position where the imaging device is installed and a shooting range of the imaging device in the camera monitoring system according to the third embodiment.

  FIG. 11A shows a state in which one imaging device 1101 is installed at the rear center position of the vehicle 200 as shown in FIG. 12A and the viewpoint conversion processing is realized in the camera monitoring system 1100. ing. In this case, the camera monitoring system 1100 functions as an electronic room mirror.

  As illustrated in FIG. 12A, the imaging device 1101 performs imaging using a shaded area 1210 behind the vehicle 200 as an imaging range. Then, based on the central image data captured by the imaging device 1101, the display control device 120 generates central display image data, and the display device 1102 displays the central display image data. The central display image data displayed by the display device 1102 is visually recognized by the driver at a central position in the passenger compartment as a virtual image via the optical member 1103.

  On the other hand, in FIG. 11B, as shown in FIG. 12B, the three imaging devices 110, 111, and 1101 are installed at the position of the side mirror or the rear center position of the vehicle 200, and the viewpoint conversion process is performed. This shows how the camera monitoring system 1110 is realized. The camera monitoring system 1110 functions as an electronic side mirror.

  As illustrated in FIG. 12B, the imaging device 1101 performs imaging using an area outside the imaging range of the imaging devices 110 and 111 as an imaging range. For this reason, the display control device 120 combines the left image data captured by the imaging device 110 with the central image data captured by the imaging device 1101 (the central image data of an area outside the imaging range of the imaging device 110). Thus, the left display image data is generated.

  Similarly, the display control device 120 combines the right image data captured by the imaging device 111 with the central image data captured by the imaging device 1101 (central image data in an area outside the imaging range of the imaging device 110). Thus, right display image data is generated. Thereby, according to the camera monitoring system 1110, the driver 300 of the vehicle 200 can visually recognize the left display image data and the right display image data including the area outside the imaging range of the imaging devices 110 and 111 as virtual images. .

[Other Embodiments]
In the first to third embodiments, in order to absorb the difference between the position of the virtual image visually recognized by the driver and the position where the imaging device is installed, the coordinates of each pixel of the image data are converted. explained. However, the method for absorbing the difference between the position of the virtual image visually recognized by the driver and the position where the imaging device is installed is not limited to this, and the position of the virtual image visually recognized by the driver and the imaging device You may design an optical member so that the position to install may correspond. This eliminates the need to convert the coordinates of each pixel of the image data.

  Note that the present invention is not limited to the configurations shown here, such as combinations with other elements, etc., in the configurations described in the above embodiments. These points can be changed without departing from the spirit of the present invention, and can be appropriately determined according to the application form.

100: Camera monitoring system 110, 111: Imaging device 120: Display control device 121: View point conversion unit 130, 131: Display device 140, 141: Optical member 200: Vehicle 300: Driver 301, 302: Concave mirror 400, 401: Virtual image 510, 520: Conversion information 701: Left image data acquisition unit 702: Left image data conversion unit 703: Left display image data generation unit 711: Right image data acquisition unit 712: Right image data conversion unit 713: Right display image data generation unit 901: Left instruction receiving unit 902: Left image data converting unit 911: Right instruction receiving unit 912: Right image data converting unit

Claims (5)

A display control device connected to an imaging device installed in a vehicle,
A conversion unit that converts coordinates of each pixel of image data captured by the imaging device;
An output unit that outputs display image data generated by converting coordinates by the conversion unit to the display unit,
The converter is
The image captured by the imaging device based on the positional relationship between the position where the display image data displayed on the display unit is visually recognized as a virtual image via an optical member and the position where the imaging device is installed A display control device that converts coordinates of each pixel of data. A storage unit for storing conversion information indicating the positional relationship;
The display control apparatus according to claim 1, wherein the conversion unit reads the conversion information from the storage unit and converts coordinates of each pixel of image data captured by the imaging device. A reception unit that receives an instruction to change the positional relationship;
The display control apparatus according to claim 2, wherein the conversion unit reads the conversion information corresponding to the instruction received by the reception unit from the storage unit. A display control method of a display control device connected to an imaging device installed in a vehicle,
A conversion step of converting coordinates of each pixel of image data captured by the imaging device;
An output step of outputting display image data generated by converting the coordinates in the conversion step to a display unit,
The conversion step includes
The image captured by the imaging device based on the positional relationship between the position where the display image data displayed on the display unit is visually recognized as a virtual image via an optical member and the position where the imaging device is installed A display control method characterized by converting coordinates of each pixel of data. An imaging device for photographing the rear of the vehicle;
A display control device connected to the imaging device;
A display device for displaying display image data output from the display control device;
An optical member that reflects light emitted from the display device, and a camera monitoring system comprising:
The display control device includes:
A conversion unit that converts coordinates of each pixel of image data captured by the imaging device;
An output unit that outputs display image data generated by converting the coordinates by the conversion unit to the display device;
The converter is
The display image data displayed on the display device is photographed by the imaging device based on a positional relationship between a position visually recognized as a virtual image through the optical member and a position where the imaging device is installed. A camera monitoring system characterized by converting the coordinates of each pixel of image data.

Images