WO2019220557A1 - Processeur d'image pour véhicules et procédé de traitement d'image - Google Patents

Processeur d'image pour véhicules et procédé de traitement d'image Download PDF

Info

Publication number
WO2019220557A1
WO2019220557A1 PCT/JP2018/018899 JP2018018899W WO2019220557A1 WO 2019220557 A1 WO2019220557 A1 WO 2019220557A1 JP 2018018899 W JP2018018899 W JP 2018018899W WO 2019220557 A1 WO2019220557 A1 WO 2019220557A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
mirror
optical mirror
electronic
optical
Prior art date
Application number
PCT/JP2018/018899
Other languages
English (en)
Japanese (ja)
Inventor
下谷　光生
中村　好孝
克治 淺賀
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to PCT/JP2018/018899 priority Critical patent/WO2019220557A1/fr
Priority to JP2020518871A priority patent/JP6921316B2/ja
Publication of WO2019220557A1 publication Critical patent/WO2019220557A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/28—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with an adjustable field of view
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/25—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view to the sides of the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/26—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view to the rear of the vehicle
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention relates to image processing of an image for an optical mirror in a combined system of an electronic mirror and an optical mirror of a vehicle.
  • the range that can be seen through the optical mirror from the driver of the vehicle (hereinafter referred to as “the field of view of the optical mirror”) is different from the range of the image captured by the camera for the electronic mirror (hereinafter referred to as “electronic mirror image”). Therefore, when an optical mirror and an electronic mirror are used in combination, it may be difficult to instantly grasp the relationship between the mirror image reflected on the optical mirror and the electronic mirror image.
  • the electronic mirror image is enlarged on the display device so that the size of the object reflected in the electronic mirror image is equal to the size of the object reflected in the optical mirror.
  • a technique for displaying images is proposed.
  • the range of the electronic mirror image displayed on the display device is narrowed, so that the advantage of the electronic mirror that a high-angle image can be displayed is impaired.
  • the present invention has been made to solve the above problems, and in a vehicle mirror system using both an optical mirror and an electronic mirror, a mirror image reflected on the optical mirror and an electronic mirror image displayed on the screen are provided.
  • the purpose is to make it easy for the driver to grasp the relationship.
  • An image processing apparatus for a vehicle includes an electronic mirror image acquisition unit that acquires an electronic mirror image captured by a camera for an electronic mirror of a vehicle, and an optical mirror direction acquisition unit that acquires information on the direction of the optical mirror of the vehicle. Based on the orientation of the optical mirror, an optical mirror field identification unit that identifies an area corresponding to the field of view of the optical mirror, which is a range that can be seen through the optical mirror from the vehicle driver, in the electronic mirror image; A synthesis processing unit that synthesizes an optical mirror visual field image that is an image showing a region corresponding to a mirror visual field, and a display processing unit that displays an electronic mirror image obtained by synthesizing the optical mirror visual field image on a display device of the vehicle. Is.
  • an image (optical mirror field image) indicating a region corresponding to the field of view of the optical mirror is synthesized with the electronic mirror image displayed on the display device of the vehicle.
  • the relationship between the mirror image reflected on the mirror and the electronic mirror image displayed on the screen of the display device can be clearly shown to the driver.
  • FIG. 1 is a block diagram showing a configuration of a vehicle mirror system according to Embodiment 1.
  • FIG. 6 is a diagram illustrating an example of a screen displayed on the display device by the image processing apparatus according to Embodiment 1.
  • FIG. 6 is a diagram illustrating an example of a screen displayed on the display device by the image processing apparatus according to Embodiment 1.
  • FIG. 3 is a flowchart illustrating an operation of the image processing apparatus according to the first embodiment.
  • 6 is a diagram illustrating an example of a screen displayed on the display device by the image processing apparatus according to Embodiment 1.
  • FIG. 6 is a diagram illustrating an example of a screen displayed on the display device by the image processing apparatus according to Embodiment 1.
  • FIG. 6 is a diagram illustrating an example of a screen displayed on the display device by the image processing apparatus according to Embodiment 1.
  • FIG. 6 is a diagram illustrating an example of a screen displayed on the display device by the image processing apparatus according to Embodiment 1.
  • FIG. 6 is a diagram illustrating an example of a screen displayed on the display device by the image processing apparatus according to Embodiment 1.
  • FIG. 6 is a diagram illustrating an example of a screen displayed on the display device by the image processing apparatus according to Embodiment 1.
  • FIG. 6 is a diagram illustrating an example of a screen displayed on the display device by the image processing apparatus according to Embodiment 1.
  • FIG. 10 is a diagram for illustrating a first modification of the first embodiment.
  • FIG. 10 is a diagram for illustrating a second modification of the first embodiment.
  • FIG. 10 is a diagram for explaining a third modification of the first embodiment.
  • FIG. 10 is a diagram illustrating a first modification of the first embodiment.
  • FIG. 10 is a diagram for illustrating a second modification of the first embodiment.
  • FIG. 10 is a diagram for explaining a third modification of the
  • FIG. 10 is a diagram for illustrating a fourth modification of the first embodiment.
  • FIG. 10 is a diagram for illustrating a fourth modification of the first embodiment.
  • FIG. 10 is a diagram for illustrating a fifth modification example of the first embodiment.
  • FIG. 10 is a diagram for illustrating a fifth modification example of the first embodiment.
  • FIG. 10 is a diagram for illustrating a sixth modification example of the first embodiment.
  • FIG. 25 is a diagram for illustrating a seventh modification example of the first embodiment. It is a figure which shows the example of the hardware constitutions of an image processing apparatus. It is a figure which shows the example of the hardware constitutions of an image processing apparatus.
  • FIG. 5 is a block diagram illustrating a configuration of a vehicle mirror system according to a second embodiment.
  • FIG. 10 is a diagram for explaining the operation of the image processing apparatus according to Embodiment 2.
  • FIG. 10 is a diagram for explaining the operation of the image processing apparatus according to Embodiment 2.
  • FIG. 6 is a flowchart illustrating an operation of the image processing apparatus according to the second embodiment.
  • FIG. 10 is a diagram for illustrating a first modification example of the second embodiment.
  • FIG. 10 is a diagram for illustrating a first modification example of the second embodiment.
  • FIG. 10 is a diagram for describing a second modification of the second embodiment.
  • FIG. 10 is a diagram for explaining the operation of the image processing apparatus according to Embodiment 2.
  • FIG. 6 is a block diagram illustrating a configuration of a vehicle mirror system according to a third embodiment.
  • FIG. 10 is a diagram illustrating an example of a screen displayed on a display device by an image processing apparatus according to a third embodiment.
  • FIG. 10 is a diagram illustrating an example of a screen displayed on a display device by an image processing apparatus according to a third embodiment.
  • FIG. 10 is a diagram illustrating an example of a screen displayed on a display device by an image processing apparatus according to a third embodiment.
  • FIG. 10 is a diagram for explaining a modification of the third embodiment.
  • FIG. 10 is a diagram for explaining a modification of the third embodiment.
  • FIG. 1 is a block diagram showing the configuration of the vehicle mirror system according to the first embodiment.
  • This vehicle mirror system is a system using both an optical mirror and an electronic mirror.
  • the vehicle mirror system includes a vehicle image processing apparatus 10 (hereinafter simply referred to as “image processing apparatus 10”), an electronic mirror camera 21, an optical mirror 22, and an optical mirror adjustment connected thereto.
  • a device 23 and a display device 24 are provided.
  • a vehicle equipped with the vehicle mirror system is referred to as “own vehicle”, and vehicles other than the own vehicle are referred to as “other vehicles”.
  • the optical mirror 22 is a physical mirror such as a side mirror or a rear view mirror that allows the driver to visually check the rear or side of the vehicle.
  • the electronic mirror camera 21 is an imaging device that captures an image for an electronic mirror (an electronic mirror image).
  • the range captured by the electronic mirror camera 21 includes the field of view of the optical mirror, which is the range that can be seen through the optical mirror 22 from the driver. In other words, the electronic mirror camera 21 captures the rear or side of the vehicle with a wider angle of view than the field of view of the optical mirror.
  • An electronic mirror image captured by the electronic mirror camera 21 is input to the image processing apparatus 10.
  • the optical mirror 22 is a left side mirror of the vehicle, and the electronic mirror camera 21 captures the left rear side of the vehicle.
  • the optical mirror adjustment device 23 is an electric drive device (for example, a remote controller) of the optical mirror 22.
  • the driver can adjust the direction of the optical mirror 22 by operating the optical mirror adjusting device 23.
  • the optical mirror adjustment device 23 stores the current orientation of the optical mirror 22, and the image processing device 10 can acquire information on the orientation of the optical mirror 22 from the optical mirror adjustment device 23.
  • the display device 24 is for displaying an electronic mirror image acquired from the electronic mirror camera 21 by the image processing device 10, and is configured by, for example, a liquid crystal display device.
  • the installation location of the display device 24 is preferably a position that can be easily seen by the driver.
  • an instrument panel or a center panel of the host vehicle can be considered.
  • the image processing apparatus 10 displays the electronic mirror image acquired from the electronic mirror camera 21 on the display device 24.
  • the image processing apparatus 10 includes an electronic mirror image acquisition unit 11, an optical mirror direction acquisition unit 12, an optical mirror visual field identification unit 13, a composition processing unit 14, and a display processing unit 15.
  • the electronic mirror image acquisition unit 11 acquires an electronic mirror image taken by the electronic mirror camera 21.
  • the optical mirror direction acquisition unit 12 acquires information on the orientation of the optical mirror 22 from the optical mirror adjustment device 23.
  • the optical mirror visual field specifying unit 13 stores a photographing range of the electronic mirror camera 21 and specifies an area corresponding to the visual field of the optical mirror 22 in the electronic mirror image based on the direction of the optical mirror 22.
  • the synthesis processing unit 14 includes an image indicating an area corresponding to the field of view of the optical mirror 22 identified by the optical mirror field identification unit 13 in the electronic mirror image acquired by the electronic mirror image acquisition unit 11 (hereinafter referred to as “optical mirror field image”). ).
  • the display processing unit 15 causes the display device 24 to display an electronic mirror image obtained by synthesizing the optical mirror visual field image.
  • FIG. 2 shows an example of a screen displayed on the display device 24 by the image processing apparatus 10.
  • the display apparatus 24 shall be arrange
  • an optical mirror 22 which is a left side mirror is shown at the left end of FIG.
  • the display device 24 displays an electronic mirror image 40 in which an optical mirror field image 41 indicating a region corresponding to the field of view of the optical mirror 22 is combined.
  • an optical mirror field image 41 indicating a region corresponding to the field of view of the optical mirror 22 is combined.
  • the body mirror side surface of the host vehicle V0 and the other vehicles V1, V2 are reflected in the electronic mirror image 40 displayed on the display device 24.
  • the inner part of the optical mirror visual field image 41 in the electronic mirror image 40 displayed on the display device 24 corresponds to the mirror image 221 reflected on the optical mirror 22. Therefore, the driver can easily grasp the relationship between the electronic mirror image 40 displayed on the display device 24 and the mirror image 221 reflected on the optical mirror 22 from the position of the optical mirror visual field image 41.
  • the optical mirror direction acquisition unit 12 continuously acquires information on the orientation of the optical mirror 22 from the optical mirror adjustment device 23, and the optical mirror visual field specification unit 13 includes the optical mirror 22 in the electronic mirror image.
  • the area corresponding to the field of view shall be specified continuously.
  • the composition processing unit 14 changes the position where the optical mirror visual field image is composited with the electronic mirror image in accordance with the visual field movement of the optical mirror 22.
  • the optical mirror visual field image 41 is displayed on the screen of the display device 24 as shown in FIG. Moves to the left. Therefore, the optical mirror visual field image 41 can show a region corresponding to the visual field of the optical mirror 22 in the electronic mirror image 40 in real time.
  • the electronic mirror image 40 shown in FIGS. 2 and 3 is merely an example, and the electronic mirror image 40 may be any image.
  • the electronic mirror image 40 may be a translucent image having the shape of a side mirror housing.
  • the shooting range of the electronic mirror camera 21 is wider than the field of view of the optical mirror. However, if the field of view of the optical mirror is partially included, the field of view of the optical mirror is narrower. May be.
  • FIG. 4 is a flowchart showing the operation of the image processing apparatus 10 according to the first embodiment. Hereinafter, the operation of the image processing apparatus 10 will be described with reference to FIG.
  • the optical mirror direction acquisition unit 12 acquires information on the orientation of the optical mirror 22 from the optical mirror adjustment apparatus 23 (step ST101).
  • the electronic mirror image acquisition part 11 acquires the electronic mirror image image
  • the optical mirror visual field specifying unit 13 specifies an area corresponding to the visual field of the optical mirror 22 in the electronic mirror image acquired in step ST102 based on the orientation of the optical mirror 22 acquired in step ST101 (step ST103). Further, the synthesis processing unit 14 synthesizes an optical mirror field image indicating a region corresponding to the field of view of the optical mirror 22 identified in step ST103, with an electronic mirror image (step ST104). And the display process part 15 displays the electronic mirror image with which the optical mirror visual field image was synthesize
  • steps ST101 to ST105 are repeated. Thereby, the electronic mirror image displayed on the display device 24 and the optical mirror field image 41 synthesized therewith are updated in real time.
  • the display processing unit 15 performs an electronic mirror image on the inner side and the outer side of the optical mirror field image 41 as shown in FIG. Forty display modes (for example, brightness and hue) may be differentiated.
  • a part of the optical mirror field image 41 may protrude from the screen of the display device 24 as shown in FIGS. 6 and 7.
  • FIGS. 6 and 7 it is conceivable that the entire optical mirror field image 41 protrudes from the screen of the display device 24 (the optical mirror field image 41 is not displayed on the display device 24). In that case, it is preferable to display on the display device 24 a warning image 42 indicating that the entire optical mirror field image 41 protrudes from the screen of the display device 24.
  • the optical mirror visual field image 41 can be displayed at a position protruding from the electronic mirror image 40.
  • meters 31 and 32 are images displayed on the display device 24 as the instrument panel 30.
  • the optical mirror field image 41 is superimposed on the image of the meter 31 as shown in FIG. 9, the visibility of the meter 31 deteriorates. Therefore, the display position of the image of the meter 31 is the optical mirror field image 41 as shown in FIG. May be automatically moved so as not to overlap.
  • step ST101 the process in which the optical mirror direction acquisition unit 12 of the image processing apparatus 10 acquires the orientation of the optical mirror 22 (step ST101) only needs to be performed once at the time of activation. Therefore, as shown in the flowchart of FIG. 11, after step ST105, the process returns to step ST102, step ST101 is executed only when the image processing apparatus 10 is started, and thereafter steps ST102 to ST105 are repeatedly executed. May be.
  • the direction of the side mirror of the vehicle is adjusted so that the vehicle body appears in the range of about 1/4 from the end of the side mirror and the road surface appears in the range of about 1/2 or 2/3 from the bottom of the side mirror. It is standard.
  • the composition processing unit 14 synthesizes the optical mirror standard visual field image 43 showing the standard visual field of the optical mirror 22 with the optical mirror visual field image 41 into the electronic mirror image 40.
  • the driver can easily set the orientation of the optical mirror 22 to the standard orientation by adjusting the orientation of the optical mirror 22 so that the optical mirror view image 41 matches the optical mirror standard view image 43.
  • FIG. 14 An example in which the electronic mirror camera 21, the optical mirror 22, and the display device 24 are connected to the image processing apparatus 10 one by one is shown.
  • a left side mirror 22 ⁇ / b> L and a right side mirror 22 ⁇ / b> R are connected to the image processing apparatus 10 via an optical mirror adjustment device 23 as an optical mirror 22, and a left electronic mirror camera 21 ⁇ / b> L is connected as an electronic mirror camera 21.
  • the electronic mirror image acquisition unit 11 acquires electronic mirror images taken by the left electronic mirror camera 21L and the right electronic mirror camera 21R.
  • the electronic mirror images taken by the left electronic mirror camera 21L and the right electronic mirror camera 21R are referred to as “left electronic mirror image” and “right electronic mirror image”, respectively.
  • the optical mirror direction acquisition unit 12 acquires information on the orientation of the left side mirror 22L and the right side mirror 22R from the optical mirror adjustment device 23.
  • the optical mirror visual field specification unit 13 Based on the orientations of the left side mirror 22L and the right side mirror 22R, the optical mirror visual field specification unit 13 and the region corresponding to the field of view of the left side mirror 22L in the left electronic mirror image and the right side mirror 22R in the right electronic mirror image An area corresponding to the visual field is specified.
  • the synthesis processing unit 14 synthesizes an image showing a region corresponding to the field of view of the left side mirror 22L (hereinafter referred to as a “left side mirror field of view image”) with the left electronic mirror image 40L, and further, An image indicating an area corresponding to the field of view of the right side mirror 22R (hereinafter referred to as “right side mirror field of view image”) is synthesized.
  • the display processing unit 15 causes the left display device 24L to display the left electronic mirror image 40L obtained by combining the left side mirror view image 41L, and further displays the right side image obtained by combining the right side mirror view image 41R.
  • the electronic mirror image 40R is displayed on the right display device 24R.
  • the driver determines the relationship between the left electronic mirror image 40L and the mirror image reflected on the left side mirror 22L, the right electronic mirror image 40R, and the right side mirror 22R from the positions of the left side mirror visual field image 41L and the right side mirror visual field image 41R. It is possible to easily grasp the relationship with the mirror image reflected on the screen.
  • FIG. 16 shows a configuration in which a rear view mirror 22B is further introduced as the optical mirror 22 and a rear electronic mirror camera 21B for photographing the rear side of the host vehicle is introduced as the electronic mirror camera 21 with respect to the configuration of FIG. However, only one display device 24 having a wide screen is connected.
  • the electronic mirror image acquisition unit 11 acquires electronic mirror images taken by the left electronic mirror camera 21L, the right electronic mirror camera 21R, and the rear electronic mirror camera 21B.
  • an electronic mirror image taken by the rear electronic mirror camera 21B is referred to as a “rear electronic mirror image”.
  • the optical mirror direction acquisition unit 12 acquires information on the orientation of the left side mirror 22L, the right side mirror 22R, and the rear view mirror 22B from the optical mirror adjustment device 23.
  • the composition processing unit 14 synthesizes the left electronic mirror image, the right electronic mirror image, and the rear electronic mirror image to generate one panoramic image. Further, the optical mirror visual field specifying unit 13 determines a region corresponding to the visual field of the left side mirror 22L in the panoramic image based on the orientations of the left side mirror 22L, the right side mirror 22R, and the rear view mirror 22B, and And an area corresponding to the field of view of the rear view mirror 22B.
  • composition processing unit 14 adds an image (left side mirror field image) indicating an area corresponding to the field of view of the left side mirror 22L and an area corresponding to the field of view of the right side mirror 22R to the electronic mirror image that is a panoramic image.
  • image shown (right side mirror field image) and the image indicating the area corresponding to the field of view of the rear view mirror 22B are synthesized.
  • the display processing unit 15 displays an electronic mirror image 40P, which is a panoramic image obtained by synthesizing the left side mirror visual field image 41L, the right side mirror visual field image 41R, and the rear view mirror visual field image 41B, on a wide screen. It is displayed on the display device 24 it has.
  • the driver uses the electronic mirror image 40P, the left side mirror 22L, and the right side mirror. It is possible to easily grasp the relationship between the mirror image reflected on each of the 22R and the rear view mirror 22B.
  • FIG. 18 is a block diagram illustrating a configuration of a vehicle mirror system according to a sixth modification of the first embodiment. This vehicle mirror system is obtained by adding an eye point estimation unit 16 as a component of the image processing apparatus 10 to the configuration of FIG.
  • the eye point estimation unit 16 estimates the position (eye point) of the driver's eyes with respect to the optical mirror 22. Any method may be used for the eye point estimation unit 16 to estimate the position of the driver's eyes. For example, a method in which an image obtained by photographing the driver's face with a camera installed in the driver's seat is analyzed and estimated may be considered. Further, for example, a simple one that estimates the position of the driver's eyes from the position of the seat of the driver's seat (the position and height of the seat surface, the inclination angle of the backrest, etc.) may be used.
  • the optical mirror visual field specifying unit 13 specifies an area corresponding to the visual field of the optical mirror 22 in the electronic mirror image based on the eye position of the driver estimated by the eye point estimating unit 16. Thereby, an optical mirror field image can be formed at a more accurate position in the electronic mirror image.
  • optical mirrors 22 are not compatible with the optical mirror adjusting device 23.
  • the optical mirror adjusting device 23 For example, as for a rear view mirror, what a driver adjusts direction by hand is common.
  • an optical mirror direction detection sensor 25 which is a sensor for detecting the orientation of the optical mirror 22, is introduced as shown in FIG. However, it is good to comprise so that the information of the direction of the optical mirror 22 may be acquired from the optical mirror direction detection sensor 25.
  • FIG. 1 are diagrams illustrating examples of the hardware configuration of the image processing apparatus 10, respectively.
  • Each function of the components of the image processing apparatus 10 illustrated in FIG. 1 is realized by, for example, the processing circuit 50 illustrated in FIG. That is, the image processing apparatus 10 acquires an electronic mirror image captured by the electronic mirror camera of the vehicle, acquires information on the direction of the optical mirror of the vehicle, and based on the direction of the optical mirror, the vehicle in the electronic mirror image
  • the area corresponding to the field of view of the optical mirror which is the range that can be seen through the optical mirror from the driver of the driver, is identified, and the optical mirror field image, which is an image showing the area corresponding to the field of view of the optical mirror, is synthesized with the electronic mirror image.
  • a processing circuit 50 is provided for causing the display device of the vehicle to display an electronic mirror image in which the mirror view image is synthesized.
  • the processing circuit 50 may be dedicated hardware, or a processor (a central processing unit (CPU), a processing device, an arithmetic device, a microprocessor, a microcomputer, or the like) that executes a program stored in a memory. It may be configured using a DSP (Digital Signal Processor).
  • DSP Digital Signal Processor
  • the processing circuit 50 When the processing circuit 50 is dedicated hardware, the processing circuit 50 includes, for example, a single circuit, a composite circuit, a programmed processor, a processor programmed in parallel, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable). Gate Array) or a combination of these.
  • the functions of the components of the image processing apparatus 10 may be realized by individual processing circuits, or the functions may be realized by a single processing circuit.
  • FIG. 21 shows an example of the hardware configuration of the image processing apparatus 10 when the processing circuit 50 is configured using a processor 51 that executes a program.
  • the functions of the components of the image processing apparatus 10 are realized by software or the like (software, firmware, or a combination of software and firmware).
  • Software or the like is described as a program and stored in the memory 52.
  • the processor 51 reads out and executes the program stored in the memory 52, thereby realizing the function of each unit. That is, the image processing apparatus 10, when executed by the processor 51, obtains an electronic mirror image taken by the vehicle electronic mirror camera, obtains information on the orientation of the optical mirror of the vehicle, and optical.
  • processing for identifying a region in the electronic mirror image corresponding to the field of view of the optical mirror which is a range visible through the optical mirror from the driver of the vehicle, and corresponding to the field of view of the optical mirror in the electronic mirror image
  • a program in which processing for combining an optical mirror field image that is an image showing a region and processing for displaying an electronic mirror image combined with the optical mirror field image on a display device of a vehicle is executed as a result.
  • a memory 52 for storing is provided. In other words, it can be said that this program causes the computer to execute the operation procedure and method of the components of the image processing apparatus 10.
  • the memory 52 is, for example, non-volatile or RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), or the like. Volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disk) and its drive device, etc., or any storage media used in the future May be.
  • RAM Random Access Memory
  • ROM Read Only Memory
  • flash memory EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), or the like.
  • Volatile semiconductor memory Volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disk) and its drive device, etc., or any storage media used in the future May be.
  • the present invention is not limited to this, and a configuration in which some components of the image processing apparatus 10 are realized by dedicated hardware and other components are realized by software or the like.
  • the functions of some components are realized by the processing circuit 50 as dedicated hardware, and the programs stored in the memory 52 are stored in the memory 52 by the processing circuit 50 as the processor 51 for other components.
  • the function can be realized by reading and executing.
  • the image processing apparatus 10 can realize the functions described above by hardware, software, or the like, or a combination thereof.
  • FIG. 22 is a block diagram illustrating a configuration of the vehicle mirror system according to the second embodiment.
  • the configuration of the vehicle mirror system in FIG. 22 is obtained by connecting an operation input device 26 to the image processing apparatus 10 and adding an electronic mirror adjusting unit 17 as a component of the image processing apparatus 10 to the configuration in FIG. is there.
  • FIG. 23 shows an example of the relationship between the shooting range ⁇ of the electronic mirror camera 21 and the display area ⁇ displayed on the display device 24.
  • FIG. 24 shows an electronic mirror image 40 taken by the electronic mirror camera 21, a display area 40a of the electronic mirror image 40, and an area corresponding to the field of view of the optical mirror (display position of the optical mirror field image 41). An example of the relationship is shown.
  • the electronic mirror adjustment unit 17 performs trimming to cut out the display area 40a to be displayed on the display device 24 from the electronic mirror image 40 captured by the electronic mirror camera 21.
  • the operation input device 26 is a user interface for inputting an operation for adjusting the range (that is, the positions of ⁇ 1 and ⁇ 2 in FIG. 24) that the electronic mirror adjustment unit 17 trims from the electronic mirror image 40.
  • the driver can adjust the position of the display area 40 a in the electronic mirror image 40 by operating the operation input device 26. Thereby, substantially the same effect as the adjustment of the photographing direction of the electronic mirror camera 21 can be obtained. For example, as shown in FIG. 25, when the driver moves the display area 40a in the electronic mirror image 40 to the left, the display area 40a of the electronic mirror image 40 displayed on the display device 24 as shown in FIG. Shifts to the right.
  • FIG. 27 is a flowchart showing the operation of the image processing apparatus 10 according to the second embodiment. Hereinafter, the operation of the image processing apparatus 10 will be described with reference to FIG.
  • the optical mirror direction acquisition unit 12 acquires information on the orientation of the optical mirror 22 from the optical mirror adjustment apparatus 23 (step ST201).
  • the electronic mirror image acquisition unit 11 acquires an electronic mirror image captured by the electronic mirror camera 21 (step ST202).
  • the optical mirror visual field specifying unit 13 specifies an area corresponding to the visual field of the optical mirror 22 in the electronic mirror image acquired in step ST202 based on the orientation of the optical mirror 22 acquired in step ST201 (step ST203). Further, the composition processing unit 14 synthesizes an optical mirror field image indicating a region corresponding to the field of view of the optical mirror 22 specified in step ST203, with an electronic mirror image (step ST204).
  • the electronic mirror adjustment unit 17 performs trimming to cut out the display area from the electronic mirror image obtained by synthesizing the electronic mirror image (step ST205). Then, the display processing unit 15 causes the display device 24 to display the trimmed electronic mirror image display area (step ST206).
  • steps ST201 to ST206 are repeated. Thereby, the display area of the electronic mirror image displayed on the display device 24 and the optical mirror visual field image 41 synthesized therewith are updated in real time. Note that the position of the display region of the electronic mirror image trimmed in step ST205 can be adjusted by the driver using the operation input device 26.
  • the display processing unit 15 may set the position of the display area 40 a of the electronic mirror image 40 according to the direction of the optical mirror 22. Specifically, when the driver operates the optical mirror adjustment device 23 to change the direction of the optical mirror 22, the display area 40 a of the electronic mirror image 40 moves following the movement of the visual field of the optical mirror 22. Like that. That is, as shown in FIG. 28, the position of the display area 40 a in the electronic mirror image 40 moves together with the optical mirror field image 41. In this case, as the position of the display area 40a in the electronic mirror image 40 changes, the image in the display area 40a shifts as shown in FIG. 29, but the position of the optical mirror visual field image 41 on the screen of the display device 24. Does not change.
  • the display area in the electronic mirror image 40 is always included in the display area 40a of the electronic mirror image 40 so that the entire area corresponding to the visual field of the optical mirror 22 (optical mirror visual field image 41) is always included.
  • An example in which the position 40a is set is shown. However, the position of the display area 40a may be set so that at least a part of the area corresponding to the field of view of the optical mirror 22 is included in the display area 40a.
  • FIG. 30 is a block diagram showing a configuration of a vehicle mirror system according to a second modification of the second embodiment. This vehicle mirror system is obtained by adding an optical mirror direction control unit 18 as a component of the image processing apparatus 10 to the configuration of FIG.
  • the optical mirror direction control unit 18 controls the direction of the optical mirror 22 in accordance with the position of the display area 40a in the electronic mirror image 40. Specifically, when the driver operates the operation input device 26 to move the display area 40a, the optical mirror direction control unit 18 follows the display area 40a and corresponds to the field of view of the optical mirror 22. The direction of the optical mirror 22 is set so that (the display position of the optical mirror visual field image 41) moves.
  • the relationship between the display area 40a in the electronic mirror image 40 and the area corresponding to the field of view of the optical mirror 22 is the same as that shown in FIGS. 28 and 29 shown in the first modification. It will be the same.
  • the display area 40a of the electronic mirror image 40 moves following the optical mirror visual field image 41.
  • the optical mirror visual field image 41 is displayed in the display area of the electronic mirror image 40. It moves following 40a.
  • the orientation of the optical mirror 22 is set so that the entire region corresponding to the field of view of the optical mirror 22 (the optical mirror field image 41) is always included in the display region 40a of the electronic mirror image 40. ing. However, the orientation of the optical mirror 22 may be set so that at least a part of the region corresponding to the field of view of the optical mirror 22 is included in the display region 40 a of the electronic mirror image 40.
  • FIG. 31 is a block diagram illustrating a configuration of the vehicle mirror system according to the third embodiment.
  • the configuration of the vehicle mirror system in FIG. 31 is obtained by adding a watch-out object detector 19 as a component of the image processing apparatus 10 to the configuration in FIG.
  • the object-of-interest detection unit 19 detects an object that may affect the traveling of the host vehicle from among objects reflected in the electronic mirror image, that is, an object existing within the shooting range of the electronic mirror camera 21.
  • an object that may affect the traveling of the host vehicle is referred to as an “interesting item”.
  • an object whose distance from the host vehicle is equal to or less than a predetermined threshold or an object whose time until contact with the host vehicle is predicted to be equal to or less than a predetermined threshold needs attention.
  • an object having a distance of 5 m or less from the host vehicle is defined as an object requiring attention.
  • the definition of the items requiring attention is not limited to this.
  • the method for detecting the object of interest by the object-of-interest detection unit 19 may be any method.
  • the object of interest may be detected by analyzing an electronic mirror image captured by the electronic mirror camera 21 or may be inconvenient.
  • An in-vehicle sensor for example, millimeter wave radar, sonar, infrared sensor, etc.
  • the traveling direction and the traveling lane of the own vehicle and the other vehicle are predicted, In consideration of the prediction result, it may be determined whether or not the other vehicle is an object requiring attention.
  • the synthesis processing unit 14 displays not only an optical mirror field image but also a warning image indicating the presence of an object of interest as an electronic mirror. Composite to the image.
  • the important object detection unit 19 detects the other vehicle V3 as the important object.
  • the composition processing unit 14 synthesizes the alert image 44 indicating the position of the other vehicle V3 with the electronic mirror image 40 as shown in FIG.
  • the alert image 44 is not limited to the one shown in FIG. 33, and may be any image.
  • the optical mirror view image 41 can also be used as the alert image 44.
  • the display aspect of the optical mirror visual field image 41 is changed as shown in FIG. 34 when an important object detection is detected by the critical object detection unit 19, the optical mirror visual field image 41 functions as the attention calling image 44.
  • the optical mirror visual field image 41 functions as the attention calling image 44.
  • FIG. 35 is a block diagram illustrating a configuration of a vehicle mirror system according to a modification of the third embodiment. This vehicle mirror system is obtained by adding an optical mirror direction control unit 18 as a component of the image processing apparatus 10 to the configuration of FIG.
  • the optical mirror direction control unit 18 controls the direction of the optical mirror 22 in accordance with the position of the target object detected by the target object detection unit 19. Specifically, the optical mirror direction control unit 18 sets the orientation of the optical mirror 22 so that at least a part of the object requiring attention enters the field of view of the optical mirror 22. In other words, the optical mirror direction control unit 18 sets the direction of the optical mirror 22 so that at least a part of the object of interest reflected in the electronic mirror image 40 is inside the optical mirror visual field image 41.
  • the optical mirror direction control unit 18 adjusts the direction of the optical mirror 22 so as to follow it.
  • the optical mirror visual field image 41 moves following the movement of the other vehicle V3 on the screen of the display device 24. Since the driver can see a cautionary object on both the display device 24 and the optical mirror 22, the driver can be surely recognized that the cautionary object is present.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Signal Processing (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Transportation (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Processing (AREA)
  • Instrument Panels (AREA)

Abstract

L'invention concerne un processeur d'image (10), dans lequel une unité d'acquisition d'image de rétroviseur électronique (11) acquiert une image de rétroviseur électronique capturée par une caméra (21) pour des rétroviseurs électroniques d'un véhicule. Une unité d'acquisition de direction de rétroviseur optique (12) acquiert des informations concernant la direction d'un rétroviseur optique (22) du véhicule. Une unité de spécification de champ visuel de rétroviseur optique (13) spécifie, sur la base de la direction du rétroviseur optique (22), une zone dans l'image de rétroviseur électronique qui correspond au champ visuel du rétroviseur optique (22), qui est une plage visible depuis le conducteur du véhicule à travers le rétroviseur optique (22). Une unité de traitement de synthèse (14) synthétise l'image de rétroviseur électronique avec une image de champ visuel de rétroviseur optique, qui est une image indiquant la zone correspondant au champ visuel du rétroviseur optique (22). Une unité de traitement d'affichage (15) amène l'image de rétroviseur électronique synthétisée avec l'image de champ visuel de rétroviseur optique à être affichée sur un dispositif d'affichage (24) du véhicule.
PCT/JP2018/018899 2018-05-16 2018-05-16 Processeur d'image pour véhicules et procédé de traitement d'image WO2019220557A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2018/018899 WO2019220557A1 (fr) 2018-05-16 2018-05-16 Processeur d'image pour véhicules et procédé de traitement d'image
JP2020518871A JP6921316B2 (ja) 2018-05-16 2018-05-16 車両用画像処理装置および画像処理方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2018/018899 WO2019220557A1 (fr) 2018-05-16 2018-05-16 Processeur d'image pour véhicules et procédé de traitement d'image

Publications (1)

Publication Number Publication Date
WO2019220557A1 true WO2019220557A1 (fr) 2019-11-21

Family

ID=68540000

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/018899 WO2019220557A1 (fr) 2018-05-16 2018-05-16 Processeur d'image pour véhicules et procédé de traitement d'image

Country Status (2)

Country Link
JP (1) JP6921316B2 (fr)
WO (1) WO2019220557A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021063398A (ja) * 2019-10-16 2021-04-22 株式会社東芝 システム及び方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003339044A (ja) * 2002-05-20 2003-11-28 Nissan Motor Co Ltd 車両用周辺監視装置
JP2006131213A (ja) * 2004-10-07 2006-05-25 Yamaha Motor Co Ltd 自動二輪車の後方視認装置
JP2008141578A (ja) * 2006-12-04 2008-06-19 Auto Network Gijutsu Kenkyusho:Kk 車両周辺視認装置
JP2008230558A (ja) * 2007-03-23 2008-10-02 Denso Corp 車両周辺監視装置
JP2012066700A (ja) * 2010-09-24 2012-04-05 Denso Corp 車両用の後退駐車支援装置および後退駐車支援装置用のプログラム
JP2013078151A (ja) * 2013-01-18 2013-04-25 Denso Corp 車両用表示装置
JP2018039394A (ja) * 2016-09-08 2018-03-15 株式会社Ｊｖｃケンウッド 車両用表示制御装置、車両用表示システム、車両用表示制御方法およびプログラム

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003339044A (ja) * 2002-05-20 2003-11-28 Nissan Motor Co Ltd 車両用周辺監視装置
JP2006131213A (ja) * 2004-10-07 2006-05-25 Yamaha Motor Co Ltd 自動二輪車の後方視認装置
JP2008141578A (ja) * 2006-12-04 2008-06-19 Auto Network Gijutsu Kenkyusho:Kk 車両周辺視認装置
JP2008230558A (ja) * 2007-03-23 2008-10-02 Denso Corp 車両周辺監視装置
JP2012066700A (ja) * 2010-09-24 2012-04-05 Denso Corp 車両用の後退駐車支援装置および後退駐車支援装置用のプログラム
JP2013078151A (ja) * 2013-01-18 2013-04-25 Denso Corp 車両用表示装置
JP2018039394A (ja) * 2016-09-08 2018-03-15 株式会社Ｊｖｃケンウッド 車両用表示制御装置、車両用表示システム、車両用表示制御方法およびプログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021063398A (ja) * 2019-10-16 2021-04-22 株式会社東芝 システム及び方法

Also Published As

Publication number Publication date
JP6921316B2 (ja) 2021-08-18
JPWO2019220557A1 (ja) 2020-12-17

Similar Documents

Publication Publication Date Title
US10116873B1 (en) System and method to adjust the field of view displayed on an electronic mirror using real-time, physical cues from the driver in a vehicle
CN108958467B (zh) 用于控制全息图的显示器的设备和方法，车辆系统
EP1972496B1 (fr) Système d'affichage externe de véhicule et appareil de contrôle d'affichage
JP3431678B2 (ja) 車両用周囲状況表示装置
US20130242101A1 (en) Method and Device for Representing Obstacles in a Parking Assistance System of Motor Vehicles
JP6384322B2 (ja) 車両用表示装置
US10846833B2 (en) System and method for visibility enhancement
CN109314765B (zh) 车辆用显示控制装置、显示系统、显示控制方法以及程序
KR102130059B1 (ko) 디지털 백미러 제어 유닛 및 방법
JP2010033106A (ja) 運転者支援装置、運転者支援方法および運転者支援処理プログラム
JP6532616B2 (ja) 表示制御装置、表示システム、及び、表示制御方法
JP6890288B2 (ja) 画像処理装置、画像表示システムおよび画像処理方法
US9598012B2 (en) Surroundings monitoring system for a vehicle
JP2005153660A5 (fr)
KR101305310B1 (ko) 차량의 어라운드 뷰 제공 방법
WO2018042976A1 (fr) Dispositif de génération d'image, procédé de génération d'image, support d'enregistrement et système d'affichage d'image
WO2019220557A1 (fr) Processeur d'image pour véhicules et procédé de traitement d'image
CN109849939B (zh) 用于控制车辆内显示器的装置和方法
US10343603B2 (en) Image processing device and image processing method
KR102100978B1 (ko) 리어 뷰를 위한 제어 유닛 및 방법
KR102125738B1 (ko) 주차 보조 시스템
JP6555240B2 (ja) 車両用撮影表示装置及び車両用撮影表示プログラム
US20210129751A1 (en) Side and rear reflection controller and side and rear reflection control method
JP2020093574A (ja) 表示制御装置、表示制御システム、表示制御方法、および表示制御プログラム
EP3315366B1 (fr) Dispositif d'imagerie et d'affichage pour véhicule et support d'enregistrement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18918819

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020518871

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18918819

Country of ref document: EP

Kind code of ref document: A1