WO2019093378A1 - 表示処理装置 - Google Patents

表示処理装置 Download PDF

Info

Publication number
WO2019093378A1
WO2019093378A1 PCT/JP2018/041347 JP2018041347W WO2019093378A1 WO 2019093378 A1 WO2019093378 A1 WO 2019093378A1 JP 2018041347 W JP2018041347 W JP 2018041347W WO 2019093378 A1 WO2019093378 A1 WO 2019093378A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
display
vehicle
display processing
converted
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2018/041347
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
祐司 石田
博彦 柳川
宗作 重村
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Denso Corp
Original Assignee
Denso Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Denso Corp filed Critical Denso Corp
Priority to DE112018005391.1T priority Critical patent/DE112018005391T5/de
Publication of WO2019093378A1 publication Critical patent/WO2019093378A1/ja
Priority to US16/869,966 priority patent/US20200267353A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/21—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor using visual output, e.g. blinking lights or matrix displays
    • B60K35/22—Display screens
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/28—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/173—Reversing assist
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/179—Distances to obstacles or vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/20—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/304—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using merged images, e.g. merging camera image with stored images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
    • B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146—Display means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/10—Longitudinal speed
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/18—Steering angle

Definitions

  • the present disclosure relates to a display processing device mounted on a vehicle.
  • Patent Document 1 an image generation apparatus that generates an image in which the same location viewed from two different virtual viewpoints is displayed at the center of each of two images, and the driver of the vehicle can accurately grasp the situation of the location to check It is disclosed.
  • the composite image IMa and the composite image IMb are displayed on the display as a display image.
  • the composite image IMa is an image generated on the basis of a virtual viewpoint VPa whose viewpoint direction is a predetermined direction from the viewpoint position with the viewpoint position near the driver of the vehicle as the viewpoint position.
  • the composite image IMb is an image generated based on a virtual viewpoint VPb whose direction of sight is a direction pointing to a specific location located on an extension of the viewing direction of the virtual viewpoint VPa, with the position outside the vehicle as the viewpoint position. It is.
  • Patent No. 5977130 gazette
  • the on-vehicle camera is disposed on the vehicle body toward the outside of the vehicle, the under-vehicle can not be imaged, and the image under the vehicle is not included in the captured image around the own vehicle captured by the on-vehicle camera. Therefore, as a result of the inventor's detailed examination, the image under the vehicle is not displayed in the display image generated by the above-described image generation device, and a problem is found that it is difficult to grasp the condition under the vehicle. It was done.
  • One aspect of the present disclosure is to provide a display processing device capable of generating a display image that can also grasp the situation under the vehicle.
  • One aspect of the present disclosure is a display processing device mounted on a vehicle, and includes an acquisition unit, a first generation unit, a calculation unit, a second generation unit, and a display processing unit.
  • the acquisition unit is configured to acquire a captured image obtained by imaging the periphery of the vehicle from at least one imaging device mounted on the vehicle.
  • the first generation unit is configured to generate a first converted image, which is an image viewed from a first viewpoint in a room of the vehicle, based on a captured image at the latest imaging time.
  • the calculation unit is configured to calculate the displacement of the vehicle.
  • the second generation unit is an image viewed from the first viewpoint at the latest imaging time based on the captured image imaged before the latest imaging time and the displacement, and includes the area under the vehicle Configured to generate a second transformed image that is an image of
  • the display processing unit is configured to display a display image obtained by combining the first converted image and the second converted image on a display device that can be viewed by an occupant of the vehicle.
  • the image of the area including the under the vehicle that is, the history image
  • the past image generated by the second generation unit By combining, a display image is generated which is an image of the periphery of the vehicle that may include under the vehicle. Therefore, the situation under the vehicle can also be roughly grasped.
  • a vehicle 1 shown in FIG. 1 includes a front camera 2a, a rear camera 2b, a left side camera 2c, a right side camera 2d, a vehicle speed sensor 3, a steering angle sensor 4, a display processing device 5, and a display device 6.
  • Each of the cameras 2a to 2d is an imaging device for imaging the periphery of the vehicle 1, and outputs a signal representing a captured image to the display processing device 5 via an LVDS communication line or the like.
  • the front camera 2a, the rear camera 2b, the left side camera 2c and the right side camera 2d are arranged so as to respectively capture the front, rear, left side and right side of the vehicle 1, as shown in FIG.
  • the front camera 2 a shoots the front of the surroundings of the vehicle 1.
  • the displayable range 21A of the captured front image 21 is, for example, approximately 180 degrees using a fisheye lens or the like.
  • the right side camera 2 d captures the right side of the surroundings of the vehicle 1.
  • the displayable range 22A of the right side image 22 taken is, for example, approximately 180 degrees using a fisheye lens or the like.
  • the overlapping range is referred to as overlapping range 31.
  • the overlapping range 31 is, for example, 90 degrees.
  • the left side camera 2 c captures the left side of the surroundings of the vehicle 1.
  • the displayable range 23A of the captured left side image 23 is, for example, approximately 180 degrees using a fisheye lens or the like.
  • the overlapping range is referred to as overlapping range 32.
  • the overlapping range 32 is, for example, 90 degrees.
  • the rear camera 2 b shoots the rear of the surroundings of the vehicle 1.
  • the displayable range 24A of the captured rear image 24 is, for example, approximately 180 degrees using a fisheye lens or the like.
  • the overlapping range is referred to as overlapping range 33.
  • the overlapping range 33 is, for example, 90 degrees.
  • overlapping range 34 a part of the displayable range 24A and a part of the displayable range 23A overlap.
  • the overlapping range is referred to as overlapping range 34.
  • the overlapping range 34 is, for example, 90 degrees.
  • the vehicle speed sensor 3 is a sensor for detecting the traveling speed of the vehicle 1.
  • the vehicle speed sensor 3 outputs a signal corresponding to the traveling speed to the display processing device 5 via an in-vehicle communication LAN such as CAN.
  • CAN is a registered trademark.
  • the steering angle sensor 4 is a sensor for detecting the steering angle of the steering of the vehicle 1.
  • the steering angle sensor 4 outputs a signal corresponding to the detected steering angle to the display processing device 5 via an on-vehicle communication LAN such as CAN.
  • the display processing device 5 is mainly configured of a known microcomputer having a CPU, a ROM, a RAM, a flash memory and the like (not shown).
  • the CPU executes a program stored in the ROM, which is a non-transitional tangible recording medium. By executing the program, a method corresponding to the program is executed. Specifically, the display processing device 5 executes an image display process shown in FIG. 3 described later according to the program.
  • the display processing device 5 may include one microcomputer or a plurality of microcomputers.
  • the display processing device 5 has, as a configuration of functions realized by the CPU executing a program, an acquisition unit 51, a first generation unit 52, a calculation unit 53, a second generation unit 54, and a first conversion unit 55. , A combining unit 56, a second conversion unit 57, and a display processing unit 58.
  • the method for realizing the function of each part included in the display processing device 5 is not limited to software, and part or all of the function may be realized using one or more hardware.
  • the electronic circuit may be implemented by a digital circuit, an analog circuit, or a combination thereof.
  • the display device 6 is a display for displaying an image.
  • the display device 6 is provided at a position where a driver of the vehicle 1 can visually recognize, and is connected to the display processing device 5 via an LVDS communication line or the like.
  • image display processing performed by the display processing device 5 will be described using the flowchart of FIG. 3.
  • the image display process is performed based on, for example, an on operation by an ignition switch, a display instruction operation by a user, an unlocking operation, or the like.
  • the display processing device 5 acquires captured images from the respective cameras 2a to 2d. Note that S11 corresponds to the processing as the acquisition unit 51.
  • the display processing device 5 executes a first image conversion process.
  • the display processing device 5 generates a first converted image obtained by converting the captured image acquired in S11 into an image viewed from the first viewpoint.
  • the first viewpoint is a virtual viewpoint in the room of the vehicle 1
  • the image viewed from the first viewpoint is a perspective view as seen from the vehicle interior. Since each of the cameras 2a to 2d is disposed on the vehicle body toward the outside of the vehicle, the captured image does not include an image of the immediate vicinity of the vehicle or under the vehicle. Therefore, the image viewed from the first viewpoint is an image of the periphery of the vehicle 1 not including the lower part of the vehicle.
  • the four captured images in front of, behind, to the left, and to the right of the vehicle 1 acquired in S11 adjust the transmittance of the captured image of the portion where the imaging ranges of the respective cameras 2a to 2d overlap. Or by using a predetermined boundary line. That is, the correspondence between the image captured by each of the cameras 2 a to 2 d and the position of each pixel included in the captured image is determined in advance, and a virtual three-dimensional three-dimensional curved surface disposed around the vehicle 1 It is synthesized by being projected. Necessary regions in the three-dimensional curved surface are cut out as an image according to the viewing direction from the set virtual viewpoint. Thereby, a first converted image viewed from the first viewpoint is generated.
  • the first converted image since the first converted image is used as a display image immediately after conversion, the first converted image has almost the same display content as at the time of imaging, and becomes an almost real time image.
  • S22 after storing the first converted image in the flash memory, the display processing device 5 ends the first image conversion processing, and shifts the processing to S13.
  • S21 and S22 correspond to the processing as the first generation unit 52.
  • the display processing device 5 executes a second image conversion process.
  • the display processing device 5 generates a top view image obtained by converting each of the images captured by the cameras 2a to 2d acquired in S11 into an image viewed from the second viewpoint.
  • the second viewpoint is a viewpoint outside the vehicle 1 of the vehicle 1 and is a viewpoint seen from directly above the vehicle 1, that is, a so-called top view. Note that S31 corresponds to the processing of the first conversion unit 55.
  • the display processing device 5 stores the top view image in the flash memory.
  • the display processing device 5 calculates the displacement of the vehicle 1 indicated by the movement distance and the rotation angle based on the information on the traveling speed and the steering angle acquired from the vehicle speed sensor 3 and the steering angle sensor 4. Note that S33 corresponds to the processing as the calculation unit 53.
  • the display processing device 5 executes an image history synthesis process.
  • the display processing device 5 determines whether the traveling direction of the vehicle 1 is positive and the steering angle direction is positive. In addition, when the advancing direction of the vehicle 1 is positive, it is a case where the vehicle 1 is advancing ahead. On the other hand, when the traveling direction of the vehicle 1 is negative, the vehicle 1 is traveling backward. Further, when the steering angle direction is positive, the vehicle 1 is turning to the left. On the other hand, when the steering angle direction is negative, the vehicle 1 is turning to the right. When the steering angle direction is positive, the state in which the vehicle 1 travels forward or backward without turning is also included.
  • the display processing device 5 determines that the traveling direction of the vehicle 1 is positive and the steering angle direction is positive in S41, the process proceeds to S42, and after the front camera 2a and the left side camera 2c are selected, the processing is performed. Transfer to S48.
  • the display processing device 5 determines that the traveling direction of the vehicle 1 is positive and the steering angle direction is not positive in S41, the process proceeds to S43, and the traveling direction of the vehicle 1 is positive and the steering angle direction is It is determined whether or not it is negative.
  • the process proceeds to S44 and after selecting the front camera 2a and the right side camera 2d, The process shifts to S48.
  • the display processing device 5 determines in S43 that the traveling direction of the vehicle 1 is positive and the steering angle direction is not negative, the process proceeds to S45, and the traveling direction of the vehicle 1 is negative and the steering angle direction is It is determined whether or not it is positive.
  • the display processing device 5 determines that the traveling direction of the vehicle 1 is negative and the steering angle direction is positive in S45, the process proceeds to S46 and after selecting the rear camera 2b and the left side camera 2c, The process shifts to S48.
  • step S45 determines in S45 that the traveling direction of the vehicle 1 is negative and the steering angle direction is not positive, that is, the traveling direction of the vehicle 1 is negative and the steering angle direction is negative.
  • step S47 the process proceeds to step S48.
  • the display processing device 5 combines a plurality of top view images different in imaging time point from each other at different positions based on the displacement between different imaging time points, and synthesizes an image viewed from the second viewpoint at the latest imaging time To generate a history composite image.
  • the history composite image is an image that may include the under-vehicle viewed from the second viewpoint at the latest imaging time.
  • S48 corresponds to the processing as the combining unit 56.
  • the history synthesis performed here complements the image of the part out of the image captureable range by movement using the top view image at time t ⁇ 1.
  • the technique of history synthesis for complementing the region outside the imaging range from the past image is processed as described in, for example, JP-A-2002-373327 and JP-A-2003-191810.
  • the traveling direction of the vehicle 1 is positive and the steering angle direction is positive, and the front camera 2 a and the left side camera 2 c are selected will be specifically described below as an example.
  • the position of the first top view image which is the top view image at time t1 of the front camera 2a stored in the flash memory, is shifted to the top view image after movement.
  • the second top view image which is a top view image at time t2 newly captured and stored in the flash memory, and the top view image after movement are combined to generate a history combined image.
  • a history composite image is generated.
  • the generation of the history composite image is performed every 0.1 seconds.
  • the selected left side camera 2c is used to complement an area outside the field of view that can not be filled with the front camera 2a in the history composite image.
  • the display processing device 5 generates a history composite image in S48, ends the image history combining processing, and shifts the processing to S35.
  • the camera image to be used is selected from the displacement direction of the vehicle, and the top view history composite image is generated using only the selected camera image, but the images of all four cameras are used And may generate a top view historical composite image.
  • the display processing device 5 generates a second converted image in which the history composite image is viewed from the first viewpoint.
  • the second converted image is an image that may include the lower part of the vehicle viewed from the first viewpoint.
  • S35 corresponds to the process of the second conversion unit 57
  • S31, S35, and S48 correspond to the process of the second generation unit 54.
  • S36 after storing the second converted image in the flash memory, the display processing device 5 ends the second image conversion processing, and shifts the processing to S14.
  • S12 and S13 may be serial processing or parallel processing.
  • a blind spot area of the camera corresponding to the lower part of the vehicle in the first converted image generated in S12 is specified, and an image corresponding to the specified area among the second converted image generated in S13.
  • the region is combined with the first transformed image to generate a display image.
  • the display processing device 5 adjusts the luminance of the second converted image stored in the flash memory to a low level, and then performs the first process.
  • the display processing device 5 causes the display device 6 to display the display image on which the image of the semi-transmissive vehicle body of the vehicle 1 stored in advance in the flash memory is superimposed. The reason why the brightness of the second converted image is adjusted to a low level is to make it possible to distinguish the first converted image and the second converted image in the display image.
  • an area where the first converted image is displayed in the display image is an area A
  • An area where B is displayed is an area B.
  • the boundary between the area A and the area B is illustrated by a broken line not displayed in the display image, the outside of the broken line is the area A, and the inside of the broken line is the area B.
  • Region B is a region for displaying the area under the vehicle.
  • the display processing device 5 continues generating the second converted image filling the entire area B.
  • a display image is displayed on the display device 6 by combining a predetermined background image, for example, a black image.
  • the second converted image is synthesized instead of at least a part of the background image, that is, the second converted image is added to the background image.
  • the display image generated by overwriting the converted image may be displayed on the display device 6.
  • the display processing device 5 determines whether the end condition of the image display processing is satisfied. Specifically, it is determined whether the operation of the ignition switch is off or another display has been performed. If the display processing device 5 determines that the end condition is not satisfied, that is, the ignition switch OFF operation or the separate display operation is not performed, the process returns to S11.
  • the display processing device 5 determines that the ending condition is satisfied in S15, that is, the ignition switch is turned off or another display operation is performed, the image display processing is ended. In addition, even if the OFF operation of the ignition switch is performed, the display processing may be ended after the display is continued until the predetermined time or the locking operation is performed so that the surrounding situation can be confirmed.
  • the first generation unit 52 generates a first converted image which is an image viewed from a first viewpoint in the room of the vehicle 1 based on the captured image at the latest imaging time point Ru.
  • the second generation unit is an image viewed from the first viewpoint at the latest imaging time based on the captured image imaged before the latest imaging time and the displacement of the vehicle 1
  • a second transformed image is generated which is an image that may include under the vehicle.
  • the display processing unit displays a display image obtained by combining the first converted image and the second converted image on the display device 6.
  • a first converted image which is an image of the periphery of the vehicle 1 which does not include the lower part of the vehicle and a second converted image which is an image which may include the lower part of the vehicle , Is synthesized.
  • a display image is generated which is an image viewed from the first viewpoint and which is an image around the vehicle 1 that may include the area under the vehicle. Therefore, the driver can also understand the situation under the vehicle.
  • a captured image obtained by capturing the periphery of the vehicle 1 is converted into a top view image viewed from the second viewpoint. Then, a history composite image is generated in which a plurality of top view images having different imaging time points are synthesized by shifting the position based on the displacement of the vehicle 1 between different imaging time points.
  • the image under the vehicle which is not affected by the perspective of the captured image is used to generate the history composite image. Therefore, an image under the vehicle with less discomfort in the display image can be displayed in the area B.
  • a display image obtained by combining the background image is displayed on the display device 6 instead of the second converted image until the second converted image is generated to fill the entire area B. According to such a configuration, it is possible not to display an incomplete image under the vehicle.
  • the display image is displayed on the display device 6 in such a manner that the first converted image and the second converted image can be distinguished.
  • the boundary between the area A displaying the captured image at the latest imaging time and the area B displaying the image other than the captured image at the latest imaging time is clarified.
  • the area B in which the image under the vehicle is displayed is dark relative to the area A in which the image around the vehicle 1 is displayed Is displayed. Therefore, it can be intuitively recognized that the driver is an image under the vehicle.
  • the vehicle 1 includes four cameras including the front camera 2a, the rear camera 2b, the left side camera 2c, and the right side camera 2d.
  • the vehicle 1 includes one or two cameras. It may be a configuration.
  • the display processing device 5 adjusts the luminance of the second converted image to a low level, thereby making it possible to distinguish the first converted image and the second converted image in the display image.
  • the identification method is not limited to this.
  • the luminance of the second converted image in the area B is adjusted by changing and superposing the color of the portion overlapping the area B in the image of the vehicle body of the vehicle 1 superimposed on the display image in a semi-transparent manner It is also good.
  • a frame surrounding the area B may be displayed on the dashed line shown in FIG. 8 in the display image.
  • the display processing device 5 moves at pixels around the moving object because the reliability of the presence of the moving object is low. You may fill the body.
  • the display processing device 5 may superimpose a specific mark on the display image.
  • the configuration in which the history composite image is generated using two cameras is illustrated in the above embodiment, for example, the configuration may be such that a history composite image is generated using one camera.
  • a history composite image is generated using one camera.
  • FIG. 9 when generating a history composite image using only the front camera 2a, an area outside the field of view that can not be filled using the front camera 2a in the history composite image is not complemented.
  • an area narrower than the history composite image as the display image it is not necessary to supplement out of the field of view.
  • the configuration is shown in which a black image is displayed as the background image, but the background image is not limited to this.
  • the second converted image stored in the previous image display processing such as the ignition switch OFF or another display operation is displayed
  • the configuration may be different.
  • the display may be performed in a mode in which the freshness of the complementary image can be known by color, contrast, luminance, icon superimposed display, or the like.
  • the function of one component in the above embodiment may be distributed as a plurality of components, or the function of a plurality of components may be integrated into one component.
  • part of the configuration of the above embodiment may be omitted.
  • at least a part of the configuration of the above-described embodiment may be added to or replaced with the configuration of the other above-described embodiment.
  • all the aspects contained in the technical thought specified from the wording as described in a claim are an embodiment of this indication.
  • the present disclosure includes a system including the display processing device 5 as a component, a program for causing a computer to function as the display processing device 5, a medium storing the program, and an image. It can be realized in various forms such as a generation method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Chemical & Material Sciences (AREA)
  • Automation & Control Theory (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Combustion & Propulsion (AREA)
  • Human Computer Interaction (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Processing (AREA)
  • Traffic Control Systems (AREA)
PCT/JP2018/041347 2017-11-10 2018-11-07 表示処理装置 Ceased WO2019093378A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
DE112018005391.1T DE112018005391T5 (de) 2017-11-10 2018-11-07 Anzeigeverarbeitungsvorrichtung
US16/869,966 US20200267353A1 (en) 2017-11-10 2020-05-08 Display processing device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017217359A JP6740991B2 (ja) 2017-11-10 2017-11-10 表示処理装置
JP2017-217359 2017-11-10

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/869,966 Continuation US20200267353A1 (en) 2017-11-10 2020-05-08 Display processing device

Publications (1)

Publication Number Publication Date
WO2019093378A1 true WO2019093378A1 (ja) 2019-05-16

Family

ID=66438385

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/041347 Ceased WO2019093378A1 (ja) 2017-11-10 2018-11-07 表示処理装置

Country Status (4)

Country Link
US (1) US20200267353A1 (enExample)
JP (1) JP6740991B2 (enExample)
DE (1) DE112018005391T5 (enExample)
WO (1) WO2019093378A1 (enExample)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210179086A1 (en) * 2019-12-13 2021-06-17 Honda Motor Co., Ltd. Parking assisting device, parking assisting method and storage medium storing program for the parking assisting device

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11140364B2 (en) * 2019-09-09 2021-10-05 Texas Instruments Incorporated Sensor fusion based perceptually enhanced surround view
JP7319593B2 (ja) * 2020-02-13 2023-08-02 トヨタ自動車株式会社 車両周辺監視装置
JP7593272B2 (ja) * 2021-09-09 2024-12-03 株式会社豊田自動織機 障害物回避装置
JP7593274B2 (ja) * 2021-09-09 2024-12-03 株式会社豊田自動織機 障害物回避装置
JP7773835B2 (ja) * 2022-04-26 2025-11-20 パナソニックオートモーティブシステムズ株式会社 表示制御装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003189291A (ja) * 2001-12-18 2003-07-04 Denso Corp 車両周辺画像処理装置及び記録媒体
JP2003244688A (ja) * 2001-12-12 2003-08-29 Equos Research Co Ltd 車両の画像処理装置
JP2005001570A (ja) * 2003-06-12 2005-01-06 Equos Research Co Ltd 駐車支援装置
JP2006298258A (ja) * 2005-04-22 2006-11-02 Aisin Aw Co Ltd 駐車支援方法及び駐車支援装置
JP2007269060A (ja) * 2006-03-30 2007-10-18 Aisin Aw Co Ltd 運転支援方法及び運転支援装置
JP2014209713A (ja) * 2013-03-28 2014-11-06 アイシン精機株式会社 周辺監視装置、及びプログラム
JP2016197785A (ja) * 2015-04-02 2016-11-24 アイシン精機株式会社 周辺監視装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4156214B2 (ja) 2001-06-13 2008-09-24 株式会社デンソー 車両周辺画像処理装置及び記録媒体
JP3886376B2 (ja) 2001-12-26 2007-02-28 株式会社デンソー 車両周辺監視システム
JP5977130B2 (ja) 2012-09-25 2016-08-24 富士通テン株式会社 画像生成装置、画像表示システム、および、画像生成方法
JP2017217359A (ja) 2016-06-10 2017-12-14 オリンパス株式会社 超音波観測装置、超音波観測装置の作動方法、及び超音波観測装置の作動プログラム

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003244688A (ja) * 2001-12-12 2003-08-29 Equos Research Co Ltd 車両の画像処理装置
JP2003189291A (ja) * 2001-12-18 2003-07-04 Denso Corp 車両周辺画像処理装置及び記録媒体
JP2005001570A (ja) * 2003-06-12 2005-01-06 Equos Research Co Ltd 駐車支援装置
JP2006298258A (ja) * 2005-04-22 2006-11-02 Aisin Aw Co Ltd 駐車支援方法及び駐車支援装置
JP2007269060A (ja) * 2006-03-30 2007-10-18 Aisin Aw Co Ltd 運転支援方法及び運転支援装置
JP2014209713A (ja) * 2013-03-28 2014-11-06 アイシン精機株式会社 周辺監視装置、及びプログラム
JP2016197785A (ja) * 2015-04-02 2016-11-24 アイシン精機株式会社 周辺監視装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210179086A1 (en) * 2019-12-13 2021-06-17 Honda Motor Co., Ltd. Parking assisting device, parking assisting method and storage medium storing program for the parking assisting device
US11697408B2 (en) * 2019-12-13 2023-07-11 Honda Motor Co., Ltd. Parking assisting device, parking assisting method and storage medium storing program for the parking assisting device

Also Published As

Publication number Publication date
JP6740991B2 (ja) 2020-08-19
DE112018005391T5 (de) 2020-06-25
JP2019087980A (ja) 2019-06-06
US20200267353A1 (en) 2020-08-20

Similar Documents

Publication Publication Date Title
WO2019093378A1 (ja) 表示処理装置
EP4028288B1 (en) Sensor fusion based perceptually enhanced surround view
JP4254887B2 (ja) 車両用画像表示システム
KR102151654B1 (ko) 화상 표시 장치
US8031225B2 (en) Surroundings monitoring system for a vehicle
CN112055963B (zh) 图像处理装置、移动装置、方法和程序
JP5697512B2 (ja) 画像生成装置、画像表示システム及び画像表示装置
JP2010274813A (ja) 画像生成装置及び画像表示システム
JP6084097B2 (ja) 画像生成装置、画像表示システム及び画像生成方法
JP2016063390A (ja) 画像処理装置、及び画像表示システム
CN108463998A (zh) 驾驶辅助装置以及驾驶辅助方法
US20190087665A1 (en) Information processing apparatus and program
WO2019058585A1 (ja) 表示制御装置
JP2014129093A (ja) 車両用周辺監視装置
JP7203205B2 (ja) 車両周辺画像生成装置、車両周辺表示システム、及び車両周辺表示方法
JP5977130B2 (ja) 画像生成装置、画像表示システム、および、画像生成方法
JP2012138876A (ja) 画像生成装置、画像表示システム及び画像表示方法
JP2012001126A (ja) 車両用周辺監視装置
US10821900B2 (en) Image processing device
WO2020017230A1 (ja) 電子制御装置及び電子制御方法
CN108431866B (zh) 显示控制装置以及显示控制方法
WO2014010179A1 (ja) 車両用視界支援装置
US20200231099A1 (en) Image processing apparatus
WO2016104504A1 (ja) 画像処理システム及び画像処理装置
JP2007249814A (ja) 画像処理装置及び画像処理プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18876685

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 18876685

Country of ref document: EP

Kind code of ref document: A1