JP2020024487A - Image processing apparatus and image processing method - Google Patents

Abstract

To provide an image processing technique that enables a user to grasp easily whether an own vehicle can pass through in advance when the vehicle is to pass through a place with obstacles.SOLUTION: An image processing apparatus includes a photographing image acquisition unit and display image generation unit. The photographing image acquisition unit acquires a photographed image obtained by a camera photographing the periphery of the own vehicle. The display image generation unit generates a display image by overlapping an image of a virtual vehicle representing the own vehicle on a translated image in which the photographing image or a converted image converted from the photographing image. In addition, the display image generation unit can move at least one of the photographing image, the converted image, or an image of the virtual vehicle as a moving target to an area of the display image, and adjusts a scale rate of the moving target according to the movement.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing technique.

  Patent Document 1 discloses a driving support system that detects an obstacle existing in front of a host vehicle by a sensor and issues an alarm when the possibility of collision between the host vehicle and the obstacle detected by the sensor is greater than a predetermined level. ing.

JP-A-2004-145725

  In Patent Document 1, as an example of the possibility of collision, a value corresponding to a time obtained by dividing the distance between the host vehicle and the obstacle by the relative speed between the host vehicle and the obstacle is given. Therefore, for example, when the own vehicle travels on a narrow road by passing with an oncoming vehicle, the driving support system disclosed in Patent Literature 1 is considered to issue an alarm regardless of whether the vehicle can pass. That is, the driving support system disclosed in Patent Literature 1 is not a system that can provide a user with useful information when the own vehicle attempts to pass a place where an obstacle exists.

  The present invention has been made in view of the above circumstances, and has as its object to provide an image processing technique that allows a user to easily understand in advance whether a vehicle can pass through a place where an obstacle exists, in advance. .

  An image processing device according to the present invention represents a photographed image acquisition unit that acquires a photographed image obtained by a camera that photographs the periphery of the vehicle, and represents the vehicle in the captured image or a converted image obtained by converting the captured image. A display image generation unit configured to generate a display image by superimposing an image of the virtual vehicle, wherein the display image generation unit moves at least one of the captured image or the converted image and the image of the virtual vehicle. The first configuration is a configuration that can be moved with respect to the area of the display image and that adjusts the enlargement / reduction ratio of the movement target according to the movement.

  In the image processing device having the first configuration, the display image generation unit may be configured to move the movement target based on an output of an operation unit that receives a user operation (a second configuration).

  In the image processing device according to the first configuration, the display image generation unit acquires information on an expected course of the host vehicle and moves the moving target based on the estimated course (third configuration). You may.

  In the image processing device according to any one of the first to third configurations, an estimating unit that estimates an area where an obstacle exists in the display image, and an image of the virtual vehicle and the area estimated by the estimating unit. And a warning unit that outputs a warning signal based on the positional relationship (fourth structure).

  In the image processing device according to any one of the first to fourth configurations, the display image generation unit generates the display image by superimposing an image of the virtual vehicle on the conversion image, and the conversion image includes a virtual viewpoint. A configuration that is a virtual viewpoint image viewed from the viewpoint (fifth configuration) may be used.

  In the image processing apparatus having the fifth configuration, the display image generation unit may be configured such that when a viewpoint position of the virtual viewpoint is substantially directly above the own vehicle and a line of sight of the virtual viewpoint is substantially directly below the own vehicle. May be a configuration (sixth configuration) in which the movement target is not enlarged or reduced.

  An image processing method according to the present invention includes a photographed image acquiring step of acquiring a photographed image obtained by a camera photographing the periphery of the vehicle, and representing the vehicle in the photographed image or a converted image obtained by converting the photographed image. A display image generating step of generating a display image by superimposing an image of the virtual vehicle, wherein the display image generating step includes moving at least one of the captured image or the converted image and the image of the virtual vehicle. (7th configuration) including an adjustment step of adjusting the enlargement / reduction ratio of the movement target according to the movement when the movement target is moved with respect to the area of the display image.

  ADVANTAGE OF THE INVENTION According to this invention, when an own vehicle tries to pass through the place where an obstacle exists, a user can easily grasp in advance whether it is possible to pass.

FIG. 1 is a diagram illustrating a configuration of an image processing apparatus according to a first embodiment. Diagram exemplifying a position where a front camera is arranged in a vehicle 4 is a flowchart illustrating an operation example of the image processing apparatus according to the first embodiment. Diagram showing an example of a display image Diagram showing an example of a display image Diagram showing an example of a display image FIG. 2 is a diagram illustrating a configuration of an image processing apparatus according to a second embodiment. FIG. 3 is a diagram illustrating a configuration of an image processing apparatus according to a third embodiment. 9 is a flowchart illustrating an operation example of the image processing apparatus according to the third embodiment. FIG. 4 is a diagram illustrating a configuration of an image processing apparatus according to a fourth embodiment. Diagram showing an example of a virtual projection plane Diagram showing an example of a display image Diagram showing an example of a display image Diagram showing an example of a display image

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

<1. First Embodiment>
<1-1. Configuration of Image Processing Apparatus According to First Embodiment>
FIG. 1 is a diagram illustrating a configuration of an image processing apparatus 201 according to the present embodiment. The image processing device 201 is mounted on a vehicle such as an automobile, for example. Hereinafter, a vehicle on which the image processing device 201 or an image processing device according to another embodiment described later is mounted is referred to as “own vehicle”. Further, the direction in which the host vehicle travels straight, and the direction from the driver's seat to the steering wheel is referred to as “forward”. The direction in which the host vehicle travels straight ahead, and the direction from the steering to the driver's seat is referred to as "rearward".

  The front camera 11, the operation unit 12, the image processing device 201, and the display device 31 shown in FIG. 1 are mounted on the own vehicle.

  FIG. 2 is a diagram exemplifying a position where the front camera 11 is arranged on the host vehicle V1.

  The front camera 11 is provided at the front end of the host vehicle V1. The optical axis 11a of the front camera 11 extends in the front-rear direction of the vehicle V1 in plan view from above. The front camera 11 photographs the front direction of the own vehicle V1. The mounting position of the front camera 11 is preferably at the center in the left and right direction of the host vehicle V1, but may be a position slightly shifted in the left and right direction from the center in the left and right direction.

  In the present embodiment, the horizontal view angle θ of the front camera 11 is 180 degrees or more. However, the horizontal angle of view θ of the front camera 11 may be less than 180 degrees. Further, different from the present embodiment, the front camera 11 may be provided in the cabin of the host vehicle V1.

  Returning to FIG. 1, the front camera 11 outputs a captured image to the image processing device 201.

  The operation unit 12 receives a user operation and outputs an operation signal according to the content of the user operation. The operation unit 12 includes, for example, a touch panel provided on a display screen of the display device 31, a key group including a direction key and an enter key, and the like. In the present embodiment, the operation unit 12 accepts a user operation for moving a captured image or CG as a movement target with respect to an area of a display image.

  The image processing device 201 processes the captured image output from the front camera 11 and outputs the processed image to the display device 31.

  The display device 31 is provided at a position where the driver of the vehicle can visually recognize the display screen of the display device 31 and displays an image output from the image processing device 201. Examples of the display device 31 include a display installed at a center console, a meter display installed at a position facing a driver's seat, and a head-up display that projects an image on a windshield.

  The image processing device 201 can be configured by hardware such as an ASIC (application specific integrated circuit) or an FPGA (field-programmable gate array), or a combination of hardware and software. When the image processing apparatus 201 is configured using software, a block diagram of a part realized by software represents a functional block diagram of the part. A function realized by using software may be described as a program, and the function may be realized by executing the program on a program execution device. Examples of the program execution device include a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory).

  The image processing device 201 includes a captured image acquisition unit 21 and a display image generation unit 22.

  The photographed image acquiring unit 21 continuously and temporally acquires an analog or digital photographed image from the front camera 11 at a predetermined cycle (for example, a 1/30 second cycle). If the acquired captured image is analog, the captured image acquisition unit 21 converts the analog captured image into a digital captured image (A / D conversion). The captured image acquisition unit 21 outputs the acquired captured image or the acquired and converted captured image to the display image generation unit 22.

  The image generation unit 22 includes a CG (Computer Graphics) generation unit 22a, a movement amount determination unit 22b, an enlargement / reduction ratio adjustment unit 22c, and a superimposition unit 22d.

  The CG generation unit 22a generates a back image CG of the virtual vehicle representing the own vehicle. In the present embodiment, CG is used as an image of the virtual vehicle representing the own vehicle, but a captured image of the own vehicle may be used instead of CG.

  The movement amount determination unit 22b determines the movement amount of the movement target based on the operation signal output from the operation unit 12.

  The enlargement / reduction ratio adjustment unit 22c adjusts the enlargement / reduction ratio of the moving object according to the moving amount of the moving object.

  The superimposition unit 22d generates a display image by superimposing the CG generated by the CG generation unit 22a on the captured image output from the captured image acquisition unit 21.

<1-2. Operation of Image Processing Apparatus According to First Embodiment>
FIG. 3 is a flowchart illustrating an operation example of the image processing apparatus 201.

  When there is a display instruction of the vehicle CG by the operation unit 12, the flow operation shown in FIG. 3 is started. The superimposition unit 22d generates a display image by superimposing the CG generated by the CG generation unit 22a on the captured image output from the captured image acquisition unit 21 at the initial position (Step S10). The initial position of the captured image output from the captured image acquisition unit 21 may be, for example, a position where the center of the region of the display image matches the center of the region of the captured image. The initial position of the CG generated by the CG generation unit 22a may be, for example, a position ahead of the own vehicle by a predetermined distance. Note that the size of the CG generated by the CG generation unit 22a may be set based on the size of the captured image output from the captured image acquisition unit 21 and the initial position of the CG generated by the CG generation unit 22a.

  FIG. 4 is an example of the display image generated in step S10. In FIG. 4 and other display images to be described later, reference numeral 1 indicates a region of the display image, and reference numeral 2 indicates a back image CG of the virtual vehicle representing the own vehicle.

  In step S20 following step S10, the display image generation unit 22 determines whether or not an operation for instructing movement of the movement target has been performed based on the operation signal output from the operation unit 12.

  For example, when the operation unit 12 is a touch panel, when the CG 2 is dragged upward from below the region 1 of the display image with the user's finger, it may be determined that there is an operation of instructing movement with the CG 2 as a movement target. . Further, for example, when a part other than the CG 2 of the display image is dragged down from above the area 1 of the display image with the user's finger, it may be determined that an operation for instructing movement has been performed with the captured image as a movement target.

  When it is determined that there is no operation for instructing the movement of the movement target (NO in step S20), the display image generation unit 22 determines whether the no-operation period has continued for a predetermined time (step S30). If the no-operation period has not continued for the predetermined time (NO in step S30), the process returns to step S20, and if the no-operation period has continued for the predetermined time (YES in step S30), the flow operation ends.

  On the other hand, when it is determined that the operation for instructing the movement of the movement target has been performed (YES in step S20), the movement amount determination unit 22b determines the movement amount of the movement target based on the operation signal output from the operation unit 12. Then, the enlargement / reduction ratio adjustment unit 22c adjusts the enlargement / reduction ratio of the moving object according to the moving amount of the moving object (Step S40). The moving amount includes the moving distance and the moving direction. When the moving object is moved in the traveling direction of the own vehicle with respect to the area 1 of the display image, the moving object is reduced as the moving distance is longer, and the moving object is moved in the reverse direction of the moving direction of the own vehicle with respect to the area 1 of the display image. When moving in the direction, the moving target may be expanded as the moving distance is longer. The reduction ratio or the enlargement ratio of the moving target may be set based on the direction of the optical axis 11a of the front camera 11 in addition to the moving distance of the moving target.

  In step S50 following step S40, the superimposing unit 22d superimposes CG2 on the captured image in a state where the moving target has been moved and the enlargement / reduction ratio of the moving target has been adjusted, and then returns to step S20.

  Therefore, when the operation unit 12 is a touch panel, when the CG 2 is dragged upward from below the region 1 of the display image with the user's finger, the CG 2 gradually decreases as the user moves. The display image shown in FIG. 5 is an example of the display image at the time when the dragging of the CG 2 is completed. When a part other than the CG 2 of the display image is dragged from above to below the display image area 1 with the user's finger, the captured image is gradually enlarged. The display image shown in FIG. 6 is an example of the display image at the time when the drag of the portion other than CG2 of the display image is completed.

  Like the display image shown in FIG. 5 or the display image shown in FIG. 6, by moving the moving target, the CG 2 and obstacles (for example, road cones) in the captured image are arranged along the left-right direction of the vehicle. When the host vehicle attempts to pass through the place where the obstacle exists, the user can easily understand whether the vehicle can pass in advance by looking at the display screen.

  Further, since the moving amount of the moving target can be changed by a user operation, it is possible to simulate whether or not the own vehicle can pass a place where an obstacle exists under various conditions.

  In the present embodiment, either the captured image or CG is set as the moving target, but both the captured image and CG may be set as the moving target. For example, when the operation unit 12 is a touch panel, the pinch-in operation is performed while the user's thumb is touching the CG2 and the user's index finger is touching a part other than the CG2 of the display image, so that both the captured image and the CG2 can be mutually connected. The display image may be moved in the reverse direction with respect to the area 1 of the display image.

<2. Second Embodiment>
FIG. 7 is a diagram illustrating a configuration of the image processing apparatus 202 according to the present embodiment. The image processing device 202 includes a predicted route information acquisition unit 23, and further, the display image generation unit 22 moves the moving target using the output signal of the predicted route information acquisition unit 23 instead of the operation signal output from the operation unit 12. The image processing apparatus 201 is different from the image processing apparatus 201 in that it is different from the image processing apparatus 201.

  The predicted course information acquisition unit 23 obtains steering angle information, vehicle speed information, and the like of the own vehicle from a vehicle control ECU (Electronic Control Unit) of the own vehicle and the like, and obtains information about the predicted course of the own vehicle based on the obtained information. By generating the information, information on the predicted course of the own vehicle is acquired. Note that the predicted route information acquisition unit 23 may obtain information on the predicted route of the vehicle from the vehicle control ECU of the vehicle.

  The predicted route information acquiring unit 23 outputs the acquired information on the predicted route of the own vehicle to the display image generating unit 22. The display image generation unit 22 uses the direction of the predicted course of the own vehicle as the moving direction of the back image CG of the virtual vehicle, or uses the reverse direction of the predicted course of the own vehicle as the moving direction of the captured image.

  When the user confirms the display image generated by the image processing device 202 according to the present embodiment, when the own vehicle attempts to pass the place where the obstacle exists along the expected path, it is possible to pass in advance. Can be easily grasped by the user. In the present embodiment, as in the first embodiment, processing is performed when there is an instruction to display the vehicle CG from the operation unit 12.

<3. Third Embodiment>
FIG. 8 is a diagram illustrating a configuration of the image processing apparatus 203 according to the present embodiment. The image processing device 203 is different from the image processing device 201 in that the image processing device 203 includes an estimation unit 24 and an alarm unit 25, and is basically the same as the image processing device 201 in other points.

  The estimating unit 24 estimates an area where an obstacle exists in the display image. The estimating unit 24 detects an obstacle that may exist in front of the host vehicle based on the image captured by the front camera 11. A known image recognition technique is used for detecting the obstacle candidate object. For example, a background difference method can be used for detecting an obstacle candidate object that is a moving object, and a moving stereo method can be used for detecting an obstacle candidate object that is a stationary object. In the present embodiment, the obstacle is detected by using the image recognition technology. However, in addition to or instead of the image recognition technology, for example, information output from a radar device mounted on the own vehicle is used. An obstacle may be detected using information obtained by communication with a cloud center, inter-vehicle communication, road-to-vehicle communication, and the like.

  The warning unit 25 outputs a warning signal to the speaker 32 based on the positional relationship between the area where the obstacle is estimated by the estimation unit 24 and the rear image CG of the virtual vehicle.

  The speaker 32 provided in the own vehicle outputs an alarm sound according to an alarm signal output from the alarm unit 25.

  FIG. 9 is a flowchart illustrating an operation example of the image processing apparatus 203. The flowchart shown in FIG. 9 is obtained by adding steps S60 and S70 to the flowchart shown in FIG.

  In the flowchart shown in FIG. 9, when the process of step S50 is completed, the process proceeds to step S60. In step S60, the alarm unit 25 determines whether the area where the obstacle estimated by the estimating unit 24 is present overlaps the back image CG of the virtual vehicle.

  If it is determined that the area where the obstacle estimated by the estimating unit 24 is present does not overlap the back image CG of the virtual vehicle (NO in step S60), the process returns to step S20.

  On the other hand, when it is determined that the area where the obstacle estimated by the estimating unit 24 exists and the back image CG of the virtual vehicle overlap (YES in step S60), the warning unit 25 outputs a warning signal ( (Step S70), and then return to step S20.

  According to the image processing apparatus 203 according to the present embodiment, when the own vehicle attempts to pass a place where an obstacle exists, the user can easily grasp in advance whether or not the vehicle can pass by the presence or absence of a warning.

  In the present embodiment, the warning is output by a warning sound, but a warning display may be used as a warning output instead of or in addition to the warning sound. In the present embodiment, a warning is output when the area where the obstacle estimated by the estimating unit 24 is present overlaps with the back image CG of the virtual vehicle. The warning unit 25 may output a warning signal when the distance between the area where the obstacle is present and the back image CG of the virtual vehicle is less than or equal to a predetermined distance. Further, in a configuration in which the warning unit 25 outputs a warning signal when the distance between the area where the obstacle estimated by the estimation unit 24 is present and the back image CG of the virtual vehicle is equal to or smaller than a predetermined distance, May be different from each other in the case where is not overlapped and the case where both are overlapped.

<4. Fourth embodiment>
FIG. 10 is a diagram illustrating a configuration of the image processing device 204 according to the present embodiment. The image processing apparatus 204 is different from the image processing apparatus 204 in that the display image generation unit 22 includes the overhead image unit 22e and the superimposition unit 22d uses the overhead image generated by the overhead image unit 22e instead of the captured image output from the captured image acquisition unit 21. , And is basically the same as the image processing apparatus 201 in the other points.

  The bird's-eye view image unit 22e projects the captured image output from the captured image acquisition unit 21 onto a virtual projection plane and converts it into a projected image. Specifically, the bird's-eye view image unit 22e projects the captured image of the front camera 11 onto a predetermined region R1 of the virtual projection plane 100 in the virtual three-dimensional space illustrated in FIG. Convert to image.

  The virtual projection surface 100 shown in FIG. 11 has, for example, a substantially hemispherical shape (bowl shape). The center part (bottom part of the bowl) of the virtual projection plane 100 is determined as a position where the host vehicle V1 exists. By making the virtual projection plane 100 include a curved surface in this way, it is possible to reduce distortion of an image of an object existing at a position distant from the host vehicle V1.

  The bird's-eye view image unit 22e generates a virtual viewpoint image viewed from the virtual viewpoint based on the projection image. Specifically, the bird's-eye view image unit 22e virtually pastes the projection image into a predetermined region R1 of the virtual projection plane 100.

  The bird's-eye view image unit 22e sets a virtual viewpoint in a virtual three-dimensional space where the virtual projection plane 100 is set. The virtual viewpoint is defined by the viewpoint position and the viewing direction. As long as at least a part of the virtual projection plane 100 enters the visual field, the viewpoint position and the visual field direction of the virtual viewpoint can be set to an arbitrary visual point position and an arbitrary visual field direction.

  In the present embodiment, the viewpoint position of the virtual viewpoint is substantially directly above the own vehicle, and the view direction of the virtual viewpoint is substantially directly below the own vehicle. Thereby, the virtual viewpoint image generated by the bird's-eye view image unit 22e becomes a so-called top-view bird's-eye view image. In the overhead view image of the top view, an object of the same size is photographed in the same size regardless of whether the object exists near the own vehicle or when it exists far from the own vehicle. Therefore, the enlargement / reduction ratio adjustment unit 22c adjusts the enlargement / reduction ratio of the movement target to approximately 1. Alternatively, the scaling ratio adjustment unit 22c does not perform scaling.

  Note that, unlike the present embodiment, the viewpoint position of the virtual viewpoint may be behind and above the own vehicle, and the viewing direction of the virtual viewpoint may be ahead and below the own vehicle. In this case, as in the first embodiment and the like, if the moving target is moved in the traveling direction of the own vehicle with respect to the area 1 of the display image, the moving target is reduced as the moving distance is longer, and the moving target is reduced. If the vehicle is to be moved in the direction opposite to the traveling direction of the host vehicle with respect to the area 1 of the display image, the moving object may be enlarged as the moving distance becomes longer. The reduction ratio or the enlargement ratio of the moving object may be set based on the direction of the optical axis 11a of the front camera 11, the viewpoint position of the virtual viewpoint, and the like, in addition to the moving distance of the moving object.

  The bird's-eye view image unit 22e generates a bird's-eye view image virtually cut out of an image of a necessary region (a region viewed from the virtual viewpoint) on the virtual projection plane 100 according to the set virtual viewpoint.

  FIG. 12 is an example of a display image generated by superimposing the CG generated by the CG generation unit 22a on the top-view overhead image at the initial position. The display image shown in FIG. 13 is an example of the display image at the time when the dragging of the CG 2 is completed. The display image shown in FIG. 14 is an example of the display image at the time when the drag of the portion other than CG2 of the display image is completed.

  According to the image processing device 204 according to the present embodiment, when the own vehicle attempts to pass a place where an obstacle exists, the user can know in advance whether the vehicle can pass by viewing the display image including the bird's-eye view image. Therefore, it is easier to determine whether the own vehicle can pass by looking at the display image including the captured image.

  When the overhead view image is a top view overhead view image, the CG 2 and the obstacle at different distances from the front camera 11 do not overlap with each other, so that the own vehicle can pass through the display image including the overhead view image. It becomes even easier to figure out. In addition, when the overhead image is a top-view overhead image, the enlargement / reduction ratio can be fixed to approximately 1 or the enlargement / reduction can be prevented, so that the burden on the adjustment process of the enlargement / reduction ratio can be reduced. Can be.

<5. Others>
Various technical features disclosed in the present specification can be variously modified without departing from the spirit of the technical creation in addition to the above-described embodiment. In addition, a plurality of embodiments and modifications shown in this specification may be implemented in combination within a possible range.

  For example, in the above-described embodiment, the traveling direction of the own vehicle is forward, but a back camera that captures the rear direction of the own vehicle is provided in the own vehicle, and the image processing apparatus acquires a captured image of the back camera. For example, the present invention can be applied even when the traveling direction of the host vehicle is backward.

  For example, in the embodiment described above, the road cone is shown as an example of the obstacle, but the obstacle is not limited to the road cone as a matter of course. For example, other vehicles, pedestrians, signs, and the like may be included in the obstacle. In addition, for example, by including the railing of the bridge in the obstacle, the user can easily grasp in advance whether the vehicle can cross the narrow bridge. In addition, for example, by including the gutter in the obstacle, the user can easily understand in advance whether the vehicle can pass without derailing.

  For example, in the above-described embodiment, the back image CG of the virtual vehicle is an image that goes straight ahead of the host vehicle, but may be an image that faces in a direction corresponding to the direction in which the moving target moves. Accordingly, the user can more accurately grasp the positional relationship between the back image CG of the virtual vehicle and the obstacle in the display image.

DESCRIPTION OF SYMBOLS 11 Front camera 12 Operation part 21 Camera image acquisition part 22 Display image generation part 201-204 Image processing apparatus

Claims (7)

A photographed image acquisition unit that acquires a photographed image obtained by a camera that photographs the periphery of the own vehicle,
A display image generation unit that generates a display image by superimposing an image of the virtual vehicle representing the host vehicle on the captured image or the converted image obtained by converting the captured image,
With
The display image generation unit can move at least one of the captured image or the converted image and the image of the virtual vehicle with respect to a region of the display image as a movement target, and the scaling ratio of the movement target An image processing device that adjusts according to the movement.   The image processing device according to claim 1, wherein the display image generation unit moves the movement target based on an output of an operation unit that receives a user operation.   The image processing device according to claim 1, wherein the display image generation unit acquires information on an expected course of the own vehicle, and moves the moving target based on the estimated course. An estimating unit that estimates an area where an obstacle exists in the display image,
An alarm unit that outputs an alarm signal based on a positional relationship between the area and the image of the virtual vehicle estimated by the estimation unit,
The image processing apparatus according to claim 1, further comprising: The display image generation unit generates the display image by superimposing an image of the virtual vehicle on the conversion image,
The image processing device according to claim 1, wherein the converted image is a virtual viewpoint image viewed from a virtual viewpoint. When the viewpoint position of the virtual viewpoint is substantially directly above the own vehicle, and the line of sight of the virtual viewpoint is substantially directly below the own vehicle,
The image processing device according to claim 5, wherein the display image generation unit does not perform scaling of the moving target. A captured image obtaining step of obtaining a captured image obtained by a camera that captures the periphery of the vehicle,
A display image generating step of generating a display image by superimposing an image of the virtual vehicle representing the host vehicle on the captured image or the converted image obtained by converting the captured image,
With
The display image generation step includes: when moving at least one of the photographed image or the converted image and the image of the virtual vehicle with respect to the display image area as a movement target, the scaling ratio of the movement target is changed. An image processing method, comprising: an adjusting step of adjusting according to the movement.

Images