JP2019169042A - Viewpoint switching device and viewpoint switching method - Google Patents

Abstract

To provide a viewpoint switching technique capable of easily recognizing how a person around a movable body sees the movable body.SOLUTION: The viewpoint switching device includes an acquisition unit, an estimation unit, and an image generation unit. The acquisition unit acquires a photographed image obtained by a camera photographing the periphery of a movable body. The estimation unit estimates a viewpoint position and a view direction of a person photographed in the photographed image. The image generation unit generates an image viewed from the viewpoint position and the view direction.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for switching the viewpoint of an image.

  2. Description of the Related Art Conventionally, a drive recorder that records a near-miss image has been proposed for the purpose of monitoring a driver's safe driving situation by a transportation company, education on safe driving for a driver, and the like (see, for example, Patent Document 1). A near-miss is an event that is one step ahead of a serious disaster or accident that does not lead to a serious disaster or accident when the driver is driving the vehicle. The name of the near-miss is derived from the fact that the driver who faces the event makes a “close” or “hat”.

JP 2010-79825 A

  The drive recorder proposed in Patent Document 1 stores a photographed image of the front monitoring camera when a near-miss occurs as a near-miss image.

  However, even if the near-miss image stored by the drive recorder proposed in Patent Document 1 is analyzed, the own vehicle can be detected by a person (surrounding person) around the vehicle (own vehicle) on which the drive recorder is mounted. It was not possible to analyze from the viewpoint of how it looked when a near-miss occurred and what the surrounding people had about the vehicle when the near-hat occurred.

  In view of the above problems, an object of the present invention is to provide a viewpoint switching technique capable of easily confirming how a moving body is viewed from a person around the moving body.

  A viewpoint switching device according to the present invention includes an acquisition unit that acquires a captured image obtained by a camera that captures the periphery of a moving body, an estimation unit that estimates a viewpoint position and a viewing direction of a person shown in the captured image, and An image generation unit that generates an image viewed from the viewpoint position and the viewing direction (first configuration).

  In the viewpoint switching device having the first configuration, the estimation unit may be configured to estimate an elevation angle or depression angle in the viewing direction based on a height of the viewpoint position (second configuration).

  In the viewpoint switching device having the first or second configuration, the estimation unit is configured to target the person detected to be performing a predetermined action among the persons shown in the captured image, and the viewpoint position and A configuration for estimating the viewing direction (third configuration) may be used.

  In the viewpoint switching device having any one of the first to third configurations, the estimation unit acquires information related to a viewing direction of a driver who drives the moving body, and the driving among the persons shown in the captured image. It may be a configuration (fourth configuration) for estimating the viewpoint position and the viewing direction for a person who is detected to be directed by the person's line of sight.

  In the viewpoint switching device having any one of the first to fourth configurations, the image generation unit is configured to display an image viewed from the viewpoint position and the viewing direction based on three-dimensional map information and position information of the moving body. Even if the image generated and viewed from the viewpoint position and the viewing direction is an image in which an image representing the moving body is superimposed on the position of the moving body in a map image (fifth configuration) Good.

  In the viewpoint switching device having the fifth configuration, the image generation unit acquires information related to the behavior of the moving object, and adjusts the movement of an image representing the moving object based on the information related to the behavior of the moving object. (6th structure) may be sufficient.

  The viewpoint switching method according to the present invention includes an acquisition process of acquiring a captured image obtained by a camera that captures the periphery of a moving object, an estimation process of estimating a viewpoint position and a viewing direction of a person shown in the captured image, and And an image generation step of generating an image viewed from the viewpoint position and the viewing direction (seventh configuration).

  ADVANTAGE OF THE INVENTION According to this invention, it can confirm easily how the said mobile body is seen from the person around the mobile body.

The figure which shows the example of 1 structure of a communication system Diagram for explaining the outline of the drive recorder The figure which shows the example of 1 structure of a drive recorder Flow chart showing an example of operation of the center server The figure which shows the example of a display of the map image of one's eyes The figure which shows the example of a display of the map image of an other party's eyes The figure which illustrates the position where four in-vehicle cameras are arranged in a vehicle Diagram showing an example of a virtual projection plane

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

<1. Configuration example of communication system>
FIG. 1 is a diagram illustrating a configuration example of a communication system. The communication system shown in FIG. 1 includes a drive recorder 1, a center server 2, and a client server 3. Although only one drive recorder 1 and one client server 3 are shown in FIG. 1, in this embodiment, the center server 2 communicates with a plurality of drive recorders 1 and communicates with a plurality of client servers 3. . When the drive recorder 1 is installed in a taxi, for example, each client server 3 may be arranged in each taxi company.

<2. Configuration example of center server>
As shown in FIG. 1, the center server 2 includes a communication unit 21, a control unit 22, and a storage unit 23.

  The communication unit 21 performs data communication with a plurality of drive recorders 1 and a plurality of client servers 3 via a network (not shown).

  The storage unit 23 stores various types of information in a nonvolatile manner. As the storage unit 23, for example, a hard disk drive or the like can be used. In the present embodiment, the storage unit 23 stores three-dimensional map information 23a, a program 23b, a driver information database 23c, and a near-miss image 23d.

  The three-dimensional map information 23a is map information having data (three-dimensional position, three-dimensional shape, color, etc.) of buildings, road surfaces, white lines, road signs, and road markings. The image generation unit 22c described later can create a map image in an arbitrary viewing direction from an arbitrary viewpoint based on the three-dimensional map information 23a. Here, the viewing direction is a direction in which a visual target such as an image or an object is viewed from the viewpoint position.

  The program 23b is a program for causing the center server 2 to function as a viewpoint switching device.

  The driver information database 23c is a database in which driver identification information (for example, a driver's license number) is associated with driver attribute information (for example, gender, age, etc.).

  The near-miss image 23d is a photographed image when a near-miss occurs, and is an image uploaded from the drive recorder 1 to the center server 2. The near-miss image 23d may be a still image when a near-miss occurs, but is preferably a moving image of a certain period from just before the near-miss occurs to immediately after. This is because, by making the near-miss image 23d as a moving image as described above, monitoring of the driver's safe driving situation by the carrier, education on the driver's safe driving, and the like become more effective.

  The control unit 22 is a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory) (not shown). The control unit 22 is connected to the storage unit 23 and the like, and processes and transmits / receives information based on the program 23b stored in the storage unit 23, thereby controlling the entire center server 2.

  The control unit 22 includes an acquisition unit 22a, an estimation unit 22b, and an image generation unit 22c. When the CPU executes arithmetic processing according to the program 23b stored in the storage unit 23, various functions of the control unit 22 such as the acquisition unit 22a are realized, and the center server 2 functions as a viewpoint switching device.

  The acquisition unit 22a acquires a captured image obtained by a camera that is mounted on the vehicle together with the drive recorder 1 and captures the periphery of the vehicle.

  The estimation unit 22b estimates the viewpoint position and viewing direction of the person shown in the captured image acquired by the acquisition unit 22a.

  The image generation unit 22c generates an image viewed from the viewpoint position and the viewing direction estimated by the estimation unit 22b.

<3. Example of drive recorder configuration>
FIG. 2 is a diagram for explaining an outline of the drive recorder 1. The drive recorder 1 is mounted on the vehicle 4. The drive recorder 1 is disposed at an appropriate position of the vehicle 4.

  The drive recorder 1 acquires an image around the vehicle photographed by the camera 5 mounted on the vehicle 4. The recording method of the drive recorder 1 may be, for example, a constant recording method in which information related to the driving situation including images taken by the camera 5 is periodically recorded on a recording medium (not shown). Lane departure), sudden braking, collision, and other events are detected. This is an event recording method that records information on driving conditions other than images captured by the camera 5 and images including event contents on a recording medium. May be. When a near-miss is detected, the drive recorder 1 uploads an image captured by the camera 5 to the center server 2 as a near-miss image. The drive recorder 1 associates information on the driving situation such as the driving location with the near-miss image. The near-miss detection is performed based on, for example, sudden braking, sudden steering, approach to a pedestrian or another vehicle, and the like.

  FIG. 3 is a diagram illustrating a configuration example of the drive recorder 1. The drive recorder 1 includes a microcomputer as a control unit that controls the entire apparatus. Specifically, the drive recorder 1 includes a CPU 11 that realizes various control functions by performing arithmetic processing, a RAM 12 that is a work area for arithmetic processing, and a nonvolatile memory 13 that stores various data. The nonvolatile memory 13 is composed of, for example, a hard disk or a flash memory. The nonvolatile memory 13 stores a program as firmware, setting parameters, and the like. The function of controlling each part of the drive recorder 1 is realized by the CPU 11 executing arithmetic processing according to a program stored in advance in the nonvolatile memory 13.

  A camera 5 and a microphone 6 are connected to the drive recorder 1. The camera 5 and the microphone 6 may be included in the drive recorder 1. The camera 5 includes a lens and an image sensor, and can acquire image data electronically. In the present embodiment, the camera 5 is disposed in the vicinity of the upper portion of the windshield with its optical axis directed toward the front of the vehicle 4. The number and arrangement of the cameras 5 may be changed as appropriate. For example, a front camera for photographing the front and a back camera for photographing the rear may be provided. For example, the microphone 6 is disposed at a proper position of the main body of the drive recorder 1 or the vehicle 4. The microphone 6 collects sound near the vehicle and acquires audio data. The number and arrangement of the microphones 6 may be changed as appropriate. For example, an external sound microphone that mainly collects sound generated outside the vehicle 4 and an in-vehicle sound microphone that mainly collects sound emitted by the driver may be provided.

  The drive recorder 1 includes an image processing unit 14 that processes image data captured by the camera 5. In the present embodiment, the image processing unit 14 is a hardware circuit. The image processing unit 14 performs predetermined image processing on a signal of image data input from the camera 5 to generate digital image data in a predetermined format. The predetermined image processing includes, for example, A / D conversion, luminance correction, contrast correction, and the like. The predetermined format may be, for example, the JPEG format. The image data processed by the image processing unit 14 is stored in the RAM 12.

  A part of the storage area of the RAM 12 is used as a ring buffer. In the ring buffer, image data processed by the image processing unit 14 and audio data acquired by the microphone 6 are always stored. In the ring buffer, when data is stored up to the last area, it returns to the first area and new data is stored. That is, in the ring buffer, new data is sequentially overwritten on the oldest data. For this reason, the RAM 12 is always in a state where image data and audio data for a certain past time are stored.

  The drive recorder 1 is connected to an operation unit 7 that receives an instruction from a driver or the like. The operation unit 7 may be included in the drive recorder 1. The operation unit 7 is disposed at an appropriate position of the vehicle 4 such as in the vicinity of a steering wheel (not shown) so that the driver can easily operate the operation unit 7. Further, the drive recorder 1 includes a card slot 15, a timing circuit 16, an acceleration sensor 17, a GPS (Global Positioning System) receiving unit 18, and a communication unit 19.

  The card slot 15 is configured so that the memory card 8 can be attached and detached. Instead of the memory card 8, another recording medium such as a hard disk drive, an optical disk, a magneto-optical disk, or a flexible disk may be used.

  Image data, audio data, and the like stored in the ring buffer of the RAM 12 are periodically recorded in the memory card 8 mounted in the card slot 15 according to an instruction from the CPU 11.

  The timer circuit 16 generates a signal corresponding to the current time and outputs it to the CPU 11.

  The acceleration sensor 17 detects acceleration indicating the magnitude of impact applied to the vehicle 4 in units of G of gravitational acceleration. The detected acceleration is an instantaneous value of acceleration at the current time, and has a magnitude in a direction corresponding to, for example, three or two axes orthogonal to each other. The acceleration sensor 17 outputs a signal corresponding to such acceleration to the CPU 11.

  The GPS receiver 18 receives signals from a plurality of GPS satellites, and acquires the vehicle position that is the position of the vehicle 4 at the current time. The GPS receiving unit 18 acquires the vehicle position as position information expressed by longitude and latitude on the earth, and outputs the acquired position to the CPU 11. The GPS receiving unit 18 can detect an accurate time based on a signal from a GPS satellite. For this purpose, the GPS receiver 18 may be used in place of the timer circuit 16. The GPS receiver 18 may be used to correct the time of the time measuring circuit 16.

  The communication unit 19 transmits / receives data to / from the communication unit 21 of the center server 2 via the network.

  In addition, vehicle travel information including data from a vehicle speed sensor is input to the CPU 11 via a not-shown CAN (Controller Area Network) bus. The vehicle travel information may include steering information, brake information, accelerator information, and the like. Further, the CPU 11 may be connected to a navigation device or a vehicle control ECU (Electronic Control Unit) via a CAN (not shown).

<4. Operation example of center server>
FIG. 4 is a flowchart illustrating an operation example of the center server 2. The center server 2 performs the flow operation shown in FIG. 4 every time the communication unit 21 receives a near-miss image from the drive recorder 1.

  First, the acquisition unit 22a acquires a near-miss image (step S10).

  Next, the image generation unit 22c determines the vehicle 4 from the viewpoint of the driver driving the vehicle 4 based on the traveling position information attached to the near-miss image, the traveling direction information of the vehicle 4, and the three-dimensional map information 23a. A map image in the viewing direction along the traveling direction (hereinafter referred to as a map image of one's own eyes) is created (step S20). In addition, the image generation unit 22c recognizes the near-miss image, extracts objects that are not included in the three-dimensional map information such as pedestrians, bicycles, and other vehicles, and generates a CG (Computer Graphics) image corresponding to the objects. Is superimposed on the position where the object should exist in the map image of the user's line of sight. Therefore, for typical objects such as pedestrians, bicycles, and other vehicles, it is preferable to store each CG image corresponding to each object in the storage unit 23 in advance. By using a map image instead of a near-miss image that is a captured image, it is possible to protect the privacy of a person or the like reflected in the near-miss image.

  FIG. 5 is a display example of a map image of one's own eyes. The display of the map image of the self-view is performed by a display device connected to the client server 3. 5 includes a CG image A1 corresponding to a bicycle. The self-viewing map image shown in FIG. 5 also includes a software key A2 for switching the display to a partner-viewing map image to be described later. By operating the software key A2 using an operation device such as a touch panel or a pointing device, the display can be switched from the map image of the user's line of sight shown in FIG.

  Returning to FIG. 4, in step S <b> 30 following step S <b> 20, the estimation unit 22 b estimates the viewpoint position and viewing direction of the person shown in the near-miss image acquired by the acquisition unit 22 a. When there are a plurality of persons shown in the near-miss image acquired by the acquisition unit 22a, the estimation unit 22b estimates the viewpoint position and viewing direction of each person. In addition, the estimation unit 22b recognizes the near-miss image, extracts a person appearing in the near-miss image, estimates the position of the viewpoint from, for example, the extracted position, and, for example, the face direction of the person shown in the near-miss image And the viewing direction may be estimated from the direction of the recognized face.

  It is desirable that the estimating unit 22b first estimates the viewpoint position, and then estimates the elevation angle or depression angle of the viewing direction based on the estimated height of the viewpoint position. For example, if the person shown in the near-miss image acquired by the acquisition unit 22a is a child, the height of the viewpoint position is lower than that of an adult, so that it is estimated that the elevation angle in the viewing direction is increased. Good. In addition, for example, when the person in the near-miss image acquired by the acquisition unit 22a is a driver on a truck, the viewpoint position is higher than the case of a pedestrian, What is necessary is just to estimate that the depression angle of a direction becomes large.

  By estimating the elevation angle or depression angle of the viewing direction based on the height of the viewpoint position as described above, it is possible to estimate the elevation angle or depression angle of the viewing direction that is difficult to estimate from the orientation of the face in the horizontal direction. Become.

  In step S40 subsequent to step S30, the image generation unit 22c detects the near-miss acquired by the acquisition unit 22a based on the traveling position information attached to the near-miss image, the estimation result of the estimation unit 22b, and the three-dimensional map information 23a. A map image of the viewpoint and viewing direction of a person shown in the image (hereinafter, a map image of the other party's line of sight) is created. By confirming the map image of the other party's line of sight, it is possible to easily confirm how the vehicle 4 is seen by those around the vehicle 4.

  When there are a plurality of persons in the near-miss image acquired by the acquisition unit 22a, the image generation unit 22c creates a map image of each person's eyes. Moreover, the image generation part 22c superimposes the CG image corresponding to the vehicle 4 on the position where the vehicle 4 should exist in the map image of the opponent's line of sight. Therefore, a CG image corresponding to the vehicle 4 may be stored in the storage unit 23 in advance. If there is a position where an object not included in the three-dimensional map information such as a pedestrian, a bicycle, or another vehicle is present in the map image of the other party's line of sight, the image generating unit 22c As in the case, the CG image corresponding to the object is superimposed on the position where the object should exist in the map image of the other party's line of sight. By using a map image instead of a near-miss image that is a captured image, it is possible to protect the privacy of a person or the like reflected in the near-miss image.

  FIG. 6 is a display example of a map image of the other party's line of sight. The display of the map image of the other party's eye is performed by a display device connected to the client server 3. The map image of the other party's line of sight shown in FIG. 6 includes a CG image A3 corresponding to the vehicle. 6 also includes a software key A4 for switching the display to the map image of the user's own line shown in FIG. 5 described above. By operating the software key A4 using an operation device such as a touch panel or a pointing device, the display can be switched from the map image of the partner line of sight shown in FIG. 6 to the map image of the partner line of sight shown in FIG. .

  In addition, when the map image of the opponent's line of sight is a moving image, the image generation unit 22c is based on the traveling position of the vehicle 4 attached to the near-miss image, and the position where the vehicle 4 should exist in the map image of the opponent's However, since the accuracy of the traveling position of the vehicle 4 is not so high, the movement of the CG image corresponding to the vehicle 4 in the map image of the opponent's line of sight cannot be matched with the actual movement of the vehicle 4. Therefore, when the map image of the opponent's line of sight is a moving image, the image generation unit 22c determines the opponent's line of sight based on information (steering information, brake information, accelerator information, etc.) related to the behavior of the vehicle 4 attached to the near-miss image. It is desirable to adjust the position where the vehicle 4 should exist in the map image. Thereby, even if the accuracy of the traveling position of the vehicle 4 is not so high, the movement of the CG image corresponding to the vehicle 4 in the map image of the opponent's line of sight can be brought close to the actual movement of the vehicle 4.

  In step S50 following step S40, the control unit 22 associates the map image of the subject's line of sight and the map image of the partner's line of sight related to the same near-miss image. Further, in this association, information associated with the near-miss image is also associated with the map image of the subject's eyes and the map image of the opponent's eyes.

  The center server 2 may transmit the map image of the user's line of sight and the map image of the other party's line of sight to the client server 3 every time the association in step S50 is completed, (For example, once a week, once a month, etc.) may be transmitted to the client server 3. For example, when the drive recorder 1 is mounted on a taxi, the map image transmitted from the center server 2 to each client server 3 is protected for privacy as described above, and therefore belongs to a taxi company different from the client server 3. It is desirable to include a map image related to the near-miss image transmitted from the drive recorder 1 mounted on the taxi. As a result, the map image related to the near-miss image transmitted from the drive recorder 1 installed in a taxi belonging to a different taxi company can also be confirmed. Monitoring of driver's safe driving situation and education for driver's safe driving become more effective.

<5. Modification>
The above-described embodiment is an example in all respects and should not be considered as limiting, and the technical scope of the present invention is indicated by the scope of the claims, not the description of the above-described embodiment. Therefore, it should be understood that all modifications that come within the meaning and range of equivalency of the claims are embraced.

  In the embodiment described above, when there are a plurality of persons in the near-miss image acquired by the acquisition unit 22a, the estimation unit 22b estimates the viewpoint position and viewing direction of each person. However, when there are many persons appearing in the near-miss image acquired by the acquisition unit 22a, a large number of map images of the opponent's line of sight are also created, and it is complicated to check the map image of the opponent's line of sight.

  Therefore, the estimation unit 22b estimates the viewpoint position and the viewing direction for a person who has been detected to be performing a predetermined action among the persons shown in the near-miss image acquired by the acquisition unit 22a. May be. This makes it possible to limit the map image of the other party's line of sight to, for example, an image viewed from the viewpoint position and viewing direction of a person who feels danger. Examples of the predetermined action include a sudden stop for a pedestrian and a sudden brake operation or a sudden steering operation for a driver on the vehicle. The detection of the predetermined action can be executed by image recognition of a near-miss image, for example.

  In addition to or in place of the above-described deformation, the estimation unit 22b acquires information on the viewing direction of the driver driving the vehicle 4 and is reflected in the near-miss image acquired by the acquisition unit 22a. You may make it estimate a viewpoint position and a viewing direction for the person in which it was detected that the eyes | visual_axis of the driver who drives the vehicle 4 was turned among the persons. This makes it possible to limit the map image of the other party's line of sight to, for example, an image viewed from the viewpoint position and viewing direction of the person that the driver driving the vehicle 4 noticed during the near-miss. As for the viewing direction of the driver driving the vehicle 4, for example, a camera that monitors the driver is provided in the vehicle 4, and the viewing direction of the driver driving the vehicle 4 can be detected from a captured image of the camera that monitors the driver. By causing information about the viewing direction of the driver driving the vehicle 4 to accompany the near-miss image, the estimation unit 22b can acquire information about the viewing direction of the driver driving the vehicle 4.

  For example, the camera 5 does not necessarily have to be mounted on an automobile, and may be any camera that can be used for a moving body such as a motorcycle, a railway, and a ship in addition to the automobile.

  For example, in the above-described embodiment, viewpoint switching is performed for a near-miss image, but the target image for viewpoint switching is not limited to the near-hat image, and for example, the viewpoint switching is performed when an traffic accident occurs. The target image may be used.

  For example, in the above-described embodiment, viewpoint switching is performed using 3D map information, but viewpoint switching may be performed by converting a captured image into a virtual viewpoint image.

  When converting a captured image into a virtual viewpoint image, captured images captured by a plurality of in-vehicle cameras are used. FIG. 7 is a diagram illustrating positions where four on-vehicle cameras (front camera 51, back camera 52, left side camera 53, and right side camera 54) are arranged on the vehicle 4.

  The front camera 51 is provided at the front end of the vehicle 4. The optical axis 51a of the front camera 51 is along the front-rear direction of the vehicle 4 in plan view from above. The front camera 51 captures the front direction of the vehicle 4. The back camera 52 is provided at the rear end of the vehicle 4. The optical axis 52a of the back camera 52 is along the front-back direction of the vehicle 4 in plan view from above. The back camera 52 captures the rear direction of the vehicle 4. The mounting position of the front camera 51 and the back camera 52 is preferably the left and right center of the vehicle 4, but may be a position slightly deviated in the left and right direction from the left and right center.

  The left side camera 53 is provided on the left door mirror M <b> 1 of the vehicle 4. The optical axis 53a of the left side camera 53 is along the left-right direction of the vehicle 4 in plan view from above. The left side camera 53 captures the left direction of the vehicle 4. The right side camera 54 is provided on the right door mirror M2 of the vehicle 4. The optical axis 54a of the right side camera 54 is along the left-right direction of the vehicle 4 in plan view from above. The right side camera 54 captures the right direction of the vehicle 4. When the vehicle 4 is a so-called door mirrorless vehicle, the left side camera 53 is attached around the rotation axis (hinge portion) of the left side door without a door mirror, and the right side camera 54 is a right side door. It is attached without a door mirror around the rotating shaft (hinge part).

  The horizontal angle of view θ of each in-vehicle camera is 180 degrees or more. Therefore, the entire periphery of the vehicle 4 in the horizontal direction can be photographed by the four in-vehicle cameras (front camera 51, back camera 52, left side camera 53, and right side camera 54). Here, the number of in-vehicle cameras is four, but the number of in-vehicle cameras necessary for creating a virtual viewpoint image to be described later using a captured image of the in-vehicle camera is not limited to four. A plurality of cameras may be used. As an example, when the horizontal angle of view θ of each in-vehicle camera is relatively wide, a virtual viewpoint image may be generated based on three captured images acquired from fewer than three cameras. Furthermore, as another example, when the horizontal angle of view θ of each in-vehicle camera is relatively narrow, a virtual viewpoint image may be generated based on five captured images obtained from more than five cameras. .

  The image generation unit 22c of the center server 2 projects four captured images on a virtual projection plane and converts them into projection images. Specifically, the image generation unit 22c projects the captured image of the front camera 51 onto the first region R1 of the virtual projection plane 100 in the virtual three-dimensional space illustrated in FIG. Conversion to the first projection image. Similarly, the image generation unit 22c captures the captured image of the back camera 52, the captured image of the left side camera 53, and the captured image of the right side camera 54 in each of the second to fourth regions R2 to R4 of the virtual projection plane 100 illustrated in FIG. The captured image is projected, and the captured image of the back camera 52, the captured image of the left side camera 53, and the captured image of the right side camera 54 are converted into second to fourth projected images.

  The virtual projection plane 100 shown in FIG. 8 has, for example, a substantially hemispherical shape (a bowl shape). A central part (bottom part of the bowl) of the virtual projection plane 100 is determined as a position where the vehicle 4 exists. Thus, by making the virtual projection plane 100 include a curved surface, it is possible to reduce the distortion of the image of the object existing at a position away from the vehicle 4. Each of the first to fourth regions R1 to R4 has a portion that overlaps with another adjacent region. By providing such an overlapping portion, it is possible to prevent the image of the object projected on the boundary portion of the region from disappearing from the projected image.

  The image generation unit 22c generates a virtual viewpoint image viewed from a virtual viewpoint based on a plurality of projection images. Specifically, the image generation unit 22c virtually pastes the first to fourth projection images on the first to fourth regions R1 to R4 of the virtual projection plane 100.

  Further, the image generation unit 22 c virtually configures a polygon model indicating the three-dimensional shape of the vehicle 4. In the model of the vehicle 4, the first region R <b> 1 is located on the front side at a position (center portion of the virtual projection plane 100) determined as a position where the vehicle 4 exists in a virtual three-dimensional space where the virtual projection plane 100 is set. The fourth region R4 is disposed on the rear side.

  Furthermore, the image generation unit 22c sets a virtual viewpoint in a virtual three-dimensional space where the virtual projection plane 100 is set. The virtual viewpoint is defined by the viewpoint position and the viewing direction. As long as at least a part of the virtual projection plane 100 enters the field of view, the viewpoint position and viewing direction of the virtual viewpoint can be set to an arbitrary viewpoint position and arbitrary viewing direction.

  The image generation unit 22c virtually cuts out an image of a necessary area (area seen from the virtual viewpoint) on the virtual projection plane 100 according to the set virtual viewpoint. In addition, the image generation unit 22 c performs rendering on the polygon model according to the set virtual viewpoint, and generates a rendered image of the vehicle 4. And the image generation part 22c produces | generates the virtual viewpoint image which superimposed the rendering image of the vehicle 4 with respect to the cut-out image.

DESCRIPTION OF SYMBOLS 1 Drive recorder 2 Center server 3 Client server 4 Vehicle 5 Camera 21 Communication part 22 Control part 23 Storage part

Claims (7)

An acquisition unit that acquires a captured image obtained by a camera that captures the periphery of the moving object;
An estimation unit for estimating a viewpoint position and a viewing direction of a person shown in the captured image;
An image generator for generating an image viewed from the viewpoint position and the viewing direction;
Comprising a viewpoint switching device.   The viewpoint switching apparatus according to claim 1, wherein the estimation unit estimates an elevation angle or depression angle in the viewing direction based on a height of the viewpoint position.   The estimation unit estimates the viewpoint position and the viewing direction with respect to a person who is detected to be taking a predetermined action among the persons shown in the captured image. The viewpoint switching device described in 1.   The estimation unit acquires information related to the viewing direction of the driver who drives the moving body, and targets a person who is detected to be directed to the driver's line of sight among the persons shown in the captured image. The viewpoint switching device according to claim 1, wherein the viewpoint position and the viewing direction are estimated. The image generation unit generates an image viewed from the viewpoint position and the viewing direction based on the three-dimensional map information and the position information of the moving body,
The image viewed from the viewpoint position and the viewing direction is an image in which an image representing the moving body is superimposed on the position of the moving body in a map image. Viewpoint switching device.   The viewpoint switching device according to claim 5, wherein the image generation unit acquires information related to the behavior of the moving object, and adjusts the movement of an image representing the moving object based on the information related to the behavior of the moving object. An acquisition step of acquiring a captured image obtained by a camera that captures the periphery of the moving object;
An estimation step of estimating a viewpoint position and a viewing direction of a person shown in the captured image;
An image generation step of generating an image viewed from the viewpoint position and the viewing direction;
A viewpoint switching method comprising:

Images