WO2010007960A1

WO2010007960A1 - View-point conversion video image system for camera mounted on vehicle and method for acquiring view-point conversion video image

Info

Publication number: WO2010007960A1
Application number: PCT/JP2009/062661
Authority: WO
Inventors: 義隆奥山
Original assignee: クラリオン株式会社
Priority date: 2008-07-14
Filing date: 2009-07-13
Publication date: 2010-01-21
Also published as: JPWO2010007960A1

Abstract

A view-point conversion video image system of a camera mounted on a vehicle is provided with a camera (11) mounted and fixed on a vehicle, a monitor device (14) that displays an image, and a view-point conversion processing unit (12) to which live image data from the camera (11) mounted on the vehicle is inputted and which carries out the view-point processing of the image data in accordance with mapping data for the generation of a view-point conversion image and outputs the view-point conversion data to the monitor device (14). Further, the view-point conversion video image system is also provided with a parameter input device (13) that inputs a parameter which represents the position and the posture of the camera (11) mounted on the vehicle with respect to a three-dimensional space and which is used for generating the mapping data. A method for acquiring a view-point conversion video image is comprised of an input step for inputting an input parameter for the generation of mapping data, a second display step for displaying the view-point conversion, and a judgment step for judging whether the view-point conversion image is suitable or not, thereby storing or making it possible to re-execute the same.

Description

In-vehicle camera viewpoint conversion video system and viewpoint conversion video acquisition method

According to the present invention, a customer mounts and fixes an in-vehicle camera on a vehicle, and a viewpoint conversion image is viewed by freely converting a viewpoint of an image acquired by the in-vehicle camera fixed on the vehicle while viewing a screen of a monitor device. The present invention relates to a viewpoint conversion video system and a viewpoint conversion video acquisition method for a vehicle-mounted camera.

In recent years, an in-vehicle camera is mounted on and fixed to a vehicle, the surroundings of the vehicle are photographed by the in-vehicle camera, a viewpoint conversion image is displayed on the screen of the monitor device by converting the viewpoint of the captured image of the surroundings of the vehicle. A system has been developed.

This viewpoint conversion video system, for example, presents a viewpoint conversion image around the vehicle to the screen of the monitor device to allow the driver to recognize and detect the approach and presence of obstacles, and to provide services such as driving safety Used.

Conventionally, in this type of viewpoint conversion video system, a calibration process is performed in order to precisely measure the distance to an obstacle or to superimpose images acquired by a plurality of in-vehicle cameras. Here, the calibration process refers to a process of acquiring a parameter for correcting an attachment error of the in-vehicle camera to the vehicle.

This calibration process is performed on each vehicle before shipment from the factory, as in Patent Document 1, for example. In this calibration process, a reference subject including a so-called reference point (or grid point) whose spatial coordinate position is known is photographed by the on-vehicle camera, and then the third order of the on-vehicle camera is obtained based on the image obtained by this photographing. The attitude to the original space is adjusted. In this adjustment process, for example, the 3D of the in-vehicle camera is adjusted so that the adjustment marker imitating the contour of the rear bumper matches the raw image actually acquired by the in-vehicle camera attached and fixed to the vehicle body. The parameter that means the attitude to the space is adjusted. And the parameter obtained by this adjustment is memorize | stored in the parameter memory | storage part (memory).

However, the calibration process has a problem that a considerable space is required for photographing the reference subject, which takes time and effort.

Further, in order to obtain a bird's-eye view image from which the distortion of the raw image has been removed by viewpoint conversion processing, an in-vehicle camera 2 is installed at the rear of the vehicle 1 as shown in FIG. The imaging subject 3 is imaged, a reference subject image 5 ′ including the known reference point 4 in FIG. 2 is acquired by this imaging, and then a calibration process is performed based on the reference subject image 5 ′, so that the vehicle is mounted on the vehicle. Camera viewpoint conversion video systems are also known.
That is, in the viewpoint conversion video system of the in-vehicle camera, the reference point of the reference subject image 5 ′ is obtained by performing the image recognition process on the reference subject image 5 ′ of FIG. 4 is obtained, the parameter indicating the attitude of the vehicle-mounted camera 2 with respect to the three-dimensional space is adjusted based on the three-dimensional coordinate of the reference point 4, and the parameter obtained by this adjustment is stored in the parameter storage unit (memory). To remember.
Moreover, in this in-vehicle camera viewpoint conversion video system, the viewpoint of the virtual camera 5 is determined, and the parameters of the virtual camera 5 are manipulated so that the attitude of the virtual camera 5 with respect to the three-dimensional space as shown in FIG. It is also known to adjust and acquire an appropriate viewpoint-converted image (overhead image) 6 from which distortion is removed as shown in FIG.
FIG. 5 shows an example in which the calibration process using the parameters of the virtual camera 5 is inappropriate. 4 and 5,

reference numerals

7 and 8 denote image areas that cannot be acquired by the vehicle-mounted camera 2.

JP 2007-256030 A

By the way, the attitude of the in-vehicle camera 2 with respect to the three-dimensional space includes the three-dimensional coordinate position of the in-vehicle camera 2, the tilt angle of the optical axis O of the in-vehicle camera 2 with respect to the horizontal and vertical planes, and the optical axis of the in-vehicle camera 2 It can be expressed using a rotation angle or the like.
In this case, the three-dimensional coordinate position of the vehicle-mounted camera 2 is represented by x, y, and z coordinates, and the tilt angle of the optical axis O of the vehicle-mounted camera 2 with respect to the horizontal plane is represented by the pitch angle θ. The tilt angle of the optical axis O with respect to the vertical plane is represented by the yaw angle φ, and the rotation angle around the optical axis of the in-vehicle camera 2 is represented by the roll angle ψ.
Therefore, the attitude of the in-vehicle camera 2 with respect to the three-dimensional space can be expressed using six parameters (x, y, z, θ, ψ, φ).
In the viewpoint conversion video system, the above-described six parameters (x, y, z, θ, ψ, φ) of the vehicle-mounted camera 2 and the virtual camera 5 are used to generate a viewpoint conversion image (overhead image). Mapping data for generating a viewpoint converted image is generated using the six parameters.
However, in such a viewpoint conversion video system, raw image data actually acquired by the in-vehicle camera 2 must be processed based on the mapping data, and it takes time to generate this mapping data. There is a disadvantage that the viewpoint conversion image cannot be displayed in real time on the screen of the monitor device by performing the viewpoint conversion processing on the raw image data acquired by the camera 2.

In addition, this in-vehicle camera viewpoint conversion video system requires two steps of the calibration process for the in-vehicle camera (real camera) 2 and the calibration process for the virtual camera 5, so that adjustment is extremely troublesome. In addition, there is a problem that it is extremely difficult for the customer himself to perform the calibration process and acquire the viewpoint conversion image.

That is, there is a problem that the customer cannot easily calibrate the in-vehicle camera, and the viewpoint conversion video system for the in-vehicle camera that is easy for the customer to handle is not available.

An object of the present invention is to provide a viewpoint-converted video system and a viewpoint-converted video acquisition method for a vehicle-mounted camera that can be easily handled by a customer and that allows a customer to freely acquire a viewpoint-converted image without requiring calibration processing of the vehicle-mounted camera. There is.

The viewpoint conversion video system for an in-vehicle camera according to the present invention includes an in-vehicle camera that is attached and fixed to a vehicle, and a monitor device that displays an image. In addition, in order to achieve the above-described object, this viewpoint conversion video system receives the raw image data from the in-vehicle camera and performs viewpoint conversion processing on the image data based on the mapping data for generating the viewpoint conversion image. A viewpoint conversion processing unit that outputs the viewpoint-converted image data to the monitor device, a parameter input device that inputs a parameter for generating the mapping data, which means the position and orientation of the in-vehicle camera in the three-dimensional space; Have

According to this in-vehicle camera viewpoint conversion video system, since the parameter input device for inputting parameters for viewpoint conversion processing is provided, the viewpoint conversion processing can be automatically performed only by inputting parameters. Can be handled easily by the customer.

In addition, since the viewpoint conversion image can be displayed on the monitor device in real time, the parameters for generating mapping data can be adjusted while viewing the viewpoint conversion image, which is easy for the customer to handle and the customer can freely convert the viewpoint conversion image. It can be acquired.

Also, the viewpoint conversion video acquisition method of the in-vehicle camera according to the present invention is a monitor device that receives raw image data from an in-vehicle camera attached and fixed to a vehicle and performs viewpoint conversion processing to convert the viewpoint. The viewpoint conversion processing unit that outputs to In addition, the viewpoint conversion video acquisition method for the in-vehicle camera of the present invention includes the following first display step, input step, second display step, and determination step in order to achieve the above-described object. In the first display step, the live camera acquired in real time based on the initial parameters that mean the position and orientation of the in-vehicle camera with respect to the three-dimensional space and that generates mapping data for generating a viewpoint-converted image. A viewpoint conversion process is performed on the image data to display a viewpoint conversion image on the screen of the monitor device. Further, in the input step, in order to change the viewpoint conversion image obtained by the initial parameters, the viewpoint conversion image means the position and orientation of the vehicle-mounted camera with respect to the three-dimensional space and the mapping data is changed Enter the input parameters to generate. Further, in the second display step, the viewpoint conversion processing is performed on the image data acquired by the vehicle-mounted camera in real time based on the input parameters input in the input step, and the viewpoint conversion image is displayed on the screen of the monitor device. . In the determination step, it is determined whether or not the viewpoint conversion image displayed on the screen of the monitor device in the second display step is appropriate. When the viewpoint conversion image is appropriate, the parameter is stored in the parameter storage unit. When the viewpoint conversion image is inappropriate, the input step can be re-executed.

According to the viewpoint conversion video acquisition method of the in-vehicle camera of the present invention, since the viewpoint conversion image can be displayed on the monitor device in real time, the mapping data generation parameters can be adjusted while viewing the viewpoint conversion image. It is easy for the customer to handle and the customer can freely acquire the viewpoint conversion image.

It is explanatory drawing for demonstrating an example of the conventional viewpoint conversion video system, Comprising: A vehicle-mounted camera is attached to the rear part of a vehicle, and it is a schematic diagram which shows the imaging | photography state of the reference object of a vehicle rear part. It is a figure which shows the raw image imaged with the vehicle-mounted camera shown in FIG. It is a figure for demonstrating the conventional concept of the viewpoint conversion process with respect to the raw image obtained with the vehicle-mounted camera. It is a figure which shows the appropriate viewpoint conversion image obtained by the viewpoint conversion process using the virtual camera. It is a figure which shows the inappropriate viewpoint conversion image obtained by the viewpoint conversion process using the virtual camera. It is a schematic diagram for demonstrating schematic structure of the viewpoint conversion video system concerning this invention. It is a figure for demonstrating the concept of the viewpoint conversion process of the viewpoint conversion video system concerning this invention. It is a block diagram which shows the detailed structure of the viewpoint conversion video system shown in FIG. It is the schematic diagram used in order to demonstrate the basic concept of a viewpoint conversion process. It is a schematic diagram of a remote control device as an example of the parameter input device shown in FIG. It is a flowchart figure for demonstrating the process sequence of the viewpoint conversion video system concerning this invention. It is explanatory drawing which shows an example of the viewpoint conversion image by which viewpoint conversion was carried out by the initial parameter used for the viewpoint conversion video system concerning this invention. It is explanatory drawing which shows an example of the inappropriate viewpoint conversion image by which viewpoint conversion was carried out with the input parameter used for the viewpoint conversion video system concerning this invention. It is explanatory drawing which shows an example of the appropriate viewpoint conversion image by which viewpoint conversion was carried out with the input parameter used for the viewpoint conversion video system concerning this invention. It is a block diagram which shows the other detailed structure of the viewpoint conversion video system shown in FIG. It is explanatory drawing which shows the three-dimensional coordinate position adjustment screen of the viewpoint conversion video system shown in FIG. It is explanatory drawing which shows the attitude | position adjustment screen of the viewpoint conversion video system shown in FIG. It is explanatory drawing which shows the viewpoint conversion image acquired by operating the instruction | indication button of the attitude | position adjustment screen shown in FIG. It is explanatory drawing of the reduction screen displayed on the screen of the monitor apparatus shown in FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Example 1]
As shown in FIG. 6, the viewpoint conversion video system 10 according to the present invention includes a vehicle-mounted camera 11, a viewpoint conversion processing unit (hardware device) 12, and a parameter input device 13. The in-vehicle camera 11 has a known structure and a wide-angle lens type. The internal configuration of the viewpoint conversion processing unit 12 will be described later. As the parameter input device 13, for example, a wired remote control device is used, but a wireless remote control device may be used.

In this viewpoint conversion video system 10, a customer purchases an assembly set for configuring a viewpoint conversion video system including a vehicle-mounted camera 11, a viewpoint conversion processing unit 12, a parameter input device 13 and the like from a dealer, It can be constructed by assembling.

The customer needs to purchase the monitor device 14 as necessary. However, when the monitor device 14 corresponding to the viewpoint conversion video system 10 is already installed, this is not necessary. FIG. 6 shows the electrical connection relationship of the viewpoint conversion video system 10. In FIG. 6,

reference numerals

15, 16, and 17 are connection cables, and 18 is an image display unit (display screen) of the monitor device 14.

FIG. 7 shows a state in which the vehicle-mounted camera 11 is attached and fixed to the rear portion 1A of the vehicle 1. The in-vehicle camera 11 is attached and fixed to the rear portion 1A of the vehicle 1 so that the optical axis O of the wide-angle lens system faces in the direction in which the reference subject 3 that the customer desires to photograph is present. The mounting position of the vehicle-mounted camera 11 on the rear portion 1A is in principle free, but it is desirable to mount it according to the instructions. The viewpoint conversion processing unit 12 is set in an appropriate place inside the vehicle 1, for example, in an instrument panel in front of the driver's seat. Moreover, it is desirable to place the parameter input device 13 at an appropriate location on the instrument panel in front of the driver seat, for example. The monitor device 14 can also be used for a navigation system.

As shown in FIG. 8, the viewpoint conversion processing unit 12 includes a parameter storage unit 12A, a calculation block unit 12B that is a calculation unit that performs mapping data calculation and the like, and a viewpoint conversion process that is a viewpoint conversion processing circuit that performs overhead conversion processing and the like. And a block portion 12C. Before describing the detailed configuration of the viewpoint conversion processing unit 12, a general principle for performing viewpoint conversion processing on raw image data captured by the in-vehicle camera 11 will be described with reference to FIG.

FIG. 9 is a schematic diagram showing a state in which the vehicle-mounted camera 11 is attached and fixed to the rear portion 1A of the vehicle 1. When the intersection of the optical axis O of the vehicle-mounted camera 11 and a CCD (solid-state imaging device) (not shown) of the vehicle-mounted camera 11 is the imaging center, the z-coordinate of the imaging center of the CCD is the coordinate position in the height direction. The x coordinate of the imaging center is the coordinate position in the vehicle longitudinal direction, and the y coordinate of the CCD imaging center is the coordinate position in the vehicle width direction. The coordinates of the CCD image pickup center are the viewpoint of the vehicle-mounted camera 11. That is, the coordinates in the three-dimensional space of the CCD imaging center are the three-dimensional coordinate position (x, y, z) as the viewpoint of the in-vehicle camera 11.
When the general principle of the viewpoint conversion process is described with reference to FIG. 9, the left and right rear wheels of the vehicle 1 are denoted by Lrh and Rrh, and the rear wheels are denoted by Lrh and Rrh rear shafts by Lrs and Rrs. The rear shafts Lrs and Rrs have a coaxial center line extending in the vehicle width direction as a rear shaft center line (rear shaft axis line) Rax. The center between the rear wheels Lrh and Rrh of the rear shaft center line Rax is defined as a rear wheel center Rsc, a line vertically lowered from the rear wheel center Rsc is defined as a virtual vertical line ScL, and a virtual vertical line ScL. And the horizontal ground EG is defined as a coordinate origin Op of spatial coordinates on the ground EG. A horizontal line passing through the rear wheel center Rsc and extending in the front-rear direction is HL, and an optical axis of the vehicle-mounted camera 11 is O.
Further, for example, an installation posture in which the direction of the optical axis O of the vehicle-mounted camera 11 is downward in the vertical direction is set as the reference posture of the camera 1. Moreover, the in-vehicle camera 11 is attached to the rear portion 1A of the vehicle 1 so that the optical axis O is directed in the vertical direction and the optical axis O is perpendicular to the horizontal line HL. The intersection between the optical axis O and the ground EG is the coordinate origin Or (x = 0, y = 0, z = 0).
Such conditions are those when the vehicle is placed on a horizontal ground EG in a state where the vehicle is not traveling.

Further, the attitude of the in-vehicle camera 11 with respect to the three-dimensional space when the vehicle 1 is traveling, for example, is the tilt angle (pitch angle θ) of the optical axis O with respect to the horizontal plane in the vertical plane, and the rotation angle (roll angle) around the photographing optical axis. ψ), which is determined by the tilt angle (yaw angle φ) of the optical axis O with respect to the vertical plane in the horizontal plane. Therefore, the position and orientation of the in-vehicle camera 2 with respect to the three-dimensional space can be expressed using the six parameters (x, y, z, θ, ψ, φ) in the three-dimensional space.

Therefore, when the vehicle 1 is not traveling, the basic posture is when the optical axis O of the in-vehicle camera 11 is directed vertically downward, and the pitch angle θ and the yaw angle φ about three mutually independent axes in the three-dimensional space. The attitude of the in-vehicle camera 11 with respect to the three-dimensional space is uniquely defined by the roll angle ψ.

It should be noted that the vehicle camera 11 attached to the vehicle 1 in any posture uses six parameters (x, y, z, θ, ψ, φ) for the position and posture of the vehicle camera 11. Can be expressed. Therefore, an arbitrary point of the raw image acquired by the in-vehicle camera 11 can be theoretically defined using these six parameters (x, y, z, θ, ψ, φ).

Therefore, for example, when the vehicle-mounted camera 11 is attached to the rear portion 1 </ b> A of the vehicle 1, the upper side portion 20 of the vehicle body bumper 19 can be imaged by the vehicle-mounted camera 11. When the upper side 20 of the vehicle body bumper 19 captured by the vehicle-mounted camera 11 is displayed on the image display unit (display screen) 18 of the monitor device 14, the upper side 20 is displayed on the lower part of the image display unit 18. To be. In order to satisfy such conditions, the six parameters (x, y, z, θ, and so on) obtained when the in-vehicle camera 11 is precisely attached and fixed to the rear portion 1A of the vehicle 1 are fixed. Let (ψ, φ) be the initial parameters (default values).

For example, the initial parameters are (x = 0, y = 0, z = 0, θ = 45 degrees, ψ = 0, φ = 0). In the following description, since it becomes complicated to include the coordinate position (x, y, z) of the vehicle-mounted camera 11 with respect to the three-dimensional space, the description thereof is omitted here.

Even when the orientation of the in-vehicle camera 11 is changed by rotating the initial orientation of the in-vehicle camera 11 by (θ ₀ , ψ ₀ , φ ₀ ), the in-vehicle camera 11 is further rotated thereafter. Even when the attitude is changed to a different attitude (θ ′, ψ ′, φ ′), the new attitude of the in-vehicle camera 11 can be represented by a rotation matrix described later. Therefore, an arbitrary point in a raw image acquired by a certain posture of the vehicle-mounted camera 11 can be expressed by being converted to another viewpoint by mathematical calculation.

Any one point in the raw image can be mathematically calculated using the rotation matrix R, assuming initial parameters (θ ₀ , ψ ₀ , φ ₀ ). That is, a coordinate point coordinate point [Xout] indicating an arbitrary point in the image is
[Xout] = R (φ ₀ ) R (θ ₀ ) R (ψ ₀ ) [Xin] R (φ) R (θ) R (ψ)
= RD (1)
It becomes.
Here, [Xin] is a coordinate point indicating an arbitrary point in the image focused as a rotation conversion target. [Xout] is an arbitrary value in the image after rotation conversion by the rotation matrix R (φ ₀ ), R (θ ₀ ), R (ψ ₀ ) using the initial parameters (θ ₀ , ψ ₀ , φ ₀ ). This is a coordinate point indicating one point.

By calculating the rotation matrix R (φ ₀ ), R (θ ₀ ), R (ψ ₀ ) from left to right with respect to [Xin], calculation is performed on a coordinate point indicating an arbitrary point in the image. Is executed.

When the rotation conversion is further performed on the coordinate point indicating an arbitrary point in the image after the rotation conversion obtained by the rotation processing, the rotation conversion processing described below is further performed.

For example, when rotating by a roll angle Δψ around the optical axis O of the in-vehicle camera 11 with respect to a coordinate point indicating an arbitrary point in the image after rotation conversion,
[Xa] = R (Δψ) [Xout]
= R (Δψ) R (φ ₀ ) R (θ ₀ ) R (ψ ₀ ) [Xin] R (Δψ) R (φ) R (θ) R (ψ)
= RDa (2)
Using this equation, it is possible to obtain a coordinate point indicating an arbitrary point in the image after the rotation conversion by the roll angle Δψ.

Here, since specific values are given to (θ ₀ , ψ ₀ , φ ₀ , Δψ), a 3 × 3 rotation matrix is used in the image after rotation conversion without using variables. A coordinate point indicating an arbitrary point can be represented. For this rotation matrix, for example, Equations 1 to 6 in JP-A-2007-256030 are used.

In general, the attitude of the vehicle-mounted camera 11 with respect to the three-dimensional coordinate space can be expressed using arbitrary θ, ψ, and φ. Therefore, when RDa = RD, the virtual posture of the in-vehicle camera 11 after the viewpoint conversion can be expressed by θ, ψ, and φ.

That is, mapping data for generating viewpoint conversion video is generated for each coordinate point using the rotation matrix, and the viewpoint conversion image data is obtained by subjecting the raw image data acquired by the in-vehicle camera 11 to the viewpoint conversion processing using the mapping data. If output is made to the monitor device 14, the viewpoint conversion image can be displayed on the image display unit 18 of the monitor device 14. Such viewpoint conversion processing is known.

Referring back to FIG. 8, the function of each component of the parameter input device 13 and the viewpoint conversion processing unit 12 will be described below.

The parameter input device 13 has a function of inputting parameters used for generating the three-dimensional state data of the in-vehicle camera 11 to the viewpoint conversion processing unit 12. This three-dimensional state data is mapping data for generating viewpoint-converted video. The parameters used for generating the mapping data are data (position of coordinate position) and attitude data (inclination direction and inclination of the optical axis O) of the in-vehicle camera 11 with respect to the three-dimensional space as data used to generate the viewpoint conversion video. Angle).

The parameter output from the parameter input device 13 or the parameter stored in the parameter storage unit 12A is input to the calculation block unit 12B. The arithmetic block unit 12B generates mapping data in real time based on parameters input by the parameter input device 13 (hereinafter also referred to as input parameters).

The raw image data output from the vehicle-mounted camera 11 and the mapping data generated by the calculation block unit 12B are input to the viewpoint conversion processing block unit 12C. In addition, the viewpoint conversion processing block unit 12C performs a viewpoint conversion process in real time based on the input raw image data and mapping data, and outputs the viewpoint converted image data to the monitor device 14.

The parameter storage unit 12A has a role of storing input parameters input by the parameter input device 13. This parameter storage unit 12A stores preset initial parameters. In addition, the initial parameter is replaced with the input parameter input by the parameter input device 13 by processing described later.

For the parameter input device 13, for example, a remote control device shown in FIG. 10 is used. The parameter input device 13, which is a remote control device, includes levers 13a to 13e, one dial 13f, and display units 13g to 13i.
The levers 13a to 13c are used as an operation unit for designating three parameters Δx, Δy, and Δz corresponding to the three-dimensional coordinate position of the viewpoint of the in-vehicle camera 11. The

levers

13d and 13e are used as an operation unit for designating the pitch angle Δθ and the yaw angle Δφ as parameters corresponding to the posture of the vehicle-mounted camera 11 with respect to the three-dimensional space. The dial 13f is used as an operation unit for designating the roll angle Δψ.
Further, when the image displayed on the image display unit 18 of the monitor device 14 is moved and operated by operating the levers 13a to 13c, the images are displayed on the display units 13g to 13i on the image display unit 18 of the monitor device 14. Used to indicate whether to translate in cm. Reference numeral 13j denotes a power button.

By such operation of the levers 13a to 13c, the raw image data is subjected to a parallel movement process in the front / rear / left / right height direction. Then, when the translation processing of the raw image data is performed, mapping data for generating a viewpoint-converted image whose viewpoint is converted by the translation is generated. Further, rotation conversion processing is performed on the raw image data by operating the

levers

13d and 13e and the dial 13f. Then, when this rotation conversion process is performed, mapping data for generating a viewpoint conversion image whose viewpoint is converted by this rotation process is generated.

Next, referring to the image shown in FIG. 2, the conceptual diagram shown in FIG. 7, the block diagram shown in FIG. 8, the parameter input device shown in FIG. 10, the flowchart shown in FIG. 11, and the images shown in FIGS. A method of acquiring a viewpoint conversion image according to an embodiment of the invention will be described.

First, the customer purchases the in-vehicle camera 11, the viewpoint conversion processing unit 12, and the parameter input device 13, so that the optical axis O of the in-vehicle camera 11 is directed in an appropriate direction toward the rear portion 1A of the vehicle 1 shown in FIG. In the following description, the vehicle-mounted camera 11 is attached and fixed.

For the first time, when setting the viewpoint conversion image, initial parameters are stored in the parameter storage unit 12A. This initial parameter is a value appropriately determined before shipment at the factory. For example, this initial parameter is (x = 0, y = 0, z = 0, θ = 45 degrees, ψ = 0, φ = 0). is there.

When the power button 13j of the parameter input device 13, the power button (not shown) of the in-vehicle camera 11, and the power button (not shown) of the monitor device 14 are turned on, the initial parameters stored in the parameter storage unit 12A are Reading is performed as shown in FIG. 11 (S.1).
The calculation block unit 12B calculates mapping data for generating a viewpoint converted image in real time based on the initial parameters. This mapping data is input to the viewpoint conversion processing block unit 12c. The in-vehicle camera 11 outputs, for example, the raw image data illustrated in FIG. 2 to the viewpoint conversion processing block unit 12c. The viewpoint conversion processing block unit 12c performs viewpoint conversion processing on the image data based on the mapping data obtained based on the initial parameters to create viewpoint converted image data (default overhead view) (S.2).
The created default overhead view is output to the monitor device 14 as viewpoint converted image data (S.3). As a result, the viewpoint conversion image 21 shown in FIG. 12 is displayed on the image display unit 18 of the monitor device 14.

This S.I. 1-S. In the process 3, the image data acquired in real time by the in-vehicle camera 11 is subjected to viewpoint conversion processing based on the initial parameters, and the viewpoint conversion image 21 obtained by this viewpoint conversion processing is displayed on the image display unit 18 of the monitor device 14. This is the first display step.
Here, the initial parameters are parameters for generating mapping data for generating viewpoint-converted video, and data (position of the three-dimensional coordinate position) and attitude (optical axis O) of the vehicle-mounted camera 11 with respect to the three-dimensional space. Data on the inclination direction and inclination angle of the image.

The customer visually recognizes the viewpoint conversion image 21 displayed on the image display unit 18 of the monitor device 14 (S.4). Next, the customer determines whether or not to change the viewpoint conversion image 21 (S.5). If no change is necessary, the process ends. In the normal case, there is almost no need for modification. Move to 6.

S. 6, parameters are input by the parameter input device 13. This S.I. Process 6 is an input step for inputting the parameters for generating the mapping data described above in order to change the viewpoint conversion image 21 obtained by the initial parameters.

The calculation block unit 12B is S.D. Based on the input parameters input in step 6, the mapping data for generating the viewpoint conversion image is calculated in real time using the rotation matrix (S.7). This mapping data is input to the viewpoint conversion processing block unit 12c. The viewpoint conversion processing block unit 12c performs viewpoint conversion processing on the raw image data based on the mapping data to create viewpoint converted image data (overhead view) (S.8). The created overhead view is output to the monitor device 14 as video data (S.9). As a result, the viewpoint conversion image 22 shown in FIG. 13 is displayed on the image display unit 18 of the monitor device 14. The viewpoint conversion image 22 is an image obtained by further adding viewpoint conversion using input parameters to the viewpoint conversion image 21 obtained using initial parameters. Here, the input parameters are the pitch angle Δθ and the roll angle Δψ.

This S. 7-S. In the processing up to 9, the image data acquired in real time by the in-vehicle camera 11 is converted to S.P. This is a second display step for performing viewpoint conversion processing based on the input parameters input in step 6 and displaying the image obtained by the viewpoint conversion processing on the image display unit 18 of the monitor device 14 as the viewpoint conversion image 22. .

The customer visually recognizes the viewpoint conversion image 22 displayed on the image display unit 18 of the monitor device 14 (S.10), and determines whether the viewpoint conversion image 22 is appropriate (S.11).

If it is determined that this viewpoint conversion image 22 is not appropriate, S.I. Returning to step 6, S. 6-S. 10 processes are performed. It is assumed that, for example, the viewpoint conversion image 23 illustrated in FIG. 14 is obtained by the second viewpoint conversion image acquisition process.

When it is determined that the viewpoint conversion image 23 is appropriate, the power button 13j is turned off. Then, the initial parameter is replaced with the input parameter input by the parameter input device 13. That is, the parameter storage unit 12A stores the input parameter input by the parameter input device 13 (S.12).
This input parameter saving process is executed by the calculation block unit 12B. In this case, the input parameters stored in the parameter storage unit 12A are already stored in the parameter storage unit 12A and parameters (Δx, Δy, Δz, Δθ, Δψ, Δφ) given by the customer's operation. It is determined by the difference from the initial parameters (x, y, z, θ, ψ, φ).

That is, when the power button 13j is turned off, an end command is transmitted from the remote control device to the calculation block unit 12B. The calculation block unit 12B is configured to input parameters (Δx, Δy, Δz, Δθ, Δψ, Δφ based on the inverse calculation of the rotation matrix. ) And the initial parameters (x, y, z, θ, ψ, φ) already stored in the parameter storage unit 12A.

The calculation block unit 12B transmits the difference parameters (x ′, y ′, z ′, θ ′, ψ ′, φ ′) to the parameter storage unit 12A. Thereby, the difference parameters (x ′, y ′, z ′, θ ′, ψ ′, φ ′) are stored in the parameter storage unit 12A, and the difference parameters (x ′, y ′, z ′, θ ′, ψ) are stored. ', Φ') is used as an initial parameter (x, y, z, θ, ψ, φ) in the next viewpoint conversion image acquisition.

In addition, S. The process No. 11 is a determination step for determining whether or not the viewpoint conversion image displayed on the image display unit 18 of the monitor device 14 in the second display step (the processes from S.7 to S.9) is appropriate. Moreover, this S.I. In the process No. 11, when it is determined that the viewpoint conversion image is appropriate, the input parameter is stored in the parameter storage unit 12A. On the other hand, this S.I. In the process of No. 11, when it is determined that the viewpoint conversion image is inappropriate, the input step can be re-executed.

In such a viewpoint conversion image operation, the viewpoint of the in-vehicle camera 11 is translated by Δx, Δy, Δz with respect to the three-dimensional space by operating the levers 13a to 13c once, and the

levers

13d and 13e are operated once. The viewpoint is rotated by Δθ, Δψ, and Δφ by the operation and the operation of the dial 13f. By such an operation, the viewpoint of the in-vehicle camera 11 can be changed by a predetermined amount with reference to the initial parameters (x, y, z, θ, ψ, φ). By this operation, the customer can perform viewpoint conversion by a predetermined amount while viewing the viewpoint conversion image in real time, so that handling becomes even easier.

[Example 2]
In the second embodiment, as shown in FIG. 15, the in-vehicle viewpoint conversion video system includes a plurality of input parameter storage units 12D to 12G that store and save input parameters input by the parameter input device 13. In the case of the second embodiment, the parameter storage unit 12A includes a ROM as an initial parameter storage unit, and the initial parameters cannot be rewritten. In the input parameter storage units 12D to 12G, input parameters are stored and saved by operations described later.

Here, the parameter input device 13 is the monitor device 14 shown in FIG. 15, and the monitor device 14 has a touch panel image display unit 18.
The touch-panel image display unit (display screen) 18 includes a three-dimensional position adjustment screen (that is, a three-dimensional coordinate position adjustment screen) 18A shown in FIG. 16 and an attitude adjustment screen 18B shown in FIGS. Switching display is possible.
On this three-dimensional position adjustment screen 18A, as shown in FIG. 16, instruction buttons 18a to 18c, a return button 24, and an enter button 25 are displayed.
The instruction buttons 18a to 18c are used to designate three parameters corresponding to the three-dimensional coordinate position (X, Y, Z) of the viewpoint of the vehicle-mounted camera 11. That is, the instruction buttons 18a to 18c have a pair of an upward arrow image directed upward and a downward arrow image directed downward, and the instruction button 18a is used to indicate the coordinate position in the X direction. The instruction button 18b is used to instruct the coordinate position in the Y direction, and the instruction button 18c is used to instruct the coordinate position in the Z direction. Moreover, the return button 24 is used to return to the previous screen, and the decision button 25 is used to complete the adjustment of the three-dimensional coordinate position and move to the next screen.
In addition, as shown in FIGS. 17 and 18, instruction buttons 18d (18d ′), 18e (18e ′), 18f (18f ′), a return button 26, and a determination button 27 are displayed on the posture adjustment screen 18B. The
The instruction buttons 18d (18d ′), 18e (18e ′), and 18f (18f ′) are displayed on the three-dimensional position adjustment screen 18A of the monitor device 14 by the operation of the determination button 25, and the tertiary of the vehicle-mounted camera 11 is displayed. It is used to designate three parameters as a pitch angle Δθ, a roll angle Δψ, and a yaw angle Δφ as postures with respect to the original space.
That is, the instruction button 18d is displayed as an up arrow image at the upper edge of the screen, and the instruction button 18d 'is displayed as a down arrow image at the lower edge of the screen. The instruction button 18e is displayed as a right turn arrow image at the upper left corner of the screen, and the instruction button 18e 'is displayed as a left turn arrow image at the upper right corner of the screen. Further, the instruction button 18f is displayed as a left arrow image at the center in the vertical direction at the left edge of the screen, and the instruction button 18f 'is displayed as a right arrow image at the center in the vertical direction at the right edge of the screen. The

instruction buttons

18d and 18d ′ are used to designate a parameter of the pitch angle Δθ as the posture of the vehicle-mounted camera 11 with respect to the three-dimensional space. The

instruction buttons

18e and 18e ′ are used to designate a parameter of the roll angle Δψ as the posture of the vehicle-mounted camera 11 with respect to the three-dimensional space. Further, the

instruction buttons

18f and 18f ′ are used for designating three parameters of the yaw angle Δφ as the attitude of the vehicle-mounted camera 11 with respect to the three-dimensional space.
A return button 26 is used to return to the previous screen. The decision button 27 is used to store parameters input after the posture adjustment for the three-dimensional space is completed in the input parameter storage units 12D to 12G in FIG.

In the three-dimensional position adjustment screen 18A of FIG. 16, display units 18g to 18i indicating the amount of parallel movement of the image on the screen are provided corresponding to the instruction buttons 18a to 18c. The customer adjusts the three-dimensional coordinate position of the in-vehicle camera 11 on the three-dimensional position adjustment screen 18A, and adjusts the attitude of the in-vehicle camera 11 with respect to the three-dimensional space on the attitude adjustment screen 18B, thereby mapping the viewpoint conversion generation. Generate data. Subsequently, the input parameter for obtaining the mapping data for generating the viewpoint transformation is stored in the input parameter storage unit 12D.

Further, for example, in the posture adjustment screen 18B shown in FIG. 17, when the instruction button 18d is pressed once, the pitch angle θ is rotated in the plus direction by Δθ, and the rotation operation corresponding to this Δθ is executed. In addition, the viewpoint conversion image shown in FIG. 18 is obtained.

The customer can acquire a plurality of input parameters for obtaining mapping data for viewpoint conversion according to his / her preference by performing the action corresponding to the flowchart of the first embodiment (see FIG. 11) a plurality of times.
For example, the input parameters corresponding to the raw image shown in FIG. 2, the input parameters corresponding to the image shown in FIG. 13, the input parameters corresponding to the image shown in FIG. Assume that the input parameters corresponding to the image obtained when the above image is taken are stored and saved in the input parameter storage units 12D to 12E.

A touch panel screen 18C as shown in FIG. 19 is displayed on the image display unit (display screen) 18 of the monitor device 14 of FIG. A plurality of viewpoint conversion images are displayed on the touch panel screen 18C when the monitor device 14 is turned on. The plurality of viewpoint conversion images are simultaneously displayed in parallel on the reduced images 31 to 34 obtained based on the input parameters stored in the plurality of input parameter storage units 12D to 12E when the monitor device 14 is turned on. Is done. Moreover, when any one of the reduced images 31 to 34 is touched, the viewpoint conversion image corresponding to the touched reduced image is enlarged and displayed on the image display unit 18 of the monitor device 14.

In the viewpoint conversion video system of the in-vehicle camera according to the embodiment of the invention described above, the calibration process of adjusting the mounting error of the in-vehicle camera 11 is abolished and the position of the in-vehicle camera (real camera) 11 in the three-dimensional space is eliminated. The (three-dimensional coordinate position) data and attitude data (such as the tilt direction and tilt angle of the optical axis O) are used as parameters for creating mapping data used to generate the viewpoint conversion image.
Moreover, in this in-vehicle camera viewpoint conversion video system, the viewpoint conversion processing is directly executed using the mapping data creation parameters. Thereby, in the viewpoint conversion video system of the vehicle-mounted camera according to the embodiment of the present invention, compared with the mapping data creation processing speed when mapping data for viewpoint conversion image generation is conventionally created using 12 parameters. Thus, the mapping data creation processing speed can be improved. As a result, in this system, the viewpoint converted image can be displayed on the screen of the monitor device in real time, and the customer can acquire the viewpoint converted image while viewing the image display unit 18 of the monitor device 14.

Further, since the viewpoint conversion image can be displayed on the monitor device 14 in real time, the parameters for mapping data generation can be adjusted while viewing the viewpoint conversion image, which is easy to handle for the customer and the customer can freely convert the viewpoint conversion image. Can be obtained.

Also, the range in which the customer can adjust the parameters may be limited. That is, the parameters may be adjustable within a range where the upper side portion 20 of the vehicle body bumper 19 appears in the lower portion of the image display unit 18 so that no blind spot is generated.

As described above, the in-vehicle camera viewpoint conversion video system according to the embodiment of the present invention includes the in-vehicle camera 11 attached and fixed to the vehicle, the monitor device 14 for displaying an image, and the in-vehicle camera 11. And a viewpoint conversion processing unit 12 that performs viewpoint conversion processing on the image data based on mapping data for generating a viewpoint converted image and outputs the viewpoint converted image data to the monitor device 14. The viewpoint conversion video system further includes a parameter input device 13 that inputs parameters for generating the mapping data, which are parameters indicating the position and orientation of the vehicle-mounted camera 11 with respect to the three-dimensional space. .

According to this configuration, since the parameter input device 13 for inputting parameters for the viewpoint conversion process is provided, the viewpoint conversion process can be automatically performed only by inputting the parameters, which is easy for the customer to handle. Become.

In addition, according to this configuration, since the viewpoint conversion image can be displayed on the monitor device 14, the parameter input device 13 inputs the parameter for generating the mapping data while viewing the viewpoint conversion image. The input can be easily adjusted, and it is easy for the customer to handle and the customer can freely acquire the viewpoint conversion image.

In addition, in the viewpoint conversion video system for the in-vehicle camera according to the embodiment of the present invention, the viewpoint conversion processing unit 12 calculates the mapping data in real time based on the input parameter input by the parameter input device 13. Unit (calculation block unit 12B), image data output from the in-vehicle camera 11 and mapping data output from the calculation unit (calculation block unit 12B) are input, and the image data is converted into the image data based on the mapping data. A viewpoint conversion processing circuit (viewpoint conversion processing block unit 12C) that performs viewpoint conversion processing in real time, and a parameter storage unit (parameter storage unit 12A, input parameter storage unit 12D˜) that stores input parameters input by the parameter input device 13 12G).
According to this configuration, since the viewpoint conversion image can be displayed on the monitor device 14 in real time, the parameter input device 13 inputs a parameter for generating mapping data while viewing the viewpoint conversion image. The input can be easily adjusted, and it is easy for the customer to handle and the customer can freely acquire the viewpoint conversion image. In addition, since the parameter storage unit that stores the input parameters input by the parameter input device 13 is provided, it is possible to easily obtain a viewpoint image according to customer preferences.

Furthermore, in the in-vehicle camera viewpoint conversion video system according to the embodiment of the present invention, preset parameter is stored in the parameter storage unit (parameter storage unit 12A), and the initial parameter is stored in the parameter input device 13. It is replaced with the input parameter input by.
According to this configuration, for example, when a vehicle having a viewpoint conversion image obtained by an initial parameter set in a factory or the like is delivered to a customer after shipment, the initial viewpoint conversion image is shifted or the customer's preference. Even if they do not match, it is possible to easily change the initial parameters and easily obtain a preferable viewpoint conversion image.

In the in-vehicle camera viewpoint conversion video system according to the embodiment of the present invention, the parameter input device 13 includes a remote control device, and the remote control device corresponds to the three-dimensional coordinate position of the viewpoint of the in-vehicle camera 11. An operation unit (lever 13a to 13c) for designating three parameters to be performed and an operation unit for designating three parameters of a pitch angle, a roll angle, and a yaw angle as the posture of the vehicle-mounted camera 11 with respect to the three-dimensional space ( A lever 13d for operating the pitch angle Δθ, a lever 13e for operating the yaw angle Δφ, and a dial 13f for operating the roll angle Δψ.

According to this configuration, since the parameter input device 13 is configured by the remote control device, there is an effect that remote operation is possible.

In the in-vehicle camera viewpoint conversion video system according to the embodiment of the present invention, the monitor device 14 is of a touch panel type. In the viewpoint conversion video system, the parameter storage unit stores an initial parameter storage unit (parameter storage unit 12A) that stores preset initial parameters and a plurality of input parameters that are input by the parameter input device 13. Input parameter storage unit (12D to 12G). In this viewpoint conversion video system, the viewpoint conversion processing unit 12 displays a viewpoint conversion image obtained based on the input parameters stored in the plurality of input parameter storage units (12D to 12G) as the touch panel type monitor device 14. Are simultaneously displayed in parallel as reduced images (31-34) on the screens (18A-18C). Moreover, when the viewpoint conversion processing unit 12 touches the reduced images (31 to 34), the reduced images (31 to 34) are enlarged and displayed on the screen of the monitor device 14.

According to the viewpoint conversion video system of the present invention, since the reduced image (thumbnail image) of the viewpoint conversion image is simultaneously displayed on the screen of the monitor device 14 and presented, the viewpoint conversion image is selected according to the preference of the customer. Can be displayed.
In the in-vehicle camera viewpoint conversion video system according to the embodiment of the present invention, the monitor device 14 is a touch panel type monitor device that also serves as the parameter input device, and the screen (18A, 18B) of the monitor device 14 includes A parameter input section (instruction buttons 18a to 18c, 18d, 18d 'to 18f, 18f') for inputting the parameters is displayed.
According to this configuration, it is not necessary to separately provide a parameter input device, so the configuration is simplified.
In the in-vehicle camera viewpoint conversion video system according to the embodiment of the present invention, the parameter input unit (instruction buttons 18a to 18c, 18d, 18d 'to 18f, 18f') inputs the three-dimensional position data of the in-vehicle camera. In order to do this, it is a movement direction instruction part displayed on the screen (18A, 18B).
According to this configuration, it is possible to easily change the viewpoint position of the monitor device 14 without moving the viewing position while viewing the image displayed on the screen (18A, 18B) of the monitor device 14. Is good.
In the in-vehicle camera viewpoint conversion video system according to the embodiment of the present invention, the movement direction instruction unit is an X movement direction instruction unit for inputting position data in the X direction, and a Y movement direction instruction for inputting position data in the Y direction. And Z movement direction indicating units (instruction buttons 18a to 18c, 18d, 18d 'to 18f, 18f') for inputting position data in the Z direction.
According to this configuration, the position data in the X direction, the Y direction, and the Z direction can be easily input individually, so that it becomes easy to finely input the position data in the three-dimensional direction.
In the viewpoint conversion video system for an in-vehicle camera according to the embodiment of the present invention, the X movement direction instruction section, the Y movement direction instruction section, and the Z movement direction instruction section are a pair of instruction buttons for moving the viewpoint position in the opposite direction. (18a-18c, 18d, 18d'-18f, 18f ').
According to this configuration, the operation of moving the viewpoint position in the three-dimensional direction can be easily performed.
In the in-vehicle camera viewpoint conversion video system according to the embodiment of the present invention, the pair of instruction buttons (instruction buttons 18a to 18c, 18d, 18d 'to 18f, and 18f') of the respective movement direction instruction units are in opposite directions. It is a directed arrow image.
According to this configuration, since the moving direction of the viewpoint position in the three-dimensional direction can be visually recognized, it is possible to easily operate the moving direction of the viewpoint position.
In the in-vehicle camera viewpoint conversion video system according to the embodiment of the present invention, three display units (18g to 18i) for respectively displaying the parallel movement amounts in the three-dimensional direction of the in-vehicle camera on the screen (18A, 18B). Is provided.
According to this configuration, the movement state of the viewpoint position and the amount of parallel movement thereof can be easily executed without moving the viewing position on the same screen.
In the viewpoint conversion video system for the in-vehicle camera according to the embodiment of the present invention, the instruction button is provided on the peripheral edge of the screen (18B).
According to this configuration, since the instruction button is provided on the peripheral portion of the screen (18B), the viewpoint image on the screen can be easily recognized.

In the viewpoint conversion video system for the in-vehicle camera according to the embodiment of the present invention, the screen of the touch panel type monitor device 14 has a three-dimensional coordinate position adjustment screen 18A and a posture adjustment screen 18B. Moreover, the viewpoint conversion processing unit 12 includes an instruction button (18a to 18c) for designating three parameters corresponding to the three-dimensional coordinate position of the viewpoint of the vehicle-mounted camera 11, a return button 24 for returning to the previous screen, and A determination button 25 that signifies completion of adjustment of the three-dimensional coordinate position and shifts to the next screen is displayed on the three-dimensional coordinate position adjustment screen 18A. Further, the viewpoint conversion processing unit 12 is displayed on the monitor device 14 by the operation of the decision button 25 and has three pitch angles, a roll angle, and a yaw angle as postures of the vehicle-mounted camera 11 with respect to the three-dimensional space. An instruction button (18d, 18d ′, 18e, 18e ′, 18f, 18f ′) for designating a parameter, a return button 26 for returning to the previous screen, and completion of posture adjustment with respect to the three-dimensional space. A determination button 27 for storing the input parameters in the input parameter storage unit (12D to 12G) is displayed on the posture adjustment screen 18B.

According to this configuration, since the touch panel type monitor device 14 is used as the parameter input device 13, the number of parts can be reduced.

Moreover, in the viewpoint conversion video acquisition method of the vehicle-mounted camera according to the embodiment of the present invention, the raw image data from the vehicle-mounted camera 11 attached and fixed to the vehicle is input and the viewpoint conversion is performed to convert the viewpoint. A viewpoint conversion processing unit 12 that outputs image data to the monitor device 14 is used, and an image captured by the vehicle-mounted camera 11 is converted into a screen of the monitor device 14 by a viewpoint conversion process to display a viewpoint conversion image. It has become. Moreover, in the viewpoint conversion video acquisition method, the vehicle-mounted camera 11 means the position and orientation of the vehicle-mounted camera 11 with respect to the three-dimensional space and is generated by the vehicle-mounted camera 11 in real time based on initial parameters that generate mapping data for generating a viewpoint converted image. A first display step of performing a viewpoint conversion process on the obtained raw image data to display a viewpoint conversion image on the screen of the monitor device; Further, in the viewpoint conversion video acquisition method, it means the position and orientation of the in-vehicle camera 11 with respect to the three-dimensional space while visually recognizing the viewpoint conversion image in order to change the viewpoint conversion image obtained by the initial parameter and An input step for inputting an input parameter for generating the mapping data; In the viewpoint converted video acquisition method, the viewpoint conversion image is displayed on the screen of the monitor device 14 by performing viewpoint conversion processing on the image data acquired by the vehicle-mounted camera 11 in real time based on the input parameters input in the input step. There is a second display step for displaying. Further, in the viewpoint converted video acquisition method, it is determined whether or not the viewpoint converted image displayed on the screen of the monitor device 14 is appropriate in the second display step, and when the viewpoint converted image is appropriate, the parameter is stored as a parameter. And a determination step that allows the input step to be re-executed when the viewpoint conversion image is inappropriate.

According to this viewpoint conversion video acquisition method of the in-vehicle camera, since the viewpoint conversion image can be displayed on the monitor device 14 in real time, the mapping data generation parameters can be adjusted while viewing the viewpoint conversion image. It is easy for the customer to handle and the customer can acquire the viewpoint conversion image freely.

In addition, in the viewpoint conversion video acquisition method for the in-vehicle camera according to the embodiment of the present invention, the viewpoint conversion processing unit 12 calculates the mapping data in real time based on the input parameters (calculation block unit). 12B), the image data output from the in-vehicle camera 11 and the mapping data output from the calculation unit are input, and the viewpoint conversion processing is performed on the image data in real time based on the mapping data for generating the viewpoint conversion image. Includes a viewpoint conversion processing circuit (viewpoint conversion processing block unit 12C).

(Claiming priority)
This application claims the priority based on Japanese Patent Application No. 2008-183201 for which it applied to Japan Patent Office on July 14, 2008, and uses the content here.

Claims

An in-vehicle camera attached and fixed to the vehicle;
A monitor device for displaying an image;
A viewpoint conversion processing unit that receives raw image data from the in-vehicle camera and performs viewpoint conversion processing on the image data based on mapping data for generating a viewpoint-converted image, and outputs viewpoint-converted image data to a monitor device;
A viewpoint conversion video system for an in-vehicle camera, comprising a parameter input device for inputting a parameter for generating the mapping data, which means a position and an attitude of the in-vehicle camera in a three-dimensional space.
The viewpoint conversion processing unit is configured to calculate the mapping data in real time based on the input parameters input by the parameter input device, the image data output from the vehicle-mounted camera, and the calculation unit. A viewpoint conversion processing circuit that performs real-time viewpoint conversion processing on the image data based on the mapping data, and a parameter storage unit that stores input parameters input by the parameter input device. The viewpoint conversion video system for a vehicle-mounted camera according to claim 1, wherein:
The viewpoint of the in-vehicle camera according to claim 2, wherein an initial parameter set in advance is stored in the parameter storage unit, and the initial parameter is replaced with an input parameter input by the parameter input device. Conversion video system.
The parameter input device includes a remote control device, and the remote control device is an operation unit for designating three parameters corresponding to the three-dimensional coordinate position of the viewpoint of the vehicle-mounted camera, and the attitude of the vehicle-mounted camera with respect to the three-dimensional space. The in-vehicle camera viewpoint conversion video system according to claim 2, further comprising: an operation unit that specifies three parameters of a pitch angle, a roll angle, and a yaw angle.
The monitor device is a touch panel type monitor device, and the parameter storage unit stores an initial parameter storage unit that stores preset initial parameters and a plurality of input parameter storages that store input parameters input by the parameter input device. And the viewpoint conversion processing unit displays a viewpoint conversion image obtained based on the input parameters stored in the plurality of input parameter storage units simultaneously on the screen of the touch panel type monitor device as a reduced image in parallel. The viewpoint conversion video system for an in-vehicle camera according to claim 2, wherein the viewpoint conversion processing unit enlarges and displays the reduced image on the screen of the monitor device when the reduced image is touched.
The in-vehicle apparatus according to claim 2, wherein the monitor device is a touch panel type monitor device that also serves as the parameter input device, and a parameter input unit that inputs the parameter is displayed on a screen of the monitor device. Camera viewpoint conversion video system.
The in-vehicle camera viewpoint conversion video system according to claim 6, wherein the parameter input unit is a movement direction instruction unit displayed on the screen for inputting three-dimensional position data of the in-vehicle camera. .
The movement direction instruction unit is an X movement direction instruction unit that inputs position data in the X direction, a Y movement direction instruction unit that inputs position data in the Y direction, and a Z movement direction instruction unit that inputs position data in the Z direction. The viewpoint conversion video system for an in-vehicle camera according to claim 7.
The in-vehicle camera according to claim 8, wherein the X movement direction instruction unit, the Y movement direction instruction unit, and the Z movement direction instruction unit each include a pair of instruction buttons for moving the viewpoint position in the opposite direction. Viewpoint conversion video system.
10. The in-vehicle camera viewpoint conversion video system according to claim 9, wherein the pair of instruction buttons of each of the movement direction instruction units is an arrow image directed in opposite directions.
8. The in-vehicle camera viewpoint conversion video system according to claim 7, wherein the screen includes three display units for displaying parallel movement amounts in the three-dimensional direction of the in-vehicle camera.
10. The in-vehicle camera viewpoint conversion video system according to claim 9, wherein the instruction button is provided at a peripheral portion of the screen.
The touch panel type monitor device has a three-dimensional coordinate position adjustment screen and a posture adjustment screen, and the viewpoint conversion processing unit is an instruction button for designating three parameters corresponding to the three-dimensional coordinate position of the viewpoint of the vehicle-mounted camera. And a return button for returning to the previous screen and a determination button for moving to the next screen, which means completion of adjustment of the 3D coordinate position, and display on the 3D coordinate position adjustment screen, An instruction button for designating three parameters of pitch angle, roll angle, and yaw angle as an attitude with respect to the three-dimensional space of the vehicle-mounted camera by operating the determination button and for returning to the previous screen Means completion of posture adjustment with respect to the back button and the three-dimensional space, and the inputted input parameters are stored in the input parameter storage unit. Viewpoint conversion image system of the in-vehicle camera according to claim 4, characterized in that to be displayed on the position adjustment screen the decision button for 憶.
Using a viewpoint conversion processing unit that receives raw image data from an in-vehicle camera attached and fixed to a vehicle and applies viewpoint conversion processing to output image data that has undergone viewpoint conversion to a monitor device. The viewpoint conversion processing is performed on the raw image data acquired by the vehicle-mounted camera in real time based on the initial parameters that mean the position and orientation of the camera with respect to the three-dimensional space and generate mapping data for generating the viewpoint conversion image. A first display step for displaying a viewpoint conversion image on the screen of the monitor device; and a method for viewing the viewpoint conversion image to change the viewpoint conversion image obtained by the initial parameter while viewing the viewpoint conversion image with respect to the three-dimensional space of the vehicle-mounted camera Input the input parameters that mean the position and orientation and for generating the mapping data And a second display step of displaying a viewpoint conversion image on the screen of the monitor device by performing a viewpoint conversion process on the image data acquired by the vehicle-mounted camera in real time based on the input parameter input in the input step And determining whether or not the viewpoint converted image displayed on the screen of the monitor device in the second display step is appropriate, and when determining that the viewpoint converted image is appropriate, storing the input parameters in the parameter storage unit, A method for acquiring a viewpoint-converted video of a vehicle-mounted camera, comprising: a determination step that allows the input step to be re-executed when it is determined that the viewpoint-converted image is inappropriate.
The viewpoint conversion processing unit receives a calculation unit that calculates the mapping data in real time based on the input parameters, image data output from the vehicle-mounted camera, and mapping data output from the calculation unit And a viewpoint conversion processing circuit for performing viewpoint conversion processing on the image data in real time based on the mapping data for generating the viewpoint-converted image. Video acquisition method.