JP2017112566A - Parameter identification device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology capable of improving continuity of an image to be displayed on a display device regardless of a loading state of a vehicle.SOLUTION: The present invention relates to a parameter identification device configured to identify camera parameters of multiple cameras that are loaded on a vehicle. The parameter identification device comprises an image acquisition part S110, an image conversion part S120, a target detection part S130, parameter identification parts S170-S250 and S280 and image update parts S260, S270 and S290. The target detection part detects a detection target that is a target captured in each of viewpoint conversion images, from each of viewpoint conversion images. In the case where the detection targets which are detected by the target detection part and based on captured images obtained by imaging specific regions neighboring to each other are discontinuous from each other, the parameter identification part identifies camera parameters of cameras generating the viewpoint conversion images in which the detection targets become continuous to each other, as calibration parameters.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a parameter specifying device that specifies camera parameters of a camera.

As described in Patent Document 1, a vehicle-mounted system that performs bird's-eye conversion on an image captured by a vehicle-mounted camera and displays the bird's-eye image on a display device is known.
In this type of in-vehicle system, a plurality of in-vehicle cameras that capture different imaging regions, an image processing device that performs image processing on images captured by each of the in-vehicle cameras, and image processing are executed by the image processing device. There has been proposed a display device that includes a display device for displaying an image. In the image processing apparatus, as image processing, a top view that performs bird's-eye conversion on images captured by each of the on-vehicle cameras and displays the bird's-eye converted images in a superimposed manner so that the same point in the real space matches. Processing is being performed.

JP 2014-82622 A

In the image processing apparatus, when performing bird's-eye conversion of an image captured by an in-vehicle camera, camera parameters indicating the mounting position and mounting orientation of the in-vehicle camera are necessary.
The camera parameters are usually specified at the stage when the in-vehicle camera is attached to the vehicle, for example, after the in-vehicle camera is attached to the vehicle at the vehicle manufacturing factory, and stored in the storage unit of the image processing apparatus.

  By the way, when a vehicle equipped with an in-vehicle camera is used, the number of persons riding on the vehicle and the weight of luggage loaded on the vehicle often differ every time the vehicle is used. Thus, when the loading state of the vehicle changes, there is a high possibility that the actual mounting posture of the in-vehicle camera at that time is different from the mounting posture stored in the storage unit.

  Then, when top view processing is performed using camera parameters that have a deviation from the actual mounting orientation, the image displayed on the display device is inconsistent at the same point in the real space reflected in the image Occurs.

That is, the conventional technique has a problem that the continuity of images displayed on the display device is likely to be impaired depending on the loading state of the vehicle.
Therefore, an object of the present invention is to provide a technique capable of improving the continuity of images displayed on a display device regardless of the loading state of a vehicle.

One aspect of the present invention made to achieve the above object relates to a parameter specifying device (20) for specifying camera parameters of a plurality of cameras (10) mounted on a vehicle (5).
Each of the plurality of cameras is mounted on the vehicle so as to capture an image of the specified area as an imaging area. The prescribed areas referred to here are different areas defined around the vehicle. Furthermore, the prescribed area is an area defined such that areas adjacent to each other are continuous in real space.

  The camera parameter is a parameter representing the position and orientation of each of the plurality of cameras. The storage device (22) stores reference parameters. The reference parameter is a camera parameter obtained when each of the plurality of cameras is mounted on the vehicle.

  The parameter identification device includes an image acquisition unit (20, S110), an image conversion unit (20, S120), a target detection unit (20, S130), a parameter identification unit (20, S170 to S250, S280), an image And an update unit (20, S260, S270, S290).

  The image acquisition unit acquires a captured image captured by each camera from each of the plurality of cameras. Then, the image conversion unit generates a viewpoint conversion image that is an image obtained by performing viewpoint conversion on each of the captured images acquired by the image acquisition unit, based on the reference parameters stored in the storage device.

  Further, the object detection unit detects a detection object that is an object reflected in each viewpoint conversion image from each of the viewpoint conversion images converted by the image conversion unit. The object referred to here is an object having continuity existing on the road surface.

  The parameter specifying unit specifies the calibration parameter when the detection target detected by the target detection unit is discontinuous and the detection targets based on the captured images obtained by capturing the prescribed regions adjacent to each other are discontinuous. The calibration parameter referred to here is each camera parameter of a camera that generates a viewpoint conversion image in which the detection objects are continuous.

The image updating unit generates a new viewpoint conversion image based on the calibration parameter specified by the parameter specifying unit.
With such a parameter specifying device, the calibration parameters of each camera can be specified.

  This calibration parameter is a camera parameter in which detection objects are continuous in a viewpoint conversion image obtained by performing viewpoint conversion on a captured image based on the calibration parameter, and the position and orientation of the camera reflecting the posture of the vehicle at that time It is.

That is, according to the parameter specifying device, when the vehicle travels, calibration parameters reflecting the posture of the vehicle can be sequentially specified.
As a result, according to the parameter identification device, it is possible to reduce the discontinuity of the detection target in the newly generated viewpoint conversion image, that is, the mismatch of the same point in the real space.

As described above, according to the parameter specifying device, the continuity of images displayed on the display device can be improved regardless of the loading state of the vehicle.
Further, in the parameter identification device, each camera parameter of a camera that generates a viewpoint-converted image in which detection objects are continuous is specified as a calibration parameter. Therefore, the calibration parameter for all cameras that generate viewpoint-converted images. Can be identified.

  In other words, according to the parameter specifying device, it is not necessary to use one of the plurality of cameras as a reference in order to specify the calibration parameter. For this reason, according to the parameter specifying device, it is possible to reduce the existence of a camera that cannot specify the calibration parameter.

It is a block diagram which shows schematic structure of a display system. It is explanatory drawing which shows the arrangement position of a camera. It is a flowchart which shows the process sequence of a bird's-eye view display process. It is explanatory drawing explaining the outline | summary of the calculation method of the parameter evaluation value performed in a bird's-eye view display process. It is explanatory drawing explaining the outline | summary of the process which maintains the orthogonality of a roll, a pitch, and a yaw performed in a bird's-eye view display process.

Embodiments of the present invention will be described below with reference to the drawings.
<1.1 Display system>
A display system 1 shown in FIG. 1 is a system that specifies camera parameters of each of a plurality of cameras 10 mounted on a vehicle 5. The display system 1 is a system that generates and displays an image obtained by performing viewpoint conversion on an image captured by each camera 10 based on camera parameters. As shown in FIG. 2, the vehicle 5 in the present embodiment is a four-wheeled vehicle.

The display system 1 includes a plurality of cameras 10, an input device 14, a sensor group 16, a control device 20, and a display device 40.
Among these, each of the cameras 10 is a known camera that captures an image of a defined area defined around the vehicle as an imaging area via a wide-angle lens. The wide-angle lens is a lens having a wide angle of view (for example, 60 degrees or more) and a short focal length. As an example of this wide-angle lens, a fish-eye lens can be considered. The fisheye lens is a lens having an angle of view of 180 degrees or more, for example.

  The defined areas referred to here are different areas defined around the vehicle 5 as shown in FIG. The defined area is defined such that areas adjacent to each other are continuous in real space. The “region defined so as to be continuous” mentioned here includes that a part of each region overlaps. The imaging region is a region to be imaged.

The camera 10 in the present embodiment includes a first camera, a second camera, a third camera, and a fourth camera.
The first camera is installed at the front end of the vehicle 5 and photographs a defined area defined in front of the vehicle 5 (hereinafter referred to as a forward defined area). The second camera is installed on the right side surface of the vehicle 5 and photographs a defined area defined on the right side of the vehicle 5 (hereinafter referred to as a right defined area). The third camera is installed at the rear end of the vehicle 5 and captures a defined area (hereinafter referred to as a rear defined area) defined behind the vehicle 5. The fourth camera is installed on the left side surface of the vehicle 5 and photographs a defined area defined on the left side of the vehicle 5 (hereinafter, left defined area).

  The input device 14 is a known device that accepts input of information. The input device 14 includes various input devices such as a keyboard, a pointing device, and a switch. The pointing device here includes a known mechanism such as a touch pad or a touch panel.

  Furthermore, the input device 14 of the present embodiment may be a diagnostic tool. The diagnostic tool referred to here is a tool (that is, a tool) that inputs / outputs information to / from the control device 20 and analyzes information acquired from the control device 20.

  The sensor group 16 is a plurality of sensors that detect the behavior of the vehicle 5. The sensor group 16 includes a wheel speed sensor that detects the wheel speed of the vehicle 5 and a yaw rate sensor that detects the yaw rate of the vehicle 5. In addition, the sensor group 16 may include other sensors capable of detecting the behavior of the vehicle 5 such as an acceleration sensor that detects acceleration applied to the vehicle 5.

The display device 40 is a known device that displays an image. A liquid crystal display is considered as an example of the display device 40.
<1.2 Control device>
The control device 20 is a device that executes processing based on images taken by the cameras 10. The control device 20 includes a storage unit 22 and a control unit 24.

Of these, the storage unit 22 is a rewritable nonvolatile storage device. As an example of the storage unit 22, a hard disk drive, a flash memory, or the like can be considered.
The storage unit 22 stores reference parameters. The reference parameter referred to here is a camera parameter obtained when each of the cameras 10 is mounted on the vehicle 5. The term “when mounted on the vehicle 5” here includes a period from when the camera 10 is attached to the vehicle 5 at the factory until the vehicle 5 is shipped from the factory.

  The camera parameter is a parameter representing the installation position and installation posture of each camera 10 in the vehicle space. Specifically, each camera parameter includes an x coordinate, ay coordinate, a height z, a pitch Θ, a roll Φ, and a yaw Ψ.

  The x coordinate is a coordinate on the vehicle space where the camera 10 is installed, and is a coordinate along the full length direction of the vehicle 5. The y coordinate is a coordinate on the vehicle space where the camera 10 is installed, and is a coordinate along the vehicle width direction of the vehicle 5. Further, the height z is a coordinate on the vehicle space where the camera 10 is installed, and is a coordinate along the vehicle height direction of the vehicle 5.

  The pitch Θ represents the rotation angle of the camera 10 around the horizontal axis. The horizontal axis referred to here is an axis parallel to the horizontal plane in the vehicle 5. The roll Φ represents a rotation angle around the lens central axis. Yaw Ψ represents a rotation angle around an axis orthogonal to the horizontal plane in the vehicle 5.

  The control unit 24 is a known control device that is configured around a known microcomputer having a ROM 26, a RAM 28, and a CPU 30. The ROM 26 stores data and programs that need to retain the stored contents even when the power is turned off. The RAM 28 temporarily stores data. The CPU 30 executes processing according to a program stored in the ROM 26 or RAM 28.

The ROM 26 of the control unit 24 stores a processing program for the control unit 24 to execute image display processing. The image display process is a process of specifying a calibration parameter and displaying on the display device 40 a viewpoint conversion image obtained by converting the viewpoint of the image captured by the corresponding camera 10 based on the specified calibration parameter. The calibration parameters referred to here are camera parameters of the cameras 10 reflecting the attitude of the vehicle 5.
<1.3 Image display processing>
Next, image display processing executed by the control unit 24 will be described.

This image display process is activated when a predetermined activation command is input. Then, it is repeatedly activated at specified time intervals.
Note that the activation command here is a command for activating the image display processing, and may be a signal indicating that the ignition switch is turned on, for example.

  When the image display process is activated, as illustrated in FIG. 3, the control unit 24 first acquires a captured image captured by the camera 10 from each of the plurality of cameras 10 (S110). The captured image referred to here is an image captured by each camera 10.

  Further, in the image display process, the control unit 24 performs viewpoint conversion on each of the captured images acquired in S110 based on the reference parameters stored in the storage unit 22 to generate a viewpoint conversion image (S120). The viewpoint conversion referred to here is a well-known process for converting a captured image into an image observed from a predetermined specific point. That is, the viewpoint converted image is an image obtained by converting the viewpoint of each captured image. As an example of this viewpoint conversion, a bird's-eye conversion that converts a point above the vehicle 5 into a bird's-eye image that is an image observed on the road surface can be considered.

  In the image display process, the control unit 24 detects a detection target from each of the viewpoint conversion images generated in S120, and generates target information that is information about the detection target (S130).

  The detection object is an object reflected in each viewpoint conversion image. Furthermore, the object is an object having continuity existing on the road surface. As an object in the present embodiment, a road marking or a lane marking drawn on the road surface with respect to road traffic can be considered. The lane markings mentioned here include the road center line, the road boundary line, the road outer line, and the like.

In addition, since the process which detects a detection target object from a viewpoint conversion image is known, detailed description here is abbreviate | omitted.
Usually, when a division line is used as a detection target, the detection target is detected as a line having a width. For this reason, the object information generated in S130 is the length and direction of the edge of the detection object on the viewpoint conversion image, the size of the width as the detection object, and the base point of the detection object. Includes reference coordinates.

Subsequently, in the image display process, the control unit 24 acquires the detection result of the sensor group 16 (S140). The detection result acquired in S140 is a vehicle behavior representing the behavior of the vehicle 5.
Further, in the image display process, the control unit 24 determines whether or not the behavior of the vehicle 5 has suddenly changed (S150). Specifically, in S150, if the detection result of the sensor group 16 in the current cycle has changed more than a threshold value from the detection result of the sensor group 16 in the previous cycle, the control unit 24 has a sudden change in the behavior of the vehicle 5. Can be determined.

  The previous cycle referred to here is a series of processes from S110 to S300 when the image display process is activated last time. The current cycle referred to here is a series of processes from S110 to S300 this time when the image display process is activated.

  As a result of the determination in S150, if the behavior of the vehicle 5 has not changed suddenly (S150: NO), the control unit 24 shifts the image display process to S160. In S160, the control unit 24 stores the object information generated in S130 in the current cycle in the storage unit 22.

  In the present embodiment, the object information stored in the storage unit 22 is at least a predetermined number of predetermined cycles. That is, the storage unit 22 stores object information for the specified number of cycles. Note that the object information for the specified number of cycles stored in the storage unit 22 may be sequentially updated.

Thereafter, the control unit 24 shifts the image display process to S170.
On the other hand, if the result of determination in S150 is that the behavior of the vehicle 5 has suddenly changed (S150: YES), the control unit 24 shifts the image display processing to S170 without executing S160.

In S170, the control unit 24 specifies a combination of temporary parameters for each of the cameras 10, and sets the combination of the temporary parameters.
The temporary parameter is a virtual camera parameter of each camera 10. The term “virtual” here refers to the assumption that it is present. Specifically, each of the temporary parameters includes an x coordinate, ay coordinate, a height z, a pitch Θ, a roll Φ, and a yaw Ψ.

  In S170 of the present embodiment, when the image display process is started and the process proceeds to S170 for the first time, the temporary parameters of each camera 10 are set to initial values. The initial value here is a value defined in advance. As an example of the initial value, only a specific parameter determined in one camera 10 is changed by a specified amount defined from a reference parameter, and other parameters and temporary parameters of other cameras 10 are changed to reference parameters. It is possible to do.

  Further, in the image display process, the control unit 24 corrects the position, inclination, and the like of the detection target stored in the storage unit 22 in S160 based on the combination of temporary parameters set in S170 (S180).

  Specifically, in S180, the control unit 24 acquires object information for the number of past specified cycles stored in the storage unit 22 from the current time. Then, by correcting the position and inclination of the detection object included in each of the acquired object information based on the temporary parameters of the corresponding camera 10, the correction object is specified and the correction object information is generated.

  The correction target is a detection target in the viewpoint conversion image when the image is subjected to viewpoint conversion based on the temporary parameter. Furthermore, the corrected object information is object information related to the correction object. The correction object information includes the length and direction of the edge of the correction object, the width of the correction object, the reference coordinates serving as the base point of the correction object, and the like.

  Subsequently, in the image display process, the control unit 24 calculates a parameter evaluation value e based on the corrected object information (S190). The parameter evaluation value e referred to here is an evaluation value representing the degree of continuity between the correction objects in an image obtained by imaging the prescribed regions adjacent to each other.

  For example, when a white line as a lane marking drawn on the road surface is used as an object, specifically, in S190, the control unit 24 images the prescribed areas adjacent to each other on each of the left and right white lines as shown in FIG. The sum of errors of the correction target obtained from the image may be calculated as the parameter evaluation value e. The calculation of the parameter evaluation value e in this case may be realized according to the following equation (1).

  However, the symbol W in the equation (1) is a weighting factor defined in advance. The symbol a is a difference in inclination between the correction objects when the correction object in the image obtained by imaging the prescribed areas adjacent to each other is a straight line. Further, symbol b is a difference in intercepts between the correction objects when the correction object in the image obtained by capturing the prescribed regions adjacent to each other is a straight line.

  Subsequently, in the image display process, the control unit 24 compares the parameter evaluation value e calculated in S190 with the optimum evaluation value stored in the storage unit 22 (S200). Further, in S200, if the parameter evaluation value e calculated in S190 is suitable as the optimum evaluation value, the control unit 24 stores and updates the parameter evaluation value e calculated in S190 as the optimum evaluation value. Specifically, in S200, if the parameter evaluation value e calculated in S190 is smaller than the optimal evaluation value stored in the storage unit 22, the control unit 24 uses the parameter evaluation value e calculated in S190 as the optimal evaluation value. What is necessary is just to determine with it being suitable.

  Furthermore, the control unit 24 determines whether or not all combinations of temporary parameters have been set (S210). If all combinations of the temporary parameters have not been set as a result of the determination in S210 (S210: NO), the control unit 24 returns the image display process to S170.

  In S <b> 170, one temporary parameter combination that has not been set is set from all the combinations of temporary parameters for each camera 10. Then, the control unit 24 executes S180 to S210.

  That is, in this embodiment, by repeating the steps from S170 to S210, combinations of the temporary parameters of the camera 10 in various cases according to the posture of the vehicle 5 are set, and the likelihood for the combination of the temporary parameters. To evaluate. As an example of changing the combination of temporary parameters, only a specific parameter determined in one camera 10 is changed from a currently set value by a specified amount, and other parameters and temporary parameters of other cameras 10 are changed. Is considered to be a value currently set. Further, when the value of a specific parameter is the final value, the target of changing the parameter value is set to the next type of parameter. Furthermore, when all types of parameters for one camera 10 have final values, the parameter values for the next camera 10 are changed.

  Thereby, the temporary parameter with the highest likelihood is a camera parameter considering the attitude of the vehicle 5. “Considering the attitude of the vehicle” means that the vehicle is regarded as a rigid body and only the inclination of the vehicle is reflected in the camera parameters. Note that examples of various cases referred to herein include a case where the front end portion of the vehicle 5 is low and a case where the rear end portion of the vehicle 5 is low. Further, examples of various cases include a case where the right side of the vehicle 5 is low and a case where the left side of the vehicle 5 is low.

On the other hand, as a result of the determination in S210, if all combinations of temporary parameters have been set (S210: YES), the control unit 24 shifts the image display process to S220.
In S220, the control unit 24 determines whether or not the optimal evaluation value is equal to or less than the determination threshold Th. The determination threshold Th referred to here is a threshold indicating that the combination of temporary parameters is more suitable for vehicle behavior than the reference parameter, and is a threshold defined in advance based on experiments or the like.

  As a result of the determination in S220, if the optimal evaluation value is equal to or less than the determination threshold Th (S220: YES), the control unit 24 uses each temporary parameter corresponding to the optimal evaluation value as a calibration parameter of the corresponding camera 10. It memorize | stores in the memory | storage part 22 (S230). That is, the calibration parameters stored in the storage unit 22 in S230 are the camera parameters of each camera 10 that generates a viewpoint conversion image in which the detection objects are continuous.

  In the present embodiment, the calibration parameters stored in the storage unit 22 are for a specific number of cycles. The specific number of cycles is a number that is defined in advance as a number of cycles that is greater than the specified number of cycles. That is, the storage unit 22 stores calibration parameters for the number of specific cycles for each camera 10. Note that the calibration parameters for the specific number of cycles stored in the storage unit 22 may be sequentially updated.

Thereafter, the control unit 24 shifts the image display process to S240.
On the other hand, as a result of the determination in S220, when the optimum evaluation value is larger than the determination threshold Th (S220: NO), the control unit 24 shifts the image display process to S240 without executing S230.

  In S240, the control unit 24 acquires all the calibration parameters of each of the stored cameras 10. Subsequently, the control unit 24 calculates the representative value of the calibration parameter acquired in S240 for each camera 10 (S250). The representative value referred to here may be an average value, a mode value, or a median value.

  Specifically, in S250, the control unit 24 calculates the representative value of the pitch Θ, the representative value of the roll Φ, and the representative value of the yaw Ψ of each camera 10, as shown in FIG. Then, the control unit 24 orthogonalizes the representative value of the pitch Θ and the representative value of the yaw Ψ with respect to the representative value of the roll Φ of the corresponding camera 10. In this case, the representative value of the pitch Θ, the representative value of the roll Φ, and the representative value of the yaw Ψ may be normalized.

  Further, in the image display process, the control unit 24 calculates the degree of variation of the calibration parameter specified during a predetermined period (S260). Specifically, in S260, the control unit 24 may calculate the standard deviation of the calibration parameters set during a predetermined period as the variation degree. In addition, the period prescribed | regulated previously here is a period when the specific cycle number is repeated the prescribed number of times, for example.

  Subsequently, in the image display process, the control unit 24 determines whether or not the calibration parameter set in S250 satisfies a preset adoption criterion (S270). The adoption standard referred to here is that the degree of variation calculated in S260 is equal to or less than a preset threshold value.

  As a result of the determination in S270, if the calibration parameter satisfies the adoption standard (S270: YES), the control unit 24 sets the representative value of the calibration parameter for each camera 10 calculated in S250 as a new camera parameter. (S280).

  Subsequently, the control unit 24 performs viewpoint conversion on the image acquired in S110 based on the camera parameter set in S280 to generate a viewpoint conversion image (S290). Thereafter, the control unit 24 shifts the image display process to S300.

  On the other hand, as a result of the determination in S270, if the calibration parameter does not satisfy the adoption criteria (S270: NO), the control unit 24 shifts the image display process to S300 without executing S280 and S290.

  In S300, the control unit 24 outputs the viewpoint conversion image to the display device 40. And the display apparatus 40 which acquired the viewpoint conversion image displays the viewpoint conversion image corresponding to each of vehicle-mounted camera.

  Note that the viewpoint conversion image displayed on the display device 40 in S300 is the viewpoint conversion image generated in S290 when the image display process proceeds to S300 through S280 and S290. On the other hand, the viewpoint conversion image displayed on the display device 40 in S300 is the viewpoint conversion image generated in S120 when the image display process proceeds to S300 without passing through S280 and S290.

Thereafter, the control unit 24 ends the image display process and waits for the next activation timing.
That is, in the image display process, when the detection target objects of the viewpoint conversion image based on the reference parameter are discontinuous, the attitude of the vehicle 5 is determined for each of the cameras 10 on the assumption that the attitude of the vehicle 5 is inclined. Obtain the appropriate camera parameters. In the image display process, as a method for obtaining an appropriate camera parameter, a search is performed for a calibration parameter that is a camera parameter of each camera 10 that generates a viewpoint-converted image in which the detection objects are continuous.

Further, in the image display process, a new viewpoint conversion image is generated and displayed on the display device 40 based on the specified calibration parameter.
In other words, the control device 20 that has executed the image display process functions as a parameter specifying device.
[2. Effects of the embodiment]
(2.1) As described above, according to the image display process of the present embodiment, the calibration parameters of each camera 10 can be specified.

  This calibration parameter is a camera parameter in which detection objects are continuous in a viewpoint conversion image obtained by performing viewpoint conversion on a captured image based on the calibration parameter, and the position of the camera reflecting the posture of the vehicle 5 at that time and It is posture.

That is, according to the image display process, when the vehicle 5 travels, the calibration parameters reflecting the posture of the vehicle 5 can be sequentially specified.
(2.2) As a result, according to the control device 20, it is possible to reduce the discontinuity of the detection target in the newly generated viewpoint conversion image, that is, the mismatch of the same point in the real space.

According to the control device 20, the continuity of images displayed on the display device can be improved regardless of the loading state of the vehicle 5.
(2.3) Furthermore, in the image display process, since each camera parameter of the camera 10 that generates a viewpoint conversion image in which detection objects are continuous is specified as a calibration parameter, a viewpoint conversion image is generated. Calibration parameters can be specified for all cameras 10.

  In other words, according to the control device 20, it is not necessary to use one of the plurality of cameras 10 as a reference in order to specify the calibration parameter. For this reason, according to the control apparatus 20, it can reduce that the camera 10 which cannot specify a calibration parameter exists.

  (2.4) By the way, in the image display process, if the behavior of the vehicle 5 is suddenly changed, the storage of the object information is avoided, and the temporary parameter is evaluated based on the object information under the circumstances. It is avoiding.

  Thereby, according to the image display process, the evaluation target of the temporary parameter can be limited to the object information based on the image taken in a situation where the vehicle 5 is stably operating, and the evaluation of the temporary parameter can be performed. Can be realized with high accuracy.

  (2.5) Further, in the image display process, when the optimum evaluation value is larger than the determination threshold Th, it is avoided to store the calibration parameter in the storage unit 22, and from the calculation of the representative value of the calibration parameter, The calibration parameter is excluded.

Thereby, according to the image display process, the representative value of the calibration parameter can be calculated in a state where the singular value is excluded, and the representative value of the calibration parameter can be made highly reliable.
(2.6) In the image display process, a viewpoint conversion image is generated and displayed on the display device 40 in accordance with the representative value of the calibration parameter whose degree of variation is equal to or less than the set threshold value.

Thereby, according to the image display device, the viewpoint conversion image displayed on the display device 40 can be a stable image, and it is possible to reduce a sudden change in the display of the image.
(2.7) By the way, when calculating the representative values of the calibration parameters, the representative values of the pitch Θ, the roll Φ, and the yaw Ψ are calculated independently. As for the representative value, orthogonality is not maintained. Then, the representative values of the pitch Θ, the roll Φ, and the yaw Ψ do not take into account the actual state of the camera, and may be far from the actual state.

However, in the image display process, the representative value of the pitch Θ and the representative value of the yaw Ψ are calculated so as to be orthogonal to the representative value of the roll Φ.
Thus, by calculating the representative value of the pitch Θ and the representative value of the yaw Ψ, the orthogonality of the roll Φ, the pitch Θ, and the yaw Ψ can be maintained, and the representative value of the roll Φ, the representative value of the pitch Θ, and the yaw Ψ The representative value of Ψ can be set in accordance with the actual situation of the camera 10.
[3. Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  (3.1) In S160 in the above embodiment, the object stored in the storage unit 22 is the object information. However, the object stored in the storage unit 22 in S160 may be each of the captured images acquired in S110. Alternatively, each of the viewpoint conversion images generated in S120 may be used.

In this case, in S180 in the image display process, the detection object may be detected from the image stored in the storage unit 22 and converted into a correction object to generate correction object information.
(3.2) In 280 in the above embodiment, the adoption criterion is that the degree of variation calculated in S270 is equal to or less than the set threshold, but the adoption criterion is not limited to this. Furthermore, S280 may be omitted.

(3.3) Part or all of the functions executed by the control unit 24 in the above embodiment may be configured by hardware by one or a plurality of ICs.
(3.4) In the above embodiment, the program is stored in the ROM 26. However, the storage medium for storing the program is not limited to this, and is stored in a non-transitional tangible storage medium such as a semiconductor memory. It may be.

  (3.5) The control unit 24 may execute a program stored in a non-transitional tangible recording medium. By executing this program, a method corresponding to the program is realized.

  (3.6) In addition to the parameter specifying device realized by the control device 20 described above, the present invention includes a display system, a program executed by a computer for specifying camera parameters, a method for specifying camera parameters, etc. It can be realized in various forms.

  (3.7) In addition, the aspect which abbreviate | omitted a part of structure of the said embodiment is also embodiment of this invention. Further, an aspect configured by appropriately combining the above embodiment and the modification is also an embodiment of the present invention. Moreover, all the aspects which can be considered in the limit which does not deviate from the essence of the invention specified by the wording described in the claims are the embodiments of the present invention.

(3.8) Symbols in parentheses described in the columns of “Claims” and “Means for Solving the Problems” indicate a corresponding relationship with the specific means described in the embodiment as one aspect. However, it is not intended to limit the technical scope of the present invention.
[4. Example of correspondence]
The function obtained by executing S110 of the image display process corresponds to an example of an image acquisition unit. The function obtained by executing S120 corresponds to an example of an image conversion unit. The function obtained by executing S130 corresponds to an example of the target detection unit.

  The function obtained by executing S170 to S250 and S280 corresponds to an example of a parameter specifying unit. The function obtained by executing S260, S270, and S290 corresponds to an example of an image update unit.

  Furthermore, the function obtained by executing S170 corresponds to an example of an assumption unit. The function obtained by executing S180 corresponds to an example of a correction unit. The function obtained by executing S190 to S250 corresponds to an example of the specifying unit.

  The function obtained by executing S190 to S230 corresponds to an example of a calculation unit. The function obtained by executing S260 corresponds to an example of a variation calculation unit.

DESCRIPTION OF SYMBOLS 1 ... Display system 5 ... Vehicle 10 ... Camera 14 ... Input device 16 ... Sensor group 20 ... Control device 22 ... Memory | storage part 24 ... Control part 26 ... ROM 28 ... RAM 30 ... CPU 40 ... Display device

Claims (7)

A parameter specifying device (20) for specifying camera parameters of a plurality of cameras (10) mounted on a vehicle (5),
Each of the plurality of cameras captures a defined area that is a different area defined around the vehicle and is defined so that adjacent areas are continuous in real space as an imaging area. Mounted on the vehicle,
The camera parameter is a parameter representing the position and orientation of each of the plurality of cameras, and the storage device (22) is a camera parameter obtained when each of the plurality of cameras is mounted on the vehicle. A certain reference parameter is stored,
An image acquisition unit (20, S110) for acquiring a captured image captured by each camera from each of the plurality of cameras;
An image conversion unit (20, S120) that generates a viewpoint conversion image that is an image obtained by performing viewpoint conversion on each of the captured images acquired by the image acquisition unit based on the reference parameters stored in the storage device;
Object detection for detecting a detection object that is an object reflected in each viewpoint conversion image from each of the viewpoint conversion images converted by the image conversion unit, with an object having continuity existing on the road surface as an object Part (20, S130),
A viewpoint conversion image in which detection objects detected by the target detection unit are discontinuous when the detection objects are discontinuous based on captured images obtained by imaging the prescribed regions adjacent to each other. A parameter specifying unit (20, S170 to S250, S280) that specifies each camera parameter of the camera that generates a calibration parameter;
A parameter specifying device comprising: an image updating unit (20, S260, S270, S290) that generates a new viewpoint conversion image based on the calibration parameter specified by the parameter specifying unit. The parameter specifying unit includes:
An assumption unit (20, S170) for setting a temporary parameter which is a virtual camera parameter of each of the cameras;
A correction unit (20, S180) for specifying a correction target obtained by correcting a detection target detected from each of the corresponding viewpoint-converted images based on the temporary parameter set by the assumption unit;
A specifying unit (20, S190 to S250) for specifying the temporary parameter as the calibration parameter, in which the correction objects specified by the correction unit are continuous;
The parameter specifying device according to claim 1, further comprising: As the temporary parameter, a plurality of cases corresponding to the posture of the vehicle are defined in advance,
The assumption unit repeatedly executes setting of one temporary parameter different from the previously specified temporary parameter from among the temporary parameters of a plurality of cases,
The correction unit identifies the correction object every time the provisional parameter is set by the assumption unit,
The specific part is:
A calculation unit (20, S190 to S230) for calculating a parameter evaluation value representing a degree of continuity between the correction objects;
The parameter specifying device according to claim 2, wherein a temporary parameter corresponding to the correction target indicating that the parameter evaluation value calculated by the calculation unit has the highest degree of continuity is specified as the calibration parameter. Each of the plurality of cameras repeatedly captures images,
The image acquisition unit acquires the image as the captured image every time an image is captured by each of the plurality of cameras.
The image conversion unit generates the viewpoint conversion image every time the captured image is acquired by the image acquisition unit,
The target detection unit detects the detection target each time the viewpoint conversion image is generated by the image conversion unit,
The parameter specifying unit specifies and accumulates the calibration parameter each time a detection target is detected by the target detection unit, and specifies a representative value of the accumulated calibration parameter as the new calibration parameter. The parameter identification device according to any one of claims 1 to 3. The camera parameters include a roll that represents a rotation angle around a lens central axis of each of the plurality of cameras, a pitch that represents a rotation angle around a horizontal axis of each of the plurality of cameras, and each of the plurality of cameras. Including a yaw representing a rotation angle about an axis perpendicular to the horizontal plane of the vehicle,
The parameter specifying unit includes:
The parameter specifying device according to claim 4, wherein the pitch and the yaw that are orthogonal to the roll are specified. The image update unit
A variation calculating unit (20, S260) for calculating a variation degree of the calibration parameter specified by the parameter specifying unit during a predetermined period;
If the variation degree calculated by the variation calculation unit is equal to or less than a preset threshold value, a new viewpoint conversion image is generated based on the calibration parameter. The parameter identification device according to one item. The object is a road marking or lane marking drawn on the road surface for road traffic;
The target detection unit
The road marking or lane marking reflected in each viewpoint conversion image is detected as the detection target object from each of the viewpoint conversion images generated by the image conversion unit, according to any one of claims 1 to 6. The parameter identification device described.