JP6943328B2 - Angle of view adjustment method - Google Patents

Description

The present invention relates to an angle of view adjusting method in a surveillance system including a camera.

For example, as a method of setting a surveillance camera, there is a method of detecting three markers and setting a surveillance area (see, for example, Patent Document 1). In this setting method, three markers are set on the horizontal plane to be monitored, and these markers are detected by image processing. In this setting method, the camera parameters are calculated by using the information of the distance between the three markers and the camera height, and the monitoring area is set by using the calculated camera parameters.

Japanese Unexamined Patent Publication No. 2001-204007

However, when a plurality of markers are arranged and the camera is set as in the conventional case, the work is complicated. For example, in a system including a plurality of cameras, it is necessary to set a marker for each camera.

An object of the present invention is to provide an angle-of-view adjustment method capable of simplifying the procedure for adjusting the angle of view in a surveillance system equipped with a camera.

The angle-of-view adjustment method of the present invention is an angle-of-view adjustment method in a surveillance system equipped with a camera, and is an existing method that exists in a shooting area by a camera, has known dimensions and a distance from the camera, and can determine a reference length. The process of setting an object as a target, the process of calculating the size of the mark captured on the image taken by the camera, the process of displaying the mark on the image so that the calculated size is obtained, and the process of displaying the mark on the image. The process of calculating the size of the mark, including adjusting the angle of view of the camera to match the display of the target to the mark, is the known data of the size of the mark based on the distance from the camera to the target. To calculate.

In this angle of view adjustment method, a known object existing in the shooting area of the camera is set as a target, and the angle of view of the camera can be adjusted using the dimensions and the distance from the camera, which are known data of this target. can. As a result, it is not necessary to bother to arrange the target object, and the procedure for adjusting the angle of view in the monitoring system can be simplified.

Further, in the step of setting an existing object as a target, an existing object whose angle with respect to the horizontal plane of the mounting surface on which the object is placed is known is set as a target. Thereby, in the step of calculating the mark arrangement, the mark arrangement can be calculated based only on the known data, the distance from the camera to the target.

Further, in the process of setting an existing object as a target, the center of the camera coordinates of the camera coincides with the center of the existing object, or the amount of deviation from the center of the existing object is known. Is set as the target. As a result, it is not necessary to measure the amount of deviation between the center of the camera coordinates and the center of the target. Alternatively, the angle of view can be adjusted in the surveillance system based on the amount of deviation of the known center of the object with respect to the camera coordinates.

According to the present invention, in a surveillance system including a short-distance camera and a long-distance camera, the procedure for adjusting the angle of view can be simplified. Further, according to the present invention, in a surveillance system equipped with a camera, the procedure for adjusting the angle of view can be simplified.

It is a perspective view which shows the state which the vehicle equipped with the obstacle detection device travels on a highway. It is a block block diagram which shows the obstacle detection device to which an angle of view adjustment method can be applied. It is a schematic side view which shows the imaging area by a short-distance camera, the imaging area by a long-distance camera, and the arrangement of a target. It is a flowchart which shows the processing procedure in the angle of view adjustment method. It is a figure which shows the target and the mark displayed on the image. It is a figure which shows the positional relationship between the image sensor of a camera, and a target. FIG. 7A is a side view showing the installation pitching angle of the camera. FIG. 7B is a plan view showing the installation pan angle of the camera. FIG. 8A is a flowchart showing a processing procedure of the angle of view adjustment method in the short-range camera during maintenance. FIG. 8B is a flowchart showing a processing procedure of the angle of view adjustment method in the long-distance camera during maintenance. Figure 9 (a), (c) is a diagram showing a target T _A and the second mark displayed on the second image. Figure 9 (b), (d) is a diagram showing a target T _A and the first mark displayed on the first image. It is a block block diagram which shows the obstacle detection apparatus to which the angle of view adjustment method which concerns on a modification is applicable. It is a flowchart which shows the processing procedure of the angle of view adjustment method which concerns on a modification.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the same parts or corresponding parts are designated by the same reference numerals, and duplicate description will be omitted.

First, an obstacle detection device (monitoring system) to which the angle of view adjustment method according to the present embodiment can be applied will be described with reference to FIGS. 1 and 2.

(Obstacle detection device)
As shown in FIG. 1, the obstacle detection device 1 is mounted on the vehicle M, for example, and detects an object (for example, a falling object B) in front of the vehicle. As shown in FIG. 2, the obstacle detection device 1 includes a camera 2, a 3DLR (3D laser radar, 3D Laser Radar) 3, a wheel speed sensor 4, and an obstacle detection unit 5.

The obstacle detection device 1 tracks the detected object, and determines the tracked object that satisfies the determination condition as an obstacle. The "obstacle" is an object that may hinder the traveling of the vehicle M, and it is necessary to change the traveling or stop and remove the object so as to avoid the object.

The camera 2 photographs the front of the vehicle and acquires image information (detection data) such as a road and an object. The image information acquired by the camera 2 is input to the obstacle detection unit 5. The camera 2 includes a short-distance camera 2A and a long-distance camera 2B. The short-range camera 2A and the long-range camera 2B are arranged at the same position in the front-rear direction of the vehicle, and are arranged side by side in the vehicle width direction, for example. The effect of distortion of the lens of the camera 2 is sufficiently small.

As shown in FIG. 3, the short-range camera 2A captures the first imaging region R _2A in front of the vehicle. The first imaging region _{R 2A} is, for example, in the range from the point _{P 1} in front of the vehicle to the point _{P 3.} The long-distance camera 2B photographs the second photographing area R _2B in front of the vehicle. The second photographing area R _2B is, for example, a range _{from the point P 2} to the point P _4. Includes a range of several hundred meters in front of the vehicle. The second shooting area R _2B includes a part of the first shooting area R _2A , and the distance from the reference position (the stop position of the vehicle M, the installation position of the camera 2) is the first shooting area R _2A (point P _3). ) Includes areas farther than. The area included in both the first photographing area R _2A and the second photographing area R _2B is referred to as a common area R _2C . The common area R _2C includes the area from the point P ₂ to the point P ₃ . Further, the first photographing area R _2A and the second photographing area R _2B may be set so as to be adjacent to each other. The common area at this time is the boundary between _{the first photographing area R 2A} and the second photographing area R _2B.

The 3DLR3 irradiates the front of the vehicle with a laser and receives the reflected laser reflected by an object on the road. The information about the reflected laser received by the 3DLR3 is input to the obstacle detection unit 5.

The wheel speed sensor 4 is a sensor that acquires information on the rotation angle of the wheels of the vehicle M. The wheel speed sensor 4 includes, for example, an encoder, which measures the rotation pulse of the wheel. The signal regarding the rotation pulse of the wheel is input to the obstacle detection unit 5.

The obstacle detection unit 5 includes an image processing unit 6, a 3DLR processing unit 7, a linear calculation unit 8, an integrated processing unit 9, and a system control unit 10. The obstacle detection unit 5 includes a CPU that performs arithmetic processing, a ROM and RAM as a storage unit, an input signal circuit, an output signal circuit, a power supply circuit, and the like.

The system management unit 10 controls the entire obstacle detection unit 5, and manages the generation of blocks and the suspension of functions of the image processing unit 6, the 3DLR processing unit 7, the linear calculation unit 8, and the integrated processing unit 9. In addition, the system control unit 10 manages data input / output with the display unit 15, the operation unit 16, and the vehicle control unit 17, which will be described later.

The image processing unit 6 performs image processing on the sensor data input from the camera 2. The image processing unit 6 performs, for example, binarization processing as image processing, and detects information on an object and information on a white line on a road. Further, the sensor data input from the camera 2 includes information on the hue of the object, information on the brightness of the object, information on the size (external dimensions) of the object, and information on the position of the object.

Further, the image processing unit 6 determines the position of the object (position in the left-right direction, position in the traveling direction of the vehicle M), size of the object (size in the width direction, size in the height direction), based on the sensor data. Calculate the hue and brightness of the object. In addition, the image processing unit 6 calculates the hue and brightness around the object.

The image processing unit 6 collates camera parameters such as the angle of view and linear information of the path with respect to the pixel size in the image, and calculates and determines the actual size of the object. The angle of view includes, for example, a focal length f (see FIG. 6), an installation pitching angle θ ₁ (see FIG. 7 (a)), an installation pan angle θ ₂ (see FIG. 7 (b)), and the like.

The 3DLR processing unit 7 performs data processing on the sensor data input from the 3DLR3. Further, the sensor data input from the 3DLR includes the position of the object (position in the left-right direction, position in the traveling direction of the vehicle M, position in the up-down direction).

The 3DLR processing unit 7 calculates the position of the object (position in the left-right direction, position in the traveling direction of the vehicle M, position in the vertical direction) based on the sensor data.

The linear calculation unit 8 calculates the moving distance of the vehicle M by using the linear information of the route stored in the storage unit and the vehicle speed acquired by the wheel speed sensor 4. "Route linear information" is information on the three-dimensional shape of the route corresponding to the distance from the starting point of the route, information on the horizontal curvature of the curve on the road, and information on the vertical gradient on the slope. Is included. The linear calculation unit 8 calculates the own vehicle position information indicating the position from the starting point of the vehicle M based on the linear information of the route and the vehicle speed.

The integrated processing unit 9 performs integrated processing for integrating sensor data of an object acquired from a plurality of sensors. The integrated processing unit 9 integrates a plurality of sensor data different in time as time-continuous data in order to obtain data suitable for tracking processing and obstacle determination processing.

The integrated processing unit 9 determines whether or not the object detected by the short-range camera 2A, the long-range camera 2B, or the 3DLR3 is an obstacle. The processing result by the integrated processing unit 9 is output to the system management unit 10.

Further, the display unit 15 and the operation unit 16 are electrically connected to the obstacle detection unit 5. The display unit 15 is, for example, a liquid crystal display device, and displays an image acquired by the camera 2. Further, based on the information output from the obstacle detection unit 5, the driver is notified by highlighting the object which is an obstacle.

Further, the operation unit 16 is, for example, a touch panel of a liquid crystal display device, and outputs a signal based on the operation input of the driver to the system control unit 10. The driver operates the operation unit 16 when performing an input operation on an object that is confirmed not to be an obstacle, for example.

Further, a vehicle control unit 17 that controls the vehicle M is electrically connected to the obstacle detection device 1. For example, an alarm device 18 and a brake 19 are electrically connected to the vehicle control unit 17. The vehicle control unit 17 can control the alarm device 18 and emit an alarm sound based on the signal from the system control unit 10. Further, the vehicle control unit 17 can operate the brake 19 to decelerate the vehicle M based on the signal from the system control unit 10.

(Angle of view adjustment method; 1st embodiment)
Next, a method of adjusting the angle of view of the camera 2 in the obstacle detection device 1 including the short-range camera 2A and the long-range camera 2B will be described. This will be described with reference to the flowchart shown in FIG. FIG. 4 is a flowchart showing a processing procedure of the angle of view adjusting method. The processing procedure shown in FIG. 4 is performed, for example, before the obstacle detection device 1 is used. For example, the angle of view adjustment method may be carried out in the factory, outdoors, or on the road before shipment. Further, the angle of view adjusting method is carried out while the vehicle M is stopped.

The angle adjusting method, first, the operator performs the step of placing the target T _A forward vehicle M (step S1). Target T _A, as shown in FIG. 5, for example, is displayed as a rectangular object on the screen. Target T _A is not limited to what is displayed as a rectangular object, for example, triangular, circular, trapezoidal, etc. linear, or intended to be displayed as the object of other shapes.

The target T _A, as shown in FIG. 3, is disposed in the common area R _2C of the vehicle M forward. Target T _A is placed into a photographable position by both near camera 2A and far camera 2B. The first imaging region _{R 2A} short-distance camera 2A is not more than the vehicle ahead of the S [m] or more 4S [m], the second imaging region _{R 2B} by long-range camera 2B is ahead of the vehicle 2S [m ] Or more and 8S [m] or less, the common area R _2C is in the range of 2S [m] or more and 4S [m] or less. In addition, "S" is an arbitrary numerical value. Distance _{L TA} from the vehicle M to the target _{T A} is, for example, 3S [m]. The distance _{L TA,} for example a camera 2 (short-range camera 2A, the long-distance camera 2B) may be the distance from to the target _{T A.}

Instead of placing the object as a target T _A, it may be used existing ones as the target T _A. For example, signs placed near the road, white lines on the road, indications on the road, other installations, and the like may be used as targets. Further, for example, in the image processing unit detects a white line on both sides of the lane, and sets the imaginary straight line intersecting these white lines may be set straight virtual as the target T _A.

Next, a step of measuring the distance from the vehicle M to the target T _A (step S2). Specifically, 3DLR3 is irradiated with laser forward vehicle, it receives a reflected laser reflected by the target T _A. 3DLR processing unit 7 performs data processing on the sensor data received from 3DLR3, detects the position of the target _{T A.} Thus, measuring the distance from the vehicle M to the target T _A. It should be noted that the 3DLR may be calibrated when the measurement is performed using the 3DLR3.

The step of measuring the distance L _TA from the vehicle M to the target T _A is not limited to being implemented using 3DLR3, other laser radar may be used range finder, in other ways , Distance _LTA may be measured. Further, when the distance L _TA from the vehicle M to the target T _A is known, it is possible to use that value.

Next, a step of calculating the arrangement of the first mark C _{1 is performed (step S3).} First mark _{C 1} is a mark showing a display target of the target _{T A} in on the first image _{J 1.} The first image J ₁ is an image taken by using the short-range camera 2A. First mark C _1, for example, has a shape corresponding to the outer shape of the target T _A. For example, if the outer shape of the target T _A is rectangular, the first mark C ₁ may be a rectangular frame body corresponding to the outer shape of the target T _A. First mark C ₁ may have a shape corresponding to a portion of the outline of the target T _A. For example, if the target T _A has a rectangular shape may be in terms of four arranged in a position corresponding to the four corners.

In step S3, for example, the image processing unit 6 of the obstacle detection unit 5 performs arithmetic processing. FIG. 6 is a diagram showing the positional relationship between the image sensor of the camera and the target. The camera in this case is the short-range camera 2A, and the image sensor 21 and the lens 22 are the image sensor 21 and the lens 22 of the short-range camera 2A.

As shown in FIG. 6, the focal length f [mm], a distance _L TA from the camera 2 (the vehicle M) to the target _{T A} [mm], the element size a of the image sensor [mm], the field of view b of the target _{T A} [Mm] satisfies the following proportional equation (1).
f: a = L _TA : b ... (1)

In the above proportional equation (1), the focal length f and the element size a are known. Distance L _TA from the camera 2 to the target T _A may be a value calculated in step S2. By substituting these values (a, b, L _TA ) into the proportional equation (1), the visual field b can be calculated.

_{In this way, the horizontal field of view b X} [mm] and the vertical field of view b _Y [mm] at the position of the target T can be obtained by using the proportional equation (1). At this time, the resolution of the image of the camera (for example, full HD: horizontal 1920 [pixel], vertical 1080 [pixel]) is known. Therefore, the resolution Gx (= 1920 / horizontal field of view b _X ) [pixel / mm] _{in the horizontal direction X and the resolution G Y} (= 1080 / vertical field of view b _Y ) [pixel / mm] in the vertical direction Y at the target position are set. Can be sought.

Because the target actual size of T _A [mm] is also known, the image processing unit 6, the above resolution Gx, by using the G _Y, the size of the first mark C ₁ to be imaged on the image [pixel] of Can be calculated.

Next, a step of displaying a first mark _{C 1} on the first image _{J 1} (step S4). In step S4, and displays a first mark C ₁ on the first image J ₁ so as to be arranged calculated in step S3. The image processing unit 6 applies the size (Tx, Ty) of _{the first mark C 1} calculated in step S3, and applies the first mark C _{1 on the first image J 1 as shown in FIG.} indicate. The position for displaying a first mark C ₁ (the display position on the image (x, y)) is the user can arbitrarily set. For example, the display position of the first mark C _{1 may be determined so that the first mark C 1} is displayed in the center of the image.

Next, a step of adjusting the angle of view of the short-range camera 2A is performed (step S5). Specifically, in the first image J on _1, as a first mark C ₁ and the target T _A displayed matches adjusts the angle of view of the short-range camera 2A. For example, the worker may manually operate the short-range camera 2A to adjust the angle of view of the short-range camera 2A, and output a signal from the obstacle detection unit 5 to drive the actuator automatically. You may adjust.

In step S5, for example, the focal length f of the short-range camera 2A, the installation pitching angle θ _{1 of} the short-range camera 2A, and the installation pan angle θ ₂ of the short-range camera 2A are adjusted. The focal length f can be changed by moving the position of the lens 22 of the short-range camera 2A. FIG. 7A is a side view showing _{the installation pitching angle θ 1} of the short-range camera 2A. As shown in FIG. 7A, for example, the angle at which the axis D extending in the horizontal direction and the optical axis L ₂ of the short-range camera 2A intersect can be set as the installation pitching angle θ _{1. For example, the installation pitching angle θ 1} can be adjusted by inclining the support base that supports the short-range camera 2A.

FIG. 7B is a plan view showing _{an installation pan angle θ 2} of the short-range camera 2A. As shown in FIG. 7B, for example, the angle at which the axis E extending in the front-rear direction of the vehicle M and the optical axis L ₂ of the short-range camera 2A intersect may be set as the installation pan angle θ _2. can. _{For example, the installation pan angle θ 2} can be adjusted by rotating the support base that supports the short-range camera 2A around an axis extending in the vertical direction.

Next, a step of calculating the arrangement of the second mark C _{2 is performed (step S6).} Second mark _{C 2} is a mark showing a display target of the target _{T A} of the on the second image _{J 2.} The second image J ₂ is an image taken by using the long-distance camera 2B. Second mark C _2, for example, has a shape corresponding to the outer shape of the target T _A. For example, if the outer shape of the target T _A is rectangular, the second mark C ₂ can be a rectangular frame body corresponding to the outer shape of the target T _A. Second mark C ₂ may have a shape corresponding to a portion of the outline of the target T _A. For example, if the target T _A has a rectangular shape may be in terms of four arranged in a position corresponding to the four corners.

In step S6, for example, the image processing unit 6 of the obstacle detection unit 5 performs arithmetic processing. The process here can be performed on the long-range camera 2B in the same procedure as in step S3. Therefore, detailed description will be omitted.

Next, a step of displaying the second mark _{C 2} on the second image _{J 2} (step S7). _{In step S7, the second mark C 2} is displayed on the second image J ₂ so as to have the arrangement calculated in step S6. The image processing unit 6 applies the size (Tx, Ty) of _{the second mark C 2} calculated in step S6 to display the second mark C _{2 on the second image J 2.} The position for displaying the second mark C ₂ (display position (x, y) on the image) can be arbitrarily set by the user. For example, the display position of the second mark C _{2 may be determined so that the second mark C 2} is displayed in the center of the image. Incidentally, FIG. 5 is illustrated as a first image _{J 1} and the second image _{J 2,} first image _{J 1} and the second image _{J 2,} a different image, each image, the target _{T A} They have different sizes, and the first mark C ₁ and the second mark C ₂ have different sizes.

Next, a step of adjusting the angle of view of the long-distance camera 2B is performed (step S8). Specifically, on the second image J _2, so that the second mark C ₂ and the target T _A displayed matches adjusts the angle of view of the long distance camera 2B. For example, the worker may manually adjust the adjustment, or a signal may be output from the obstacle detection unit 5 to drive the actuator to make the adjustment automatically.

In step S8, for example, the focal length f of the long-distance camera 2B, the installation pitching angle θ _{1 of} the long-distance camera 2B, and the installation pan angle θ ₂ of the long-distance camera 2B are adjusted. The focal length f can be changed by moving the position of the lens 22 of the long-distance camera 2B. _{Further, for example, the installation pitching angle θ 1} can be adjusted by inclining the support base that supports the long-distance camera 2B. _{Further, for example, the installation pan angle θ 2} can be adjusted by rotating the support base that supports the long-distance camera 2B around an axis extending in the vertical direction.

As described above, according to this angle of view adjustment method, both the first shooting area R _{2A shot by the short-distance camera 2A and the second shooting area R 2B} shot by the long-distance camera 2B. common area R _2C to set the target T _a, using the same target T _a, it is possible to perform the angle adjustment in the short-range camera 2A, and a view angle in the far camera 2B contained .. Thus, the target T _A to be used in the view angle of the short-range camera 2A, there is no need to arrange separately the target T _A to be used in the view angle of the long-range camera 2B, procedures in view angle Can be simplified. Furthermore, by capturing the same target T _A, easily compare the image data of the obtained target T _A short distance camera 2A, and the image data of the target T _B acquired in long distance camera 2B can do. Therefore, it is possible to improve the detection accuracy of the object based on the acquired image data.

For example, when one target is prepared for the short-range camera 2A, one target is prepared for the long-range camera 2B, a total of two targets are prepared, and the angle of view is adjusted. Since the target installation location is different for each camera, installation error may occur in each camera. In this case, since each camera shoots a different target and adjusts the angle of view, the detection position of the same object existing in the common area that is shot simultaneously by both cameras, including the installation error at each target. There is a possibility that the image processing unit 6 may be out of alignment, and the accuracy of the image processing (object detection) in the image processing unit 6, the integrated processing in the integrated processing unit 9, the determination processing, and the like may be affected.

In contrast, using the same target T _A that is allocated to the common area R _2C, when adjusting the angle of view of the short-range cameras 2A and far camera 2B as installation errors of the target T _A , Only one common installation error will occur. Also, look at the same target T _A, it is possible to adjust the angle of view, in the object detection by image processing in a subsequent stage, with the short-range camera 2A and long range camera 2B, detection of the same object The difference in position becomes smaller.

(Angle of view adjustment method; second embodiment)
Next, as a second embodiment, a method of adjusting the angle of view at the time of maintenance will be described. The maintenance time is, for example, a periodic inspection performed after the obstacle detection device 1 has been used for a certain period of time. In the description of the angle of view adjusting method at the time of maintenance, the same description as the above-mentioned angle of view adjusting method will be omitted. Further, the angle adjusting method at the time of maintenance, for example, the vehicle M is disposed at the same position as the view angle adjusting method performed before use, direct the vehicle M in the same direction, placing the target T _A at the same position.

As shown in FIG. 4, first, the processes of steps S1 to S4 are performed in the same manner as in the procedure in the angle of view adjusting method before use.

Next, as shown in FIG. 8 (a), a step of measuring a target _{T A} and the first deviation amount between the first mark _{C 1} in the above first image _{J 1} (step S21). The arrangement of the first mark C ₁ is, for example, the same as the first time. As a first shift amount, for example, may be measured and the center point of the target T _A, the deviation amount of the position of the center point of the first mark C ₁ [mm] and a corner portion of the target T _A, the corresponding may be measured a deviation amount between the positions of the first mark C ₁ of the corner, it may measure the amount of deviation of position between the other corresponding. The measurement of the first deviation amount can be performed, for example, by performing arithmetic processing in the image processing unit 6.

Next, a step of storing the first deviation amount is performed (step S22). The obstacle detection unit 5 stores the data related to the first deviation amount measured in step S21 in the storage unit of the obstacle detection unit 5.

Next, a step of determining whether or not the first deviation amount is equal to or greater than the determination threshold value is performed (step S23). The obstacle detection unit 5 includes a determination unit that determines whether or not the first deviation amount is equal to or greater than the determination threshold value. The determination unit of the obstacle detection unit 5 determines whether or not the first deviation amount measured in step S21 is equal to or greater than the determination threshold value (first determination threshold value). This determination threshold is a threshold for determining whether or not the angle of view of the short-range camera 2A needs to be adjusted, and can be determined based on, for example, experimental data and past data.

In step S23, if the first deviation amount is equal to or greater than the determination threshold value (step S23; YES), the process proceeds to step S5, the step of adjusting the angle of view of the short-range camera 2A is performed, and then the process proceeds to step S6. .. On the other hand, when the first deviation amount is less than the determination threshold value (step S23; NO), the process proceeds to step S6 without executing the step of adjusting the angle of view in step S5.

Then, after performing the steps S6, S7 shown in FIG. 4, as shown in FIG. 8 (b), the target _{T A} and the second deviation amount between the second mark _{C 2} of the above second image _{J 2} Is performed (step S31). A second shift amount, for example, the center point of the target T _A, the deviation amount of the position of the center point of the second mark C ₂ [mm] may be measured, and the corner portions of the target T _A, the corresponding The amount of deviation of the position of the second mark C _{2 from} the corner may be measured, or the amount of deviation between other corresponding positions may be measured. The measurement of the second deviation amount can be performed, for example, by performing arithmetic processing in the image processing unit 6.

Next, a step of storing the second deviation amount is performed (step S32). The obstacle detection unit 5 stores the data related to the second deviation amount measured in step S31 in the storage unit of the obstacle detection unit 5.

Next, a step of determining whether or not the second deviation amount is equal to or greater than the determination threshold value is performed (step S33). The obstacle detection unit 5 includes a determination unit for determining whether or not the second deviation amount is equal to or greater than the determination threshold value. The determination unit of the obstacle detection unit 5 determines whether or not the second deviation amount measured in step S31 is equal to or greater than the determination threshold value (second determination threshold value). This determination threshold is a threshold for determining whether or not the angle of view of the long-distance camera 2B needs to be adjusted, and can be determined based on, for example, experimental data and past data.

In step S33, if the second deviation amount is equal to or greater than the determination threshold value (step S33; YES), the process proceeds to step S8, the step of adjusting the angle of view of the long-distance camera 2B is performed, and then the process proceeds to step S8. .. On the other hand, when the second deviation amount is less than the determination threshold value (step S33; NO), the process ends without executing the step of adjusting the angle of view in step S8.

According to such an angle of view adjusting method, the angle of view of the short-distance camera 2A can be adjusted in step S5 only when the first deviation amount is equal to or greater than the determination threshold, and the first deviation amount is less than the determination threshold. In this case, it is possible not to adjust the angle of view of the short-distance camera 2A. Thereby, for example, when the first deviation amount is less than the determination threshold value and the amount is small, the angle of view adjustment is not performed, so that it is possible to prevent the first deviation amount from being significantly changed by the angle of view adjustment. In this case, the object detection in the subsequent image processing can be executed in consideration of the first deviation amount stored in the storage unit. Therefore, the object can be detected with high accuracy while maintaining a slight first deviation amount. Since it is not necessary to adjust the angle of view unnecessarily, the procedure can be simplified.

Further, the angle of view of the long-distance camera 2B can be adjusted in step S8 only when the second deviation amount is equal to or greater than the determination threshold value, and when the second deviation amount is less than the determination threshold value, the long-distance camera 2B can be adjusted. It is possible to prevent the 2B angle of view adjustment from being performed. As a result, for example, when the second deviation amount is less than the determination threshold value and the amount is small, the angle of view adjustment is not performed, so that it is possible to prevent the second deviation amount from being significantly changed by the angle of view adjustment. In this case, the object detection in the subsequent image processing can be executed in consideration of the second deviation amount stored in the storage unit. Therefore, the object can be detected with high accuracy while maintaining a slight second deviation amount. Since it is not necessary to adjust the angle of view unnecessarily, the procedure can be simplified.

(Angle of view adjustment method; 3rd embodiment)
Next, as a third embodiment, a method of adjusting the angle of view at the time of maintenance will be described. In the description of the third embodiment, the same description as in the first and second embodiments described above will be omitted.

In the view angle method, after view angle for the first time (e.g., prior to use), the position of the target T _A on the first image J _1, allowed to store data such as the size in the storage unit. Similarly, after view angle for the first time, the position of the target T _A on the second image J _2, allowed to store data such as the size in the storage unit.

Then, when the angle of view adjustment method is carried out at the time of the next maintenance, the deviation amounts K ₁ and K ₂ are detected as compared with the data of the first time (previous), and the first deviation amount on the first image J _{1 is detected.} A step of comparing K ₁ with the second deviation amount K ₂ on the second image J _{2 is performed.}

First, the second deviation amount K _{2 on the second image J 2} shown in FIG. 9 (a) and the first deviation amount K _{1 on the first image J 1} shown in FIG. 9 (b). The comparison with will be described. 9 (a) is a second image J ₂ taken by the long-range camera 2B, a diagram illustrating a target T _A and the second mark C ₂ displayed on the second image J _2. FIG. 9 (b), a first image J ₁ taken by the short distance camera 2A, a diagram illustrating a target T _A and the first mark C ₁ which is displayed on the first image J _1.

9 (a), the position where the second mark C ₂ is displayed is the same as the position of the target T _A after view angle for the first time, a data stored in the storage unit. Before maintenance after use for a certain period after view angle for the first time, if there is no change in the angle of view of the short-range camera 2A, the target T _A and the second mark displayed on the second image J ₂ C ₂ and the position is likely to match.

In FIG. 9 (a), the display position of the view angle before the target T _A during maintenance is offset from the position after the angle adjustment of the first (second mark C _2). On the second image J ₂ is shifted to the right obliquely upward. At this time, an arrow may be displayed _{on the second image J 2} as a display indicating the second deviation amount K _2.

In FIG. 9 (b), the position where the first mark C ₁ is displayed is the same as the position of the target T _A after view angle for the first time, a data stored in the storage unit. Before maintenance after use for a certain period after view angle for the first time, if there is no change in the angle of view of the short-range camera 2A, the target T _A and the first mark is displayed on the first image J ₁ There is a high possibility that C _{1 and the position will match.}

9 (b), the display position of the view angle before the target T _A during maintenance is offset from the position after the angle adjustment of the first (first mark C _1). On the first image J ₁ is shifted to the right obliquely upward. At this time, an arrow may be displayed _{on the first image J 1} as a display indicating the first deviation amount K _1.

For example, the image processing unit 6 compares the first shift amount _{K 1} in the above first image _{J 1,} and a second shift amount _{K 2} of on the second image _{J 2.} For example, the obstacle detection unit 5, a first direction of displacement amount K ₁ in the above first image J _1, and a second direction shift amount K ₂ of on the second image J ₂ is coincident Judge whether or not. In the case shown in FIG. 9 (a), (b), since the direction toward the arrow indicating the shift amount K _1, K ₂ are the same, the obstacle detection unit 5, first the on the first image J ₁ It is determined that the direction of the _{1 deviation amount K 1 and} the direction of the second deviation amount K _{2 on the second image J 2} coincide with each other.

As shown in FIGS. 9A and 9B, when the deviations are in the same direction, the effects of the deviations of the installation pitching angle and the installation pan angle on the short-range camera 2A and the long-range camera 2B, respectively. Is small, and there is a possibility that the installation position of the vehicle M at the time of adjusting the angle of view this time, the orientation of the vehicle M, and the like are deviated from those at the time of adjusting the angle of view last time. In this case, the determination unit of the obstacle detection unit 5 can decide not to adjust the angle of view.

Next, the second deviation amount K _{2 on the second image J 2} shown in FIG. 9 (c) and the first deviation amount K on _{the first image J 1} shown in FIG. 9 (d). _{The comparison with 1} will be described. 9 (c) is a second image J ₂ taken by the long-range camera 2B, a diagram illustrating a target T _A and the second mark C ₂ displayed on the second image J _2. FIG. 9 (b), a first image J ₁ taken by the short distance camera 2A, a diagram illustrating a target T _A and the first mark C ₁ which is displayed on the first image J _1.

Since FIG. 9 (c) is the same as FIG. 9 (a), the description thereof is omitted here. In FIG. 9 (d), the _{position where the first mark C 1} is displayed is the same as in FIG. 9 (b).

In FIG. 9 (d), the display position of the view angle before the target T _A during maintenance is offset from the position after the angle adjustment of the first (first mark C _1). On the first image J ₁ is shifted to the right obliquely downward. At this time, an arrow may be displayed _{on the first image J 1} as a display indicating the first deviation amount K _1.

For example, the image processing section 6 includes a first shift amount _{K 1} in the above first image _{J 1,} compared with the second shift amount _{K 2} of on the second image _{J 2,} on the first image _{J 1} determining a first direction of the displacement amount K _1, a second direction displacement amount K ₂ of on the second image J ₂ is, whether or not the match. FIG. 9 (c), the in the case shown in (d) is because the direction toward the arrow indicating the shift amount K _1, K ₂ different, the obstacle detection unit 5, the first shift in on the first image J ₁ It is determined that the direction of the _{quantity K 1 and} the direction of the second deviation amount K _{2 on the second image J 2} do not match.

As shown in FIGS. 9C and 9D, at least one of the installation pitching angle and the installation pan angle in at least one of the short-range camera 2A and the long-range camera 2B when the deviations are in different directions. May be out of alignment. The cause of the deviation of the installation pitching angle and the installation pan angle is considered to be the influence of vibration due to the running of the vehicle M. In this case, the determination unit of the obstacle detection unit 5 can decide to adjust the angle of view. In this case, the step of adjusting the angle of view of the short-distance camera 2A (step S4) and the step of adjusting the angle of view of the long-distance camera 2B (step S8) are executed.

Further, for example, a first shift amount _{K 1} in the above first image _{J 1,} in the step of comparing the second shift amount _{K 2} of on the second image _{J 2,} the displacement amount _K 1, _{K 2} size You may compare the sa (length of the arrow). Further, "the outer shape of the target T _A in the first image J ₁ magnification change of the first mark C ₁ contour", "the outer shape and the second mark of the outline of the target T _B in the second image J ₂ It may be compared with "magnification of change". As a result, it is possible to determine whether the angle of view of the camera is deviated from the previous time of adjusting the angle of view, or whether the angle of view of the camera is not deviated but the deviation is caused by other causes. For example, when the magnification of the change in the outer shape _{is different between the first image J 1} and the second image J ₂ , there is a possibility that a problem may occur in the adjustment of the focal length f due to the influence of the vibration of the vehicle M.

(Angle of view adjustment method; modification example)
Next, the obstacle detection device and the angle of view adjusting method according to the modified example will be described with reference to FIGS. 10 and 11. In the description of the modified example, the same description as in the first to third embodiments described above will be omitted.

The obstacle detection device 1 according to the modified example shown in FIG. 10 is different from the obstacle detection device 1 of the above embodiment shown in FIG. 2 in place of the short-distance camera 2A and the long-distance camera 2B. The point is that one camera 2 is provided and the 3DLR is not provided. Further, the obstacle detection unit 5 of the obstacle detection device 1 is different from the obstacle detection unit 5 of the obstacle detection device 1 in that the obstacle detection unit 5 is not provided with the 3DLR processing unit 7.

In the angle of view adjusting method according to the modified example, as shown in FIG. 11, the operator sets an existing object as a target (step S11). This existing object exists in the photographing area by the camera 2. The dimensions (eg, height) of this existing object are known, and the distance from the camera 2 to the existing object is also known. This existing object is an object whose reference length is known. The reference length of an existing object is, for example, the width of a road or the width of a rail.

Further, in step S11, an existing object whose angle with respect to the horizontal plane of the mounting surface on which the object is mounted is known is set as a target. For example, an object whose reference length is known in the longitudinal direction, the horizontal direction, and the angle is set as a target. The angle of the mounting surface on which the object is mounted with respect to the horizontal plane is stored in the storage unit of the obstacle detection device 1.

Further, in step S11, the existing object whose deviation amount between the center of the camera coordinates of the camera 2 and the center of the existing object is known is set as a target. The center of the existing object may coincide with the center of the camera coordinates of the camera 2.

In step 11, for example, the operator inputs an operation using the operation unit 16, specifies an existing object, and sets the object as a target. The obstacle detection unit 5 recognizes an existing object as a target based on the information input from the operation unit 16.

Next, the obstacle detection unit 5 reads the distance from the camera 2 to the target (step S12). The obstacle detection unit 5 reads data (known data) related to the distance to the target stored in the storage unit.

Next, a step of calculating the arrangement of marks is performed (step S13). The mark is a mark indicating a display target of the target on the image. The image here is an image taken by using the camera 2.

Next, a step of displaying the mark on the image is performed (step S14). In step S14, the mark is displayed on the image so as to have the arrangement calculated in step S13. The image processing unit 6 applies the mark size calculated in step S13 to display the mark on the image.

Next, a step of adjusting the angle of view of the camera 2 is performed (step S15). Specifically, the angle of view of the camera 2 is adjusted so that the displayed mark and the target match on the image.

According to such an angle of view adjustment method, a known object existing in the shooting area of the camera 2 is set as a target, and the camera 2 uses the dimensions and the distance from the camera 2 which are the known data of the target. The angle of view can be adjusted. As a result, it is not necessary to bother to arrange the target object, and the procedure for adjusting the angle of view in the obstacle detection device can be simplified.

Further, in the step of setting an existing object as a target (step S11), an existing object whose angle with respect to the horizontal plane of the mounting surface on which the object is mounted is known is set as a target. As a result, in the subsequent step of calculating the mark placement, the mark placement can be calculated based only on the known data, the distance from the camera to the target.

Further, in the step of setting an existing object as a target (step S11), the known object whose deviation amount from the center of the camera coordinates of the camera 2 is known is set as a target. As a result, it is not necessary to measure the amount of deviation between the center of the camera coordinates and the center of the target. Alternatively, the angle of view of the obstacle detection device 1 can be adjusted based on a known deviation amount of the center of the object with respect to the camera coordinates.

The present invention is not limited to the above-described embodiment, and various modifications as described below are possible without departing from the gist of the present invention.

For example, in the angle-of-view adjustment method of the above-described embodiment, the processing procedures can be appropriately replaced and executed. In the above embodiment, as shown in FIG. 4, steps S3 to S5 are executed to adjust the angle of view of the short-range camera 2A, and then steps S6 to S8 are executed to adjust the angle of view of the long-range camera 2B. However, steps S6 to S8 may be executed to adjust the angle of view of the long-distance camera 2B, and then steps S3 to S5 may be executed to adjust the angle of view of the short-range camera 2A. ..

Also, run the step of calculating the placement of the second mark C2 steps and S6 calculating a first arrangement of the marks C ₁ in step S3 in advance, the calculation result may be stored in the storage unit. Then, in step S4, the first display mark C ₁ to read first on the image J ₁ data from the storage unit, similarly in step S7, the second mark on the read second image J ₂ data from the storage unit C ₂ may be displayed.

Further, in the above embodiment, the angle of view adjustment method in the surveillance system including the short-distance camera 2A and the long-distance camera 2B is described, but the surveillance system including three or more types of cameras having different shooting areas The angle of view adjustment method can also be applied. In this case, it can be applied to any two cameras out of three or more types of cameras.

Further, in the above embodiment, the case where the obstacle detection system is mounted on the vehicle is described, but the obstacle detection system may be mounted on a moving body other than the vehicle. For example, an obstacle detection system may be mounted on a robot, an airplane, a ship, a railroad, or the like. Further, the obstacle detection system may be installed in another place or object without being mounted on the moving body. For example, an obstacle detection system may be installed in a factory, a store, a hospital, an airport, a station, a building around a roadway, or the like to detect a potentially dangerous object as an obstacle. The object is not limited to a stationary object, but may be a moving object, a person, or an animal. Further, the obstacle is not limited to a dangerous object, but may be an obstacle object or an object that meets specific conditions.

1 Obstacle detection device (monitoring system)
2 cameras,
2A Short-range camera 2B Long-range camera 3 3DLR (3D laser radar)
4 Wheel speed sensor 5 Obstacle detection unit 6 Image processing unit 7 3DLR processing unit 8 Linear calculation unit 9 Integrated processing unit 10 System control unit 15 Display unit 16 Operation unit 17 Vehicle control unit 18 Alarm device 19 Brake 21 Image sensor 22 Lens a Element size b View field f Focal length C1 1st mark C2 2nd mark D Horizontally extending axis E Axis extending in the front-rear direction of the vehicle J ₁ 1st image J ₂ 2nd image K ₁ 1st deviation amount K ₂ second shift amount L ₂ optical axis L _TA distance M vehicle R _2A first imaging region R _2B second imaging region R _2C common region T _a target theta ₁ installed pitching angle theta ₂ installed pan angle from the vehicle to the target

Claims (3)

It is a method of adjusting the angle of view in a surveillance system equipped with a camera.
A process of setting an existing object existing in the shooting area by the camera, whose dimensions and distance from the camera are known, and whose reference length can be determined, as a target.
The process of calculating the size of the mark captured on the image taken by the camera, and
The process of displaying the mark on the image so that the size is calculated, and
Including a step of adjusting the angle of view of the camera so that the display of the target matches the mark on the image.
The step of calculating the size of the mark is an angle of view adjusting method for calculating the size of the mark based on the distance from the camera to the target, which is known data. The angle of view adjusting method according to claim 1, wherein in the step of setting the existing object as a target, the existing object whose angle with respect to the horizontal plane on which the object is placed is known. .. In the step of setting the existing object as a target, the center of the camera coordinates of the camera coincides with the center of the existing object, or the amount of deviation from the center of the existing object is known. The angle of view adjusting method according to claim 1 or 2, wherein the object of the above is set as the target.

Images