JP3228603B2 - Image monitoring device and its use - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image monitoring apparatus for extracting the characteristics of an image picked up by a television camera and obtaining a distance in order to monitor the approach of a crane equalizer to an overhead transmission line. . For example, there is a case where work is performed at a high place using a crane vehicle near a power transmission line. in this case,
There is a possibility that the crane truck or the worker may come into contact with the power transmission line. Therefore, during work, it is necessary to monitor the distance from the transmission line for safety. In the past, one of the workers was constantly watching the transmission lines and monitoring for danger. However, doing so is wasteful because the observer cannot participate in the work. Also, an accident may occur the moment the observer takes his eyes off.

[0002]

2. Description of the Related Art It is conceivable to take an image of the periphery of a work place with a television camera, process the image, and automatically measure the distance from a power transmission line to confirm whether or not it is safe.
Japanese Patent Application No. 63-260018 provides an "abnormal approach detector" for such purpose. This is a dynamic approach detection method. Television towers, power lines, crane cars, etc. are imaged by television cameras. A hazardous area according to the degree of danger is set in advance around the power transmission line, and
When the vehicle comes in, it is determined that the vehicle is approaching abnormally. The captured image is digitally converted and stored in a memory.

[0003] As the crane vehicle approaches the transmission line further, the pixel signal of the digital image in the dangerous area changes. This change is detected by the density change detection unit. As a result, an abnormal approach of the crane vehicle is detected. A plurality of different dangerous areas are provided around the power transmission line, and it is possible to determine the approach and separation of the crane vehicle by the temporal relationship between the concentration changes. When the crane approaches the transmission line to a dangerous extent, a warning is issued from the speaker and the operator or driver is notified.

[0004] Japanese Patent Application No. 3-98193 discloses that a plurality of cameras arranged in a straight line simultaneously capture the same image, digitally convert each of them, extract feature points, and capture the same feature points with each camera. In this case, there is a relational expression that should be satisfied. Therefore, it is determined whether or not the relational expression is satisfied, and the distance between the camera and the characteristic point is calculated using only correct characteristic point information. At this time, the range of the distance from the camera is limited to shorten the calculation time.

[0005] Japanese Patent Application No. 3-98194 also discloses that a plurality of cameras arranged in a straight line use a transmission line, a tower, a crane.
The same thing such as a car is imaged at the same time.
Digital conversion, extraction of feature points, determination of whether or not a relational expression that holds when the same feature point is captured by each camera are satisfied or not are determined, and cameras and feature points are obtained using only correct information. By taking the time derivative of the feature points, the distance is calculated only for the remaining feature points. This is to detect an abnormal approach of a crane vehicle to a transmission line by focusing only on dynamic feature points.

The above two inventions attempt to avoid an error by imaging an object with a camera arranged on a plurality of straight lines, detecting a common feature point, and checking whether or not feature point extraction is correct. I have. Further, the feature points to be considered are limited to shorten the calculation time.

[0007]

[Problems to be Solved by the Invention]
No. 3-260018 sets a dangerous area around a power transmission line, and detects an approach of a crane vehicle to the dangerous area based on a change in image density inside the dangerous area. But this method
Correct results can only be given if the change in density is due to the movement of the crane. Other objects may move by chance and cause a change in density, which may cause malfunction. Also, the danger area must be properly set. The position of the camera must be such that it can accurately divide the danger area.

That is, the dangerous area is not defined three-dimensionally, but merely observes a two-dimensional projection viewed from a camera. It is essentially a two-dimensional operation. If the danger area is not well positioned in the camera, the danger area and the safety area overlap, and it is not known whether the crane vehicle is actually in the danger area or in the safe area. This is because the image captured by the camera is handled as a two-dimensional image as it is.

Japanese Patent Application No. 3-98193, Japanese Patent Application No. 3-98
No. 194 arranges three or more cameras on a straight line, images the same feature point with three or more cameras, and checks whether there is a certain relationship between positions on the screen. After confirming that the feature points are the same except for those that are not relevant, the distance and angle to the feature point are determined to determine three-dimensional coordinates. This is completely different from the two-dimensional method described above. The positions of the feature points are obtained three-dimensionally. Since the positions of the transmission line, the tower, the crane and the like can be determined three-dimensionally, the distance between them can be calculated.

[0010] Distance measurement using two cameras has long been used as triangulation. This is because when the same object is viewed by two cameras, the angles are different, so that the distance can be known from the distance between the cameras and the angle. Because there are many similar points, it is difficult to identify feature points of an object in real time with two cameras,
It is possible with three or more units.

The method using three or more cameras arranged in a straight line still has the following drawbacks. This means that distance measurement cannot be performed on an object parallel to the straight line on which the cameras are arranged.
If the cameras are arranged horizontally, observing a plurality of horizontal transmission lines with them does not reveal the perspective relationship.
It suffices if one point of the transmission line can be determined as a feature point, but if it is not, it is impossible to know even the difference in perspective. Distance measurement is of course not possible. Conversely, when cameras are arranged along the vertical axis, it is not possible to measure the distance to objects arranged in the vertical direction, such as telephone poles and towers.

That is, when an object parallel to the base axis where cameras are arranged is targeted, the distance to the object cannot be measured. The present invention solves such difficulties, and any arrangement,
A first object is to provide an image monitoring device capable of determining a three-dimensional position of a shaped object. Danger areas can be set three-dimensionally around transmission lines and towers, and the position of the crane car can be determined three-dimensionally, and the distance between the crane car and the transmission line or tower can be detected to avoid danger. It is a second object of the present invention to provide such an image monitoring device.

[0013]

The present invention uses a camera group having three or more cameras arranged on two orthogonal axes. Thus, the same object can be observed, and the distance and the azimuth to the object can be determined based on the difference in position on the screen.

A horizontal object such as a transmission line can be accurately located by a group of cameras arranged vertically, and an object extending vertically such as a tower or a telephone pole can be accurately located by a group of cameras arranged horizontally. .

The feature point of the object is determined by using the difference in density of the image. The density difference is obtained by differentiating the density of the image. Accurate position determination can be performed by a camera group arranged in the same direction as the direction in which the density change occurs.

A dangerous area is set three-dimensionally around a power transmission line or a steel tower so that it is possible to calculate whether a crane vehicle or the like is in the dangerous area. When a crane enters a danger area, a warning is issued by some means.

[0017]

FIG. 1 is a schematic diagram showing the configuration of the present invention. Three or more camera groups are arranged in the horizontal direction (y direction). These are denoted by C ₁ , C ₂ , C ₃ ,.... There is another group of cameras, which are arranged vertically (z direction). These are referred to as D ₁ , D ₂ , D ₃ ,. Let P be a partial point of the target object. Here, the y axis is taken horizontally and the z axis is taken vertically. All cameras are oriented in the x-axis direction perpendicular to the yz plane. Since the center lines of sight of the cameras are parallel, there is a certain relationship in the images for the same object point. Position measurement is performed after setting such conditions.

An image of the point P is formed on all cameras. Although the images are the same point, the position of the image differs depending on the position of the camera.
The camera screen can be located at a focal length f behind the lens. Since the camera's line of sight is parallel to the x-axis, the screen is y
This is a two-dimensional plane composed of a z-plane. The position of the image on the screen may be called a phase.

Since the central axes of the camera groups are parallel, the z coordinate of the image of P is equal in the camera groups arranged in the horizontal direction (y-axis). However, the y coordinates are different. The y coordinate of the object P is obtained from the difference in the y coordinate. The distance from the observation point (the center of the camera group) can also be obtained.

The images of the object P in the camera group arranged in the vertical direction (z axis) have the same y coordinate. However, the z coordinate differs depending on the height of the camera. The z coordinate of the object P can be accurately obtained by using the difference in the z coordinate.

The cameras C ₁ and C arranged in the horizontal direction (y direction)
_2, C _3, by ... group, not that it is impossible to determine the z coordinates of the object P. Of course, the z coordinate can be determined. In the above-mentioned Japanese Patent Application Nos. 3-98193 and 3-98194, the three-dimensional position of an arbitrary object P can be determined by such a camera arrangement. It has been described that the z-coordinates of the images of the object P of the camera groups C ₁ , C ₂ , C ₃ ,... The z-coordinate of the object P is obtained from the z-coordinate and the distance already obtained.

When the target is a clear point like this,
Even when observing only the horizontal camera group, the position of the point can be determined accurately. However, in the case of a horizontally long object such as a power transmission line, it is difficult to associate feature points of the same object in the horizontal camera group, and the distance cannot be accurately obtained depending on the camera group parallel to the horizontal camera group.

Therefore, according to the present invention, the camera groups C ₁ , C ₂ , C ₃ ,..., D ₁ , D ₂ ,
D ₃ , ... are arranged. In this case, since the camera groups work complementarily, the three-dimensional position of an object having any shape can be accurately determined.

First, it is shown that the coordinates of an object in a direction parallel to the camera group can be accurately obtained by a camera group arranged on a straight line. Camera groups C ₁ and C arranged in the horizontal direction (y-axis)
_2, C _3, ... is the y coordinate of the object, camera group D ₁ arranged in the vertical direction, D _2, D _3, ... can determine the z coordinates of the object accurately.

FIG. 2 shows a camera group D ₁ , D ₂ ,
D ₃ ,... Are shown as examples. Since this figure is a projection on the xz plane, the y coordinate is not known. The distance from the origin of the camera D _j is h _j . The x, y, and z coordinates of the object P are represented by X, Y, and Z, respectively. Since the camera is oriented in the x direction, an angle formed by a straight line connecting the optical axis, the object P, and the center of the camera is α _j . The z coordinate of the position of the image on the screen of the camera D _j that phase v _j. Since the screen is separated from the lens by f, the phase v _j is

V _j = ftan α _j (1)

## EQU1 ## The same α _j satisfies the following relationship with the object.

H _j −Z = X tan α _j (2)

This relationship applies to all cameras D₁ , D_Two , D
_Three , ... hold. Unknowns are X, Z, α_j In
You. h_j Is determined from the beginning by the distance of the camera from the origin
I have. v _j Is the observed quantity. Any two cameras D_i ,
D_j Since the above relationship holds,

[0030] _{(v i -v j) / f} (h i -h j) = 1 / X (3)

The following holds. (H _i -h _j ) f is a known amount. (V _i -v _j) is the observed quantity. Therefore, the X coordinate of the object P is determined by a combination of any two cameras. Since there are three or more cameras, the number of combinations of two cameras is quite large. Assuming that the number of cameras is n, the number of combinations is n (n-1) / 2.

The measurement is performed with redundancy. This is because a feature point is identified, erroneous data is discarded, and highly accurate measurement is performed. For example, the above expression is made between adjacent cameras, and it is checked whether or not the value of X matches in all combinations. Alternatively, the above equation may be made between the central camera and the other cameras to confirm the feasibility.

In any way, as long as three or more cameras are used, it is possible to confirm whether or not the feature point identification is correct. Also, after discarding the erroneous feature points, the average value of X obtained by these equations can be used as X, so that the accuracy can be improved by the processing after measurement.

For example, when the distance between adjacent cameras is the same, the unit distance is h, the phase of the image between adjacent cameras is v _j , v _{j + 1} , and (n-1) equations ,

(V _j −v _{j + 1} ) / fh = 1 / X (4)

Holds. When the number of cameras is five, four such equations hold. If the same numerical value is given within the allowable error range, it means that the same feature point is being viewed. If this is not the case, proceed with the calculation excluding the one that did not. Thus, the distance X of the object P to the plane yz including the camera group is obtained. The height can be obtained by multiplying the phase v _j by X / f. This can be determined by the number of cameras. Further y coordinate camera _{D 1, D 2, D 3} ,
.. Can be obtained from the horizontal phases (which are equal).

The above description is for the case where the camera groups are arranged in the height direction (z direction). The same applies to the case where the camera groups are arranged in the horizontal direction. In this case, the position or phase of the z direction of the image expressed by u _m, the y-direction position from the origin of the camera as k _m, holds the same equations (3).

(U _m -u _k ) / f (k _m -k _k ) = 1 / X (5)

Thus, the distance X of the object P from the camera plane can be obtained. From this, the y coordinate of the object P can be determined by multiplying the phase u by X / f. Also, the z coordinate is determined.

As described above, if the object P is a point, the vertical distance X from the camera plane is determined by the row of any camera group.
Alternatively, three-dimensional coordinates can be obtained. However, when the object P is linear, it is difficult to determine the position of the feature point on the line. Therefore, the distance is measured using a camera group arranged in a direction perpendicular to the line. Here is one of the gist of the present invention.

Since the present invention further monitors the approach of an object to a dangerous object, a dangerous area is defined around the dangerous object. FIG. 3 shows an example in which a dangerous area is provided around a transmission line. This can be determined arbitrarily. However, in the case of a wire like a transmission line, it can be determined that a range of a radius from the line is a dangerous area. Danger areas can therefore be defined by certain inequalities. The dangerous area can be defined not as one but as double as shown (danger area E, dangerous area F).

There is a crane vehicle near the vehicle. Three-dimensional coordinates are obtained for the external points of the crane vehicle, and it is determined whether these coordinates are outside or inside the dangerous area. This is easy if the formula defining the danger area is simple.
This is because it is easy to calculate how far any point on the crane vehicle is from the transmission line. Of course, the three-dimensional coordinates can be obtained only for the outline points visible from the camera group. Therefore, the location of the surveillance cameras should be devised so that they can be clearly seen from the crane car and the transmission line.

This is the case of a transmission line, but in the case of a vertical object such as a steel tower, the dangerous area is also a vertically long cylindrical space. Now, image processing will be described. FIG. 4 shows the outline. An image is obtained by observing the outside with a camera. The image is divided into information for each pixel, and A / D converted to obtain a digital signal for each pixel. The pixel is the minimum unit of an image having N × M pixels arranged vertically and horizontally. For example, there are various cameras of 512 pixels × 1024 pixels. Feature points are extracted from pixel data based on the difference between light and dark.

Actually, there are not isolated feature points, but feature points are arranged in accordance with the outline of the object. Then, the same points are associated with each other according to the outer shape. The images of the same point from different cameras correspond to each other. If you can handle it, calculate the above from the position of that point on the screen,
The feature points are identified. Further, the three-dimensional coordinates of the point are obtained by the above-described calculation method.

FIG. 6 shows a frame memory for storing the values of the pixels constituting the screen. If the pixel value is s bits, an s-bit frame memory is used. Pixels are represented by vertical and horizontal coordinates (u, v). Let the pixel value be g _uv . G ₀₀ , g ₁₀ ,... Are arranged from the upper left of the screen. There are (I + 1) pixels horizontally and (J + 1) pixels vertically. It may be a color image or a black and white image. The value g of these pixels is made to correspond to the degree of light and dark. If there is an object, a difference in brightness occurs, so that the outline of the object can be known.

As shown in FIG. 5, transmission line J, tower A,
Assume that there are B, building K, and the like. Crane near power line J
It is assumed that there is a car C and these are observed by the monitoring device M. The feature point image viewed by one camera is as shown in FIG. Since the images are viewed by many cameras, similar but slightly different feature point images are obtained by the number of cameras.

As for the feature points, since the object exists continuously, the feature points also form a continuous curve. All lines of the tower are also a set of feature points, and lines of transmission lines are also a set of feature points.
A building is also a set of feature points.

The three-dimensional coordinates are calculated for all the feature points in all the objects. There are some feature points that are easily associated and some that are difficult to associate. The top of the tower, the apex of the antenna, the corner of the building, the advertisement mark, etc. can be easily associated. However, it is difficult to provide a fixed position for the feature points of the tower. In the case of a transmission line, it is difficult to determine the position in the lateral direction because they are the same in the lateral direction.

However, this is not a problem in the present invention. This is because the camera group whose feature point is easy to define complementarily is used.

In the case of an object extending vertically as shown in FIG. 8, the distance is measured by a group of cameras C ₁ , C ₂ , C ₃ ,. Since the camera group axis is orthogonal to the spread of the object, the y-coordinate of the image of the feature point hardly differs on the screen even if the feature point identification is slightly shifted in the vertical direction (in the z direction). The important thing is the x-coordinate and the y-coordinate because they extend vertically.
The y-coordinate on the screen is obtained to confirm the validity of the above-described constraint condition. Then, when it is found that they are the same feature points (they may be slightly shifted in the z direction), x
Find coordinates. Further, the y coordinate is also obtained. The z coordinate is also obtained, which is obtained from the phase v of the camera in the z direction. This may have some errors.

In the case of an object extending horizontally as shown in FIG. 9, the distance is measured by a group of cameras D ₁ , D ₂ , D ₃ ,. Since the camera group axis and the spread of the object are orthogonal to each other, it may be slightly shifted laterally when identifying the feature points. What matters is the x and z coordinates. In each camera image, the z-coordinate of the feature point is obtained to check whether or not the above-described constraint condition is satisfied. Then, for those that are satisfied, the x coordinate is calculated, and the z coordinate is further obtained. Similarly, the y coordinate is obtained.

In the case of a crane wheel whose contour is bent and the feature points can be easily identified, the three-dimensional coordinates of the feature points can be correctly obtained by either camera group. Such an operation repeats in time. When the coordinates are obtained for all the contour lines, the same is repeated, not the end. Since there are moving objects in the meantime, it is necessary to always obtain the latest position information.

In this way, three-dimensional coordinates are obtained for characteristic points of a transmission line, a tower, a crane car and the like. However, since it is obtained from the difference between the feature point and the light and dark densities, it is a contour of a continuous object. That is, three-dimensional coordinates are obtained for all points on the contour. The coordinates can be obtained only for the part that falls within the field of view of the camera, but if the position of the camera is set at an appropriate position, the
It can constantly monitor the distance between vehicles and the nearest points, such as power lines and towers.

The calculation time increases as the number of cameras and the number of feature points increase. Therefore, the following may be performed in order to reduce the calculation time. One is to obtain the coordinates of the feature points of the object only in the area near the crane wheel. Since the range is limited, calculation time can be reduced. As a method of imposing spatial restrictions,
A camera whose distance from the camera group is within a certain range may be selected.

FIG. 10 shows an example in which the target area is limited by the distance from the monitoring apparatus. It covers only the front surface of the monitoring device and the vertical distance from a to b. This is the area where the crane vehicle is located. Because the purpose is to ensure the safety of the crane vehicle, it is not necessary to monitor the vehicle too far away from the crane vehicle.
When the restriction is made as shown in FIG. 10, only a part of the transmission line and the crane exist in the image to be processed as shown in FIG. This limits the range and reduces processing time.

Alternatively, since a moving object such as a crane wheel is dangerous, the coordinates may be obtained for only the moving object by calculating the time derivative. Of course, it is after the data of the stationary transmission line and the tower are obtained first, the coordinates of the outline are obtained, and the dangerous area is set. After this, there is no need to update the data for stationary ones.

[0057]

FIG. 12 is a schematic perspective view of an image monitoring apparatus according to an embodiment of the present invention. In this example, three cameras are arranged horizontally and three cameras are arranged vertically. There are a total of five. This is a minimal case. More cameras are available. The center camera is commonly included in the upper, lower, left, and right camera groups, but need not be. One group of cameras is a gantry 6,
7 are connected to each other. The camera group is on the platform 8. The head 8 is supported by a tripod 9. This is a simplification and in practice it is better to use a more solid base. In addition, it is desirable to be able to freely turn between the base and the camera platform.

The cameras constitute a visual device, which is connected to the main unit 10 by an appropriate number of cables. The main unit 10 is connected to a power supply device 11. An alarm device base station 12 for issuing an alarm is provided near the main unit. This sends an alert wirelessly. In the vicinity of a crane car, etc., an alarm device mobile station 1
3 is provided with an alarm issuing device 14, which alerts a driver or worker of a crane vehicle or the like by sounding an alarm transmitted wirelessly.

FIG. 13 is a diagram showing a schematic configuration of the apparatus of the present invention. This is roughly divided into a camera 20, an image measurement unit 21, an image output unit 22, an access restriction area update unit 23,
System control unit 24, approach monitoring operation unit 25, alarm data output unit 26, man-machine interface units 27 and 28,
It comprises an alarm device 29 and the like. As described above, three or more cameras are arranged along two orthogonal axes such that their optical axes are parallel.

Although all the feature points of one object may be observed by all the camera groups arranged vertically and horizontally, the following is more effective. That is, the camera group arranged in parallel to the X axis observes the objects arranged in the Y axis direction. The camera group arranged in parallel with the Y axis observes an object arranged in the X axis. That is, the position of the target is determined by using a group of cameras arranged in a direction orthogonal to the longitudinal direction of the target.

The same can be done with a row of cameras. First, an object is imaged by a camera group arranged in a horizontal direction, and then the same object is imaged as a camera group arranged in a vertical direction by rotating the camera group by 90 degrees. This can do substantially the same thing as a camera group along two orthogonal axes.

The surroundings of the heavy equipment (crane vehicle) are imaged by the camera 20. Image data is obtained for each camera. This is a set of vertical and horizontal pixel data. It is analog data.
The image data is sent to the image measuring unit 21. The image measurement unit 21 performs A / D conversion of the image data and extracts feature points. The characteristic point can be defined by the gradation of the density. Actually, since the object has a continuous outline, the outline is a set of feature points. Data captured by each camera for each feature point is stored in an image memory. Do the same for each camera. The three-dimensional coordinates of the feature point are determined using data of the same feature point between different cameras. The coordinates can be obtained as coordinates based on the entire monitoring device.

The image measuring section 21 supplies image data to the image output section 22. The image output section 22 outputs image data to the man-machine interface section 27. The access restriction area updating unit 23 receives the image data and sets an access restriction area around a transmission line or a steel tower. For example, in the case of a transmission line, a cylindrical space having a fixed distance is set as the access restriction area. The approach monitoring operation unit 25 includes a crane car, a transmission line,
The distance measurement data of the tower is received from the image measurement unit 21.

When a crane vehicle enters an access restriction area such as a transmission line, alarm data is generated. The alarm data is output from the alarm data output unit 26 to the alarm device 29. As already described, the alarm device comprises an alarm device base station on the monitoring device side and an alarm device mobile station located near the crane car. The system control unit 24 includes an image measurement unit 21, an access restriction area update unit 23, an approach monitoring operation unit 2,
5. Control the alarm data output unit 26.

Next, more specific examples of the conditions will be described. Camera: a camera unit equipped with four small CCD cameras having a resolution of 410,000 pixels. These are arranged vertically and horizontally. A total of eight cameras are used.

A / D converter: converts an image from the camera into 8-bit 256-level luminance data.

Feature point extraction unit: During the monitoring operation, the luminance data
The data is subjected to a one-dimensional spatial differential filter to calculate the contrast. Those having a large value are output as feature points.
Processing is performed at a video rate using a dedicated DSP.

Image memory: Stores image data or feature point data.

Image measuring section: The operation differs depending on the initial setting and the approach monitoring. At the time of initial setting, the distance measuring unit measures the distance of the transmission line. At the time of approach monitoring, the distance measuring unit measures the three-dimensional coordinates of the heavy equipment. These processes are
The processing of each processing unit is performed in parallel.

Access-restricted area updating unit, approach monitoring operation unit: At the time of initial setting, a database of three-dimensional hazardous areas is created based on the three-dimensional coordinates of the transmission line obtained by the image measuring unit. During the approach monitoring operation, the three-dimensional coordinates of the object obtained from the image measurement unit and the coordinates of the dangerous area are compared to determine the degree of risk of the approach state.

System control section: for controlling the gantry turning of the camera, controlling the alarm device, etc. Manmashininfa-
Input / output data to / from the data section. At the time of initial setting, the image measurement unit,
Inputs and outputs setting values to the image output unit. At the time of the approach monitoring operation, the gantry control and the control of the alarm device are performed. For example, a 32-bit personal computer can be used.

Alarm device: An alarm device base station, an alarm device mobile station, and an alarm issuing device. The operation of the above configuration will be described.

[Operation at Initial Setting] The camera is installed in the monitoring direction. Start the system. Adjust the optical axis of the camera for distance measurement. The access limit distance is determined based on the voltage of the transmission line, and is input. On the monitor screen, a part of the transmission line is designated for each camera using a cursor or the like.

The image processing section extracts feature points from the designated area on each image. Based on these, transmission lines are extracted from the entire image. Next, correspondence is made with the extracted transmission line images of the left and right cameras. Then, parallax between different screens is obtained. From this, the three-dimensional coordinates of the transmission line are calculated.

The space near the transmission line is divided into small areas, and the degree of danger is determined for each area based on the input danger area. This is created as a dangerous area database.

[Approach Monitoring Operation] The feature points of the object on the image of the camera for distance measurement are extracted. The correspondence between these feature points is performed. When the corresponding point is known, the three-dimensional coordinates are calculated. The three-dimensional coordinates of the measured object are converted to the coordinate system (reference coordinate system) used when creating the dangerous area based on the turning angle of the gantry. Then, it is checked whether or not the object is in the dangerous area by comparing with the database of the dangerous area. If the object is in the danger area, an alarm is issued according to the degree of danger.

[0077]

The object is observed by two orthogonal camera groups, and the same feature point is determined to determine three-dimensional coordinates. When the target object is long horizontally, a feature point is determined using a vertical camera group, and distance measurement can be accurately performed. When the target object is vertically long, the distance measurement can be accurately performed using the horizontal camera group. The three-dimensional position is determined more accurately. Since the setting of the danger area can be performed three-dimensionally, it is extremely effective in improving the safety of the operation of the crane car or the like near the power transmission line or the steel tower.

[Brief description of the drawings]

FIG. 1 is a perspective view of an image pickup system in which three or more camera groups are arranged vertically and horizontally, showing the principle of the present invention.

FIG. 2 is a schematic diagram showing the position of an image when the same object is observed by a plurality of cameras arranged on a straight line.

FIG. 3 is a perspective view showing that a danger area is set around the power transmission line when the crane vehicle works near the power transmission line.

FIG. 4 is a block diagram showing a process of performing image processing on data captured by a camera.

FIG. 5 is a plan view showing an example of a periphery when a crane vehicle is working near a power transmission line.

FIG. 6 is a diagram illustrating a structure of a memory that stores an image captured by a camera after the image is A / D converted and then stored in the memory.

FIG. 7 is an example of a screen obtained by imaging the arrangement example of FIG. 5 with a camera.

FIG. 8 is a perspective view showing a state in which a distance is measured by a group of cameras arranged horizontally for an object extending vertically.

FIG. 9 is a perspective view showing a state where distance measurement is performed by a camera group arranged vertically in the case of an object extending horizontally.

FIG. 10 is a plan view showing an example in which the range of distance measurement is limited by the distance from a camera group.

FIG. 11 is a diagram illustrating an example after image processing by a camera when a measurement range is limited by a distance.

FIG. 12 illustrates a camera group, a processing device, and a camera according to an embodiment of the present invention.
FIG. 2 is a perspective view showing an alarm device and the like.

FIG. 13 is a configuration diagram showing functions of the device of the present invention.

[Explanation of symbols]

 Reference Signs List 1 camera 2 camera 3 camera 4 camera 5 camera 6 gantry 8 pan head 9 tripod 10 main unit 11 power supply unit 12 alarm device base station 13 alarm device mobile station 14 alarm alarm device

Continued on the front page (72) Inventor Takehiko Kikuchi Tokyo Electric Power Co., Inc. 1-3-1, Uchisaiwaicho, Chiyoda-ku, Tokyo Tokyo (72) Inventor Takeshi Ishibashi 1-3-1 Uchisaiwaicho, Chiyoda-ku, Tokyo Tokyo Electric Power Stock In-company (56) References JP-A-4-307309 (JP, A) JP-A-5-45136 (JP, A) JP-A-2-107904 (JP, A) (58) Fields studied (Int. . ^7, DB name) G01B 11/00 - 11/30 102 G01C 3/00 - 3/32 G06T 1/00 G06T 7/00 G08B 25/00

Claims (2)

(57) [Claims] 1. A group of three or more cameras arranged on a straight line and having optical axes parallel to each other, a group of three or more cameras arranged on a straight line substantially orthogonal to the straight line and having optical axes parallel to each other, An A / D conversion unit for performing A / D conversion of image data captured by a camera for each pixel; a feature point extraction unit for extracting a feature point based on a change in gradation of a pixel; and an image memory for storing data for each pixel And an image measurement processing unit that identifies feature points between different cameras and calculates the three-dimensional coordinates of the feature points in a three-dimensional coordinate system defined by the camera. Set the original dangerous area, calculate the coordinates of the target heavy equipment and determine whether it is in the three-dimensional dangerous area, and the dangerous equipment is too close to the heavy equipment from the data of the risk determination part Alarm device that generates an alarm to notify An image monitoring device comprising: 2. A group of three or more cameras A arranged on a straight line and having optical axes parallel to each other, and a group of three or more cameras B arranged on a straight line substantially orthogonal to the straight line and having optical axes parallel to each other. And the image data captured by the camera is converted into A /
An A / D conversion unit for D-conversion, a feature point extraction unit for extracting a feature point based on a change in pixel gradation, an image memory for storing data for each pixel, and identifying feature points between different cameras;
An image measurement processing unit that calculates the three-dimensional coordinates of feature points in the three-dimensional coordinate system defined by the camera, and a three-dimensional hazardous area set near dangerous objects such as power lines and steel towers, Includes a danger determination unit that calculates coordinates to determine whether the vehicle is in a three-dimensional danger area, and an alarm device that generates an alarm that notifies that heavy equipment and dangerous goods are too close from each other based on data from the danger determination unit. In the image monitoring device, the camera group A monitors a monitoring target that is arranged substantially at right angles to the straight line of the camera group A, and the camera group B detects a monitoring target that is arranged at substantially right angle to the straight line of the camera group B. A method of using an image monitoring apparatus, wherein the method is to monitor.