JP3569627B2 - Image interpretation device - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention displays a symbol or the like on a map by combining a disaster point identified on an image obtained from a television camera mounted on an aircraft such as a helicopter with position information obtained from a GPS mounted on the map and displaying a symbol or the like on the map. The present invention relates to a system for interpreting a position on an image for specifying a point.
[0002]
[Prior art]
In the event of a disaster such as a fire or a traffic accident, it is indispensable to grasp where these events occurred on a map in order to manage disasters comprehensively, for example, take rescue measures. is there.
The initial information about the disaster location is often provided by a helicopter-mounted TV camera, so it is necessary to quickly identify the location from this image, which is a matter of seconds. .
[0003]
As an example of a disaster, JP-A-5-46887 discloses a system for automatically monitoring or rediscovering a fire. However, since the camera is installed on the ground, the monitoring range is limited.
Japanese Patent Application Laid-Open No. 8-324499 discloses a fire extinguisher dropping device for reliably dropping a fire extinguisher at a fire position confirmed by an infrared camera mounted on a helicopter.
Japanese Patent Application Laid-Open No. 4-3638989 discloses a technique for transmitting a signal from a video camera on a helicopter to the ground.
A description will be given of a conventional helicopter-mounted disaster spot identification system that combines these known technologies within a range that can be easily combined.
[0004]
FIG. 11 is a diagram showing a basic configuration of a position interpretation system for specifying an occurrence point when a disaster occurs. In the figure, 1 is a helicopter equipped with a camera 21 and a GPS 22 for photographing the ground, 2 is a moving image 32 sent from the helicopter 1 by wireless transmission 25, and a still image 34 containing positional information every second (these information is shown). , A device for storing (storing) the information 20 transmitted from the receiving device 2, a display of a moving image and a still image, and a display of the helicopter 1 that has captured them. This is a damage information reading device that displays a position on a map and inputs a disaster occurrence position by looking at an image by a person. The devices after the data receiving device 2 are installed on the ground.
FIG. 12 is a diagram for explaining the operation of the device shown in FIG.
In the figure, reference numeral 32 denotes a moving image taken by the television camera 21 (for example, the upper side is in front of the helicopter), 34 denotes a still image, 34A denotes position data, and 35 denotes a map obtained from CDROM data (not shown) based on the position data 34A. In the image, this is a northward view. Reference numeral 13 denotes a fire as an example of a disaster occurrence for the sake of explanation.
[0005]
Next, the operation of the conventional system will be described. When the helicopter 1 flies, information 20 including a moving image 32 captured by a camera 21 mounted on the helicopter 1 and a still image 34 at one-second intervals including positional information from the GPS 22 is stored in the data receiving device 2 and the data storage device. It is sent to the position reading system 4 via the device 3. The position interpretation system 4 displays the received moving image 32 on the screen in real time, and at the same time, displays a map 35 based on the position information of the helicopter 1 added to the still image 34 and displays the map 35 on the map 35. A symbol representing the current position of the helicopter 1 is displayed. The user checks the still image 34 after watching the moving image 32, or remembers the damage occurrence point discovered while viewing the moving image 32 based on the surrounding scenery and the like. According to the method, the damage occurrence point is registered as a symbol 53 by a mouse cursor or the like (not shown) while comparing the still image of the called damage occurrence point with a map around the same point.
[0006]
In the above-mentioned conventional example, since the work is performed by one person, it is not possible to input the damage information while receiving and displaying the moving image 32 and viewing it. Since the moving image 32 and the map 35 do not have the same orientation and scale, finding the position on the moving image on the map is not an easy task if you are not used to it.
[0007]
[Problems to be solved by the invention]
The conventional helicopter and TV position interpretation system using images are configured and operating as described above.To register the damage position on the map, the information sent from the helicopter must be completely transmitted. Either wait until all data has been received, or use a data storage device (video playback device) to pause each time.
If a spot is found on a moving image that is received and displayed in real time during the flight of the helicopter, the user needs to remember to some extent the location on the map. Therefore, in order to locate the damage point on the map at the stage of damage location registration after receiving all the information, it is necessary to sequentially check the still images near the place memorized when watching the moving image In addition, it is necessary to check more than necessary still images, and it takes a long time to perform the operation, and there is a problem that fatigue is intense and a check may be omitted.
[0008]
In the event of a disaster, it is important to quickly understand and respond to the situation, and these issues cannot be overlooked in that the real-time nature of the information is impaired and that information leakage may occur. .
[0009]
The present invention has been made in order to solve the above-described problems, and it is possible to read and register a damage point in real time while watching a video (moving image) from a helicopter camera, and to obtain a position where a situation can be quickly grasped. The aim is to obtain a reading system.
[0010]
[Means for Solving the Problems]
A position reading device on an image according to the present invention is mounted on an aircraft and constantly outputs a television image information by photographing a vertically downward position, and an infrared television camera mounted on the aircraft and synchronizing with the imaging of the infrared television camera. GPS that outputs position information including altitude and direction of the vehicle, receiving the TV image information and the position information, displaying the TV image, and determining the current position from map data stored in advance based on the position information. A position reading device on an image having image display means for selecting map data including and displaying a map image , wherein the high-temperature position automatic device identifies a high-temperature portion on an infrared image and designates and inputs the high-temperature position. designating means, the image frame on the map image based on the angle of view of the television camera are calculated, the with the specified input hot position on the map image, the image frame Those having a picture frame calculation means for displaying in real time.
[0011]
Also, a television camera mounted on the aircraft and constantly shooting vertically downward to output television image information, and a positional information including the altitude and orientation of the aircraft output in synchronization with the imaging of the television camera mounted on the aircraft. Receiving the GPS, the television image information and the position information, displaying the television image, selecting map data including the current position from map data stored in advance based on the position information, and displaying the map image And an image display means for performing position reading on an image, comprising: a position input means for designating an arbitrary point on the television image; and an image frame on the map image based on an angle of view of the television camera. Calculated based on the altitude data of the camera obtained by the radar altimeter mounted on the aircraft or the focal length meter of the camera, and the calculated data is input on the map image. With any point was, and has a picture frame calculation means for displaying the image frame in real time.
[0012]
Below, the position read system on an image that is a basis of the present invention, by using the block diagram in FIG. 1, also be described with reference to diagram showing an image in Figure 2.
In the following drawings, parts that are the same as or correspond to those in the related art are denoted by the same reference numerals, and detailed description thereof is omitted.
[0013]
In FIG. 1, reference numeral 1 denotes a helicopter equipped with a television camera 21 (hereinafter referred to as a camera) for photographing the ground and outputting television image information, and a GPS 22. Reference numeral 2 denotes a moving image 32 transmitted from the helicopter 1 by wireless transmission 25; , A receiving device for receiving the positional information (information 20) of the GPS 22, a device for storing the information 20 sent from the receiving device 2, a moving image 41 and a still image (moving image is temporarily stopped) according to the present invention. ) And a damage information reading device for displaying the position of the helicopter 1 where the images were taken on the map, and inputting information on the damage occurrence position, 47 is an image display means, and 43 is an image frame. Details of the calculation means will be described later.
Reference numeral 48 denotes map data (CDROM), and reference numeral 49 denotes position input means, which represents main contents of work performed by an operator (human).
[0014]
Here, the camera 21 is installed in a gimbal device having a built-in gyro so that the camera 21 can always photograph vertically below regardless of the attitude and acceleration of the helicopter 1.
However, the upper side of the captured screen always faces the front of the helicopter 1. The GPS 22 synchronizes and outputs position information (longitude and latitude) including the altitude and direction of the helicopter 1 for each screen frame captured by the camera 21.
[0015]
FIG. 2 is a diagram for explaining the operation of FIG.
In the figure, 32 is a moving image, 32A is corresponding position data, and 35 is a map image obtained from map data (CDROM data) 48 based on the position data 32A.
Reference numeral 12 denotes an auxiliary explanation line indicating the photographing range (angle) of the camera, and reference numeral 13 denotes a fire for explanation as an example of a disaster occurrence. Reference numeral 32 denotes an image of a moving image captured by the camera 21 of the helicopter 1 displayed on a screen, and an arrow 42 denotes a mouse cursor. Reference numeral 33 denotes the x-coordinate and y-coordinate in the image frame when the fire place is indicated (clicked) with the mouse cursor 42 by dotted lines, and indicates the position in the image frame for easy understanding for convenience of explanation. is there.
[0016]
35 is a map in which the position of the helicopter 1 is centered and north is displayed at the top, 51 is a projection of the image frame of the 33 television screen on the map 35, and 52 is a coordinate line of 33 in the image frame 51. The projection 53 is a symbol indicating the fire position. In the following description, it is assumed that the camera 21 always faces vertically downward regardless of the attitude and acceleration of the helicopter 1.
[0017]
Next, the operation will be described.
First, when the helicopter 1 flies, a moving image 32 of the ground surface captured by the camera 21 is displayed on the screen in real time. When a damaged part is found on the moving image 32, the user clicks the portion 13 on the moving image 32 with the mouse 42. The clicked point corresponds to the position 31 in the image frame 33. The map 35 is updated and displayed every moment based on the position data 32A added to the moving image 32. Then, the image frame 51 of the camera 21 is displayed on the map 35 by calculating from the latitude / longitude, altitude, direction, and angle of view of the helicopter 1.
[0018]
With reference to FIG. 3, a method of projecting an image frame on a map from the latitude / longitude, altitude, orientation, and angle of view θ of the helicopter 1 will be described. 3, reference numeral 21 denotes a camera mounted on the helicopter 1, reference numeral 51 denotes a frame of the camera, reference numeral 133 denotes a center of the frame 51 (the center of the image), and reference numeral 134 denotes a vertex of the frame 51, which is right forward (45) in the traveling direction of the helicopter 1. °), 135 indicates the forward direction of the helicopter, and 136 indicates the north direction (the direction that serves as a reference in managing the map).
[0019]
In order to project the image frame 51 on the map 35, it is necessary to know the positions of the four vertices of the image frame 51. The center coordinates of the image frame 51 are (0, 0), the + direction of the y-axis is north, and the coordinates of the vertex 134 of the image frame are (x1, y1).
First, assuming that the distance from the center 133 of the image frame to the vertex 134 is L, L is calculated using the altitude h of the helicopter and the angle of view θ of the camera.
L = h · tan θ
It can be expressed as Next, when x1 and y1 are expressed using this L and the traveling direction (azimuth) θ2 of the helicopter 1 measured with the north taken as 0 °,
x1 = L · sin (θ2 + π / 4)
y1 = −L · cos (θ2 + π / 4)
It becomes. Similarly, by calculating the other three vertices, the image frame 51 can be projected on the map 35. The above series of operations is executed by the image frame calculation means 43 of FIG.
[0020]
Here, since the position of the image frame of the moving image 32 coincides with the position of the image frame 51 on the map 35, a damage point (point 31 in FIG. 2) on the map 35 is calculated based on these comparisons. A symbol 53 indicating a damaged part can be displayed.
As described above, the correspondence between the points on the moving image and the points on the map is calculated in real time from the image of the camera 21 mounted on the helicopter and the position information from the GPS 22 so that the moving image data can be viewed during the flight of the helicopter. Damage points can be checked in real time. As a result, the time required to specify a damage point and register damage information on a map is greatly reduced.
In performing the above operation, the operator on the ground does not need to look at the map 35 at all, and can perform the operation only by watching the moving image 32.
[0021]
In the above description, the altitude h (basically, altitude above sea level) obtained by the GPS 22 is used as it is. However, it is more accurate to calculate by reducing the ground surface height (H) included in the map information. Needless to say,
[0022]
In the above description of the embodiment , the operator must keep an eye on the ever-changing moving image 32, although it is possible to work in real time, and he / she cannot be overlooked, so the work may be painful.
Therefore, in order to improve this point, the operator performs a specific key input 62 as shown in FIG. 4 to temporarily stop the moving image 32 (stop data output from the data storage device 3) and confirm the disaster. Also, the position input can be facilitated. The restart of the moving image may be performed by key input again, or may be automatically started after a predetermined time.
[0023]
Further, instead of the key input 62, the operator may stop at intervals of several seconds, for example, and the operator may work while viewing only the still image 34.
Alternatively, only when the mark 42 is added to the moving image 32, the still image 34 may be obtained for confirmation.
[0024]
BEST MODE FOR CARRYING OUT THE INVENTION
Embodiment 1 FIG.
Hereinafter, embodiments of the present invention will be described. In the above embodiments , the user (operator) needs to click the damage point with the mouse while viewing the moving image 32 while viewing the screen, which is a manned operation required by the operator. According to the present invention , when a disaster is a fire, the infrared camera 61 is mounted on the helicopter 1 and the temperature of the ground surface is monitored using the image of the fire. by automatically determining that it is obtained by such that unmanned automatically performs position input.
[0025]
FIG. 5 shows the configuration of the first embodiment. Reference numeral 1 denotes a helicopter equipped with an infrared camera 61 and a GPS 22. Reference numeral 44 automatically extracts a high temperature (that is, a point where a fire has occurred) from image information of the infrared camera 61, and displays position data on the screen as an image frame. This is a high-temperature position automatic designation means to be transmitted to the calculation means 43.
[0026]
FIG. 6 is an image diagram for explaining the operation, in which 12 is a line representing the shooting range (angle) of the camera, and 13 is a building where a fire has started. Reference numeral 36 denotes a video image captured by the infrared camera 61. Reference numeral 60 denotes a portion where a high heat like a fire is confirmed, 62 denotes an x coordinate line passing through 60, and 63 denotes a y coordinate line passing through 60. 35 is a map centered on the position of the helicopter 1, 51 is an image frame, 52 is a coordinate line corresponding to a coordinate line 63, and 53 is a symbol indicating a fire position.
[0027]
Next, the operation will be described. When the helicopter 1 flies, an infrared moving image 36 of the ground surface taken by the infrared camera 61 is displayed on the screen in real time, and at the same time, the screen temperature of the infrared image 36 is checked by the high-temperature position specifying means 44. If a portion 60 having a temperature higher than a certain temperature is found in the screen, it is determined that a damage has occurred, and the coordinate value thereof is extracted. The rest is projected onto the map 35 as in FIGS. 1 and 2, and a symbol is drawn at the point of damage.
As described above, even if the user does not give any particular instruction, the check of the damage point and the registration of the symbol are automatically performed, so that it is possible to achieve unmanned operation (at least a reduction in the burden on the operator).
[0028]
Embodiment 2 FIG.
In the first embodiment, the altitude of the helicopter 1 is described as using the altitude obtained from the GPS as it is, but the altimeter or altitude radar mounted on the helicopter, or the focal length of the camera (all shown in the figure). Can be used to determine the altitude of the helicopter above the ground. As a result, the position of the image frame can be calculated more accurately, and as a result, the position on the map at the position clicked with the mouse on the moving image can be more accurately obtained.
[0029]
FIG. 7 shows the configuration of the second embodiment. In FIG. 7, altitude correction is performed when the image frame of the still image 34 is projected on the map 35. Specifically, as shown in FIG. 8, the measured altitude 72 of the altitude radar, that is, the actual distance from the camera to the ground surface is used instead of the sea level altitude 71 obtained from the GPS 22. By setting this altitude as h 'and applying the altitude to the above formula, the position of the image frame corresponding to the coordinate system on the map can be calculated more accurately.
[0030]
The altitude information obtained from the GPS 22 is the altitude above sea level. The altitude above the ground, which is captured by a camera, is not negligibly lower than the altitude of a helicopter flying during a disaster. In this way, by using the altitude measurement value instead of the information of the GPS 22 as the altitude of the helicopter, it is possible to calculate the damage point more accurately.
[0031]
Although the above description is intended for advanced correction of the position reading device on the images, it can be performed more accurately automatic determination of the damage portion by performing the same altitude correction to those of the first embodiment It becomes.
[0032]
That is, the correction is performed when the image frame of the infrared image 36 is projected on the map 35 in FIG. Specifically, as shown in FIG. 8, instead of the altitude 71 obtained from the GPS 22, the altitude radar measurement 72, that is, the actual distance from the camera to the ground surface is used. By applying this altitude as h ′ and applying the above-described formula to the above formula, the position of the image frame 51 corresponding to the coordinate system on the map can be calculated more accurately.
[0033]
By using the infrared image and the altitude radar information in this way, it is possible to automatically and more accurately identify a damage point and register information on a map without bothering the user. .
[0034]
Embodiment 3 FIG.
In the implementation of Embodiment 1, 2, although of differences performed automatically or by hand to check the damage point it is, is performed automatically in the ratio calculation of all image frame 51 with respect to registration of the symbol, Conventionally, a method of directly marking on a map by human judgment (for example, it is possible to distinguish between a fire and a simple bonfire) is also excellent in reliability. Therefore, if the user registers a symbol by a method as shown in FIG. 10, a reliable marking similar to the conventional one can be more easily performed, and the symbol can be placed at an accurate position.
[0035]
FIG. 10 shows the configuration of the third embodiment, and its operation will be described.
When the helicopter 1 flies, a moving image 2 of the ground surface captured by the camera 21 is displayed on the screen in real time. Here, when a damage point is found on the screen, an arbitrary point on the screen is clicked with a mouse (or a predetermined appropriate key input is performed), and the still image at that time is taken as a snapshot 74 by the GPS. Stored with the data. After the flight or shooting of the helicopter 1 is completed, the snapshot 74 and the GPS data saved up to that time are called up sequentially and the map 35 at the corresponding position is compared and compared, and the symbol 53 is displayed at the portion corresponding to the damage point on the map 35. Register and go.
[0036]
In this way, when a damaged point is found, instead of checking that point, the nearby still image itself is captured, so it is possible to deal with cases where the damaged point is scattered in a small area, etc. Since the image and the map are not random in terms of orientation and scale as in the case of, the work becomes easier, and the reliability of the registration position of the symbol is improved by leaving the symbol registration part to human judgment. And the likelihood of registration errors being reduced.
[0037]
Here, the technique described in the first embodiment is used, that is, the infrared camera 61 is used instead of the camera 21, and the symbol registration operation can be automated assuming that the snapshot 74 is an infrared image. . Further, it is needless to say that the image position can be corrected more accurately by correcting the altitude information obtained by the GPS 22 with the map information or the altimeter mounted on the helicopter 1.
[0038]
In the above description, a helicopter is used as an aircraft, but the same effect can be obtained with an airplane, an airship, and a balloon. It is sufficient if a camera or the like can be loaded as well as a model as well as an aircraft. Particularly, in the case of a helicopter, an airship, or a balloon capable of flying at a low speed or stopping, a more stable position can be read.
Further, all the parts described as the ground equipment may be provided on the machine.
Obviously, electronic still cameras can be used instead of television cameras.
[0039]
【The invention's effect】
As described above, according to the present invention, it is not necessary for the user to wait until the helicopter has finished flying when identifying a disaster position and registering a symbol on a map at the time of occurrence of a disaster, so that the position can be determined quickly. In addition to being able to do so, operator fatigue is reduced, information oversight is reduced, and information leakage is less likely to occur.
[0040]
In addition, when an infrared camera is used, an effect is obtained that a fire can be found and its position can be specified completely unattended.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a position reading device on an image on which the present invention is based .
FIG. 2 is a diagram for explaining the operation of FIG. 1;
FIG. 3 is an explanatory diagram of a method of calculating an angle of view in FIG. 1;
FIG. 4 is an explanatory diagram illustrating an operation of a position reading device on an image according to another embodiment .
FIG. 5 is a configuration diagram of an apparatus for reading a position on an image according to the first embodiment of the present invention ;
FIG. 6 is an operation explanatory diagram of FIG. 5;
FIG. 7 is an explanatory diagram of an operation of the apparatus for reading a position on an image according to the second embodiment.
FIG. 8 is an explanatory diagram of an altitude correction calculation.
FIG. 9 is an application example obtained by modifying FIG. 7 of the second embodiment.
FIG. 10 is an explanatory diagram of an operation of the position reading device on an image according to the third embodiment.
FIG. 11 is a configuration diagram of a conventional position reading device that specifies a disaster point.
FIG. 12 is a diagram for explaining the operation of FIG. 11;
[Explanation of symbols]
1 aircraft (helicopter), 21 TV camera,
22 GPS, 32 moving images,
32A, 34A position information, 34 still images,
35 map images, 43 image frame calculation means,
44 high temperature position automatic designation means, 47 image display means,
49 position input means, 51 picture frame,
θ angle of view, 61 infrared camera,

Claims (2)

An infrared television camera mounted on an aircraft that constantly shoots vertically below and outputs television image information;
A GPS mounted on the aircraft and outputting position information including the altitude and orientation of the aircraft in synchronization with shooting by the infrared television camera;
Receiving the television image information and the position information, displaying the television image, selecting map data including a current position from map data stored in advance based on the position information, and displaying a map image. A position reading device on an image having image display means,
A high-temperature position automatic specifying means for identifying a high-temperature portion on the infrared image and specifying and inputting the high-temperature position;
Image frame calculating means for calculating an image frame on the map image based on the angle of view of the television camera, and displaying the image frame in real time together with the designated and input high-temperature position on the map image. Characteristic position reading device on the image. A TV camera mounted on an aircraft that always shoots vertically below and outputs TV image information;
A GPS mounted on the aircraft and outputting position information including the altitude and orientation of the aircraft in synchronization with shooting by the television camera;
Receiving the television image information and the position information, displaying the television image, selecting map data including a current position from map data stored in advance based on the position information, and displaying a map image. A position reading device on an image having image display means,
Position input means for specifying an arbitrary point on the television image,
An image frame on the map image based on the angle of view of the television camera is calculated based on the altitude data of the camera obtained by a radar altimeter mounted on the aircraft or a focal length meter of the camera, and is calculated on the map image. And an image frame calculating means for displaying the image frame in real time together with the input arbitrary point .

Images