JPH1042282A - Video presentation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system for superimposing accurate map information for photographed video images to the photographed video images and presenting them. SOLUTION: The photographed video images are obtained by a camera 31 loaded to a helicopter 30. Photographing conditions including the three- dimensional position, three-dimensional posture and viewing angle of the camera 31 are detected by a sensor 32. Video data obtained by the camera 31 and the respective kinds of the photographing conditions detected by the sensor 32 are transmitted to the ground and received by the receiver 34 of a ground station 33. In a CG (computer graphics) unit 35, threedimensional map data stored beforehand are perspectively transformed based on the photographing conditions and CG map image data are obtained. The images of a form in which the map information is superimposed on the actually photographed video images obtained by the camera 31 are displayed at a display 36. Also, for a traveling automobile 37 to which a CPS with a builtin communication function is attached, the automobile 37 is visually recognized and tracked in the actually photographed video images on which the map information is superimposed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video presenting system for presenting a photographed video, and more particularly to a video presenting system for presenting a photographed video by superimposing map information of the photographed place.

[0002]

2. Description of the Related Art Television (TV) reports on the Great Hanshin-Awaji Earthquake that caused enormous damage have caused aerial photography by cameras mounted on flying objects such as light airplanes and helicopters due to the cutting of land routes. And the reporter of the pilot of the vehicle or the broadcaster on board the vehicle,
Broadcast programs where announcers conveyed the situation at the site were mainstream. And many viewers watched such a relay program broadcast in real time with great interest.

[0003] However, in such a series of aerial photography reports, information for specifying a photographing place such as a place name and a direction has often been lacking and incorrect. For example, a reporter or announcer who relays, with the advice of a helicopter pilot or a staff member of the station, hastily corrects information such as previously reported place names and directions, especially in nighttime aerial photography reports. Was seen.

[0004]

The causes of such a phenomenon include a change in the cityscape due to the earthquake, a fire caused by a fire, or a building that serves as a landmark due to a nighttime power outage. Various bad conditions can be considered, such as that the confirmation could not be performed, and that the live broadcaster was unfamiliar with aerial photography. However, whatever the cause, in the early stage after an earthquake where emergency situations such as escape and rescue are concentrated, even such a false alarm flows through the public radio wave even for a moment, because of the influence of the TV media. It is undoubtedly a serious problem that can affect many lives, even from the perspective of power.

[0005] The problem is that one of the important missions of TV reporting is the immediacy of the information, and as long as the judgment of the location information in the on-site relay is left to the interpretation of a small number of relay staff, Hanshin -It can be said that it exists universally without being limited to unprecedented disasters such as the Great Awaji Earthquake.

By the way, GPS (Global Positioning S)
ystem) is used for navigation of the object, navigation of land and sea, etc., and it is possible to easily obtain accurate real-time position information of the object and track the trajectory of the object on the map screen. However, information for visually recognizing the surrounding situation in real time is not provided, and there is a problem that information is insufficient in an emergency or the like.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an image presentation system capable of preventing a false alarm at the time of relay using a photographed image by displaying map information superimposed on the photographed image in real time. The purpose is to provide.

It is another object of the present invention to provide a video presentation system capable of displaying accurate position information on a captured video superimposed on the captured video.

Still another object of the present invention is to provide a video presenting system which can easily search and track a target by superimposing map information on a photographed video of a moving target. .

[0010]

According to a first aspect of the present invention, there is provided an image display system for detecting a three-dimensional position, a three-dimensional posture, and an angle of view of a camera to be photographed in a system for presenting an image photographed by a camera. And storage means for storing three-dimensional map data, and perspective conversion of the three-dimensional map data stored in the storage means based on the detection result of the detection means, and map data corresponding to the photographed image. And a means for displaying the acquired map data together with the image captured by the camera.

According to a second aspect of the present invention, there is provided an image display system, comprising: a camera provided on the flying object; detecting means provided on the flying object for detecting a three-dimensional position, a three-dimensional attitude and an angle of view of the camera; Storage means provided on the body for storing three-dimensional map data; and three-dimensional map data provided on the flying body and stored in the storage means based on the detection result of the detection means. And means for acquiring map data according to the image captured by the camera, and means provided on the flying object and displaying the acquired map data together with the image captured by the camera. .

According to a third aspect of the present invention, there is provided an image display system, comprising: a camera provided in the flying object; a detecting means provided in the flying object to detect a three-dimensional position, a three-dimensional attitude, and an angle of view of the camera; Transmitting means provided on the body for transmitting the data of the image captured by the camera and the data of the detection result of the detecting means to the ground, receiving means for receiving the transmitted data from the transmitting means, and three-dimensional map data , And perspectively transforms the three-dimensional map data stored in the storage means based on the detection result of the detection means to obtain map data corresponding to a video taken by the camera. Means, and means for displaying the acquired map data together with the video taken by the camera.

A video display system according to a fourth aspect of the present invention stores a camera rotatably provided at a fixed position, detection means for detecting a three-dimensional attitude and an angle of view of the camera, and three-dimensional map data. A storage unit, and a unit for performing a perspective transformation of the three-dimensional map data stored in the storage unit based on the detection result of the detection unit to obtain map data corresponding to a video image captured by the camera. Means for displaying the map data together with the video taken by the camera.

According to a fifth aspect of the present invention, there is provided an image display system for presenting an image captured by a camera including a moving object, wherein the position identification is performed on the object to identify a three-dimensional position of the object. Means, transmitting means for transmitting the obtained position identification information, detecting means for detecting the three-dimensional position, three-dimensional attitude and angle of view of the camera for photographing, and storage means for storing three-dimensional map data Means for perspective-transforming the three-dimensional map data stored in the storage means based on the detection result of the detection means to obtain map data corresponding to the photographed video; and obtaining the obtained map data by the camera. Means for displaying the image together with the photographed image including the object.

A video display system according to a sixth aspect is characterized in that, in any one of the first, fourth and fifth aspects, the camera is a camera that performs aerial photography.

According to a seventh aspect of the present invention, in the video display system according to any one of the first to fourth aspects, the camera is a camera that performs underwater photography.

An image display system according to an eighth aspect of the present invention, according to any one of the first to seventh aspects, further comprises means for recording data of a detection result of the detection means and data of an image captured by the camera. Features.

In the video presenting system of the present invention, the photographing conditions of the camera are detected by the detection means, and the corresponding three-dimensional map data is perspective-transformed based on the detection result to obtain map data for display. When the photographed video is displayed, the map data is superimposed and displayed. Therefore, the information value of the camera image can be increased. In the present invention, map information such as a place name and a boundary of a municipal village, a building serving as a landmark,
It can be displayed in real time at the same time on the shot video,
For example, in a broadcast report at the time of occurrence of a disaster using a flying object, misinformation of the place name and direction, which has conventionally occurred, is prevented. In addition, the position information of the shooting site can be provided to the viewer at a glance.

The camera for photographing may be an aerial photographing camera mounted on a flying object such as a light aircraft, helicopter, balloon, etc., and moving with the flying object, or a building such as a high-rise building or a tower. The camera may be a fixed-point camera fixed to the camera or a waterproof camera provided underwater.

The detecting means is mounted on the flying object in addition to the camera, and the storing means, the obtaining means and the displaying means are installed on the ground, and the image data obtained by the camera and the detection result by the detecting means are transferred from the flying object to the ground. And the photographed video of the actual photograph on which the map information is superimposed may be viewed on the ground.
Alternatively, if all of the detection means, storage means, acquisition means, and display means are mounted on the flying object in addition to the camera, the photographed image of the actual photograph on which the map information is superimposed can be viewed inside the flying object. It is possible. Of course, it goes without saying that the storage means, the acquisition means, and the display means may be provided both in the flying body and on the ground.

Another image presenting system according to the present invention, in addition to the above-described detection means, storage means, acquisition means and display means, is a position identification which is attached to a moving object and identifies a three-dimensional position of the object. Means for transmitting the obtained position information. When the position information of the moving object is transmitted, the flying object searches for the object according to the position information, confirms its existence, and then tracks the object. In this way, it is possible to visually search, track, and confirm the position of a target object on a photographed image of a real image on which map information is superimposed.

Further, in these video presenting systems, there is further provided a means for recording video data captured by a camera and a detection result of the detection means. If the information of the detection result and the data of the photographed video are recorded and left, they can be used as information sources for various analysis processes later.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be specifically described below with reference to the drawings showing the embodiments.

FIG. 1 is a schematic diagram showing an outline of a video presenting system according to the present invention.
31 and the three-dimensional position of the camera 31 exhibiting unstable behavior.
GPS, gyro, to detect 3D attitude and angle of view
A sensor 32 including an angle of view encoder and the like is mounted.
While the helicopter 30 flies, the image data obtained by the camera 31 and various types of camera information detected by the sensor 32 are sequentially transmitted to the ground, and the receiver 34 of the ground station 33
Received at. Camera information is CG (Computer Graphics)
Sent to unit 35. In the CG unit 35, the three-dimensional map data stored in advance is perspective-transformed based on the camera information to obtain CG map image data. The obtained map image data is sent to the display 36. The received captured video data is also sent to the display 36. Then, the display 36 displays an image in a mode in which the map information is superimposed on the actual photographed video obtained by the camera 31.

The vehicle 37 running on the ground is equipped with, for example, a GPS incorporating a communication function.
The data of the three-dimensional position information of the automobile 37 is sent to the ground station 33. By rotating the camera 31 mounted on the helicopter 30,
If there is a car 37 in the shooting area, the display 36
Displays the position of the car 37. So the car 37
The vehicle 37 can be visually recognized in the photographed image on which the map information is superimposed, and the vehicle 37 can be tracked.

Next, a specific example of the video presentation system of the present invention will be described in detail. (First Embodiment) FIG. 2 is a schematic diagram showing a configuration example of a video presentation system (first embodiment) of the present invention.
The components in the video presentation system of the present invention are classified into members mounted on a flying object such as a light aircraft, helicopter, balloon, and the like, and members provided on a terrestrial broadcasting station.

The flying object includes a camera 1, a gyro sensor 2 for detecting a three-dimensional attitude of the camera 1, an encoder 3 for detecting an angle of view of the camera 1, and a signal for converting a detection signal of the gyro sensor 2 into attitude data. A converter 4, a signal converter 5 for converting a detection signal of the angle-of-view detection encoder 3 into angle-of-view value data, a GPS device 6 for detecting a position of the camera 1, a GPS
A receiving antenna 7, a GPS error correcting device 8, a control computer 9, a monitor TV 10, a data encoder 11, and a transmitting device 12 are mounted. On the other hand, a terrestrial broadcasting station is provided with a receiving device 21, a data decoder 22, a host computer 23, an internal or external storage medium 24, a recording device 25, and a video editing device 26.

The camera 1 is a general TV camera or a broadcast video camera, and sends a video signal obtained by aerial photography to a monitor TV 10 and a data encoder 11.

A gyro sensor 2 for detecting the three-dimensional attitude of the camera 1 is attached to the camera 1. The gyro sensor 2 includes a head angle (a direction east, west, north and south of the camera 1) and a pitch angle (a camera angle). 1 tilt up and down),
The bank angle (the inclination of the camera 1 in the left-right direction) is detected, and the detection signal is output to the signal converter 4. Signal converter 4
Indicates the detection signal as HPB data (H (Head): direction data of camera 1, P (Pitch): tilt data of camera 1 in the vertical direction, B (Bank): tilt data of camera 1 in the horizontal direction)
And outputs it to the control computer 9. As such a gyro device for detecting the attitude of the camera 1, for example, a gyro mini dt manufactured by Datatec, Trimble
Japan's TANS Vector ^™ can be used.

An encoder 3 for angle of view detection is attached to a zoom ring attached to the lens of the camera 1, detects the rotation angle of the zoom lens of the camera 1, and outputs a detection signal to the signal converter 5. I do. The signal converter 5 converts the detection signal into FV data (FV (Focusing Value): camera 1).
The focal length of the lens) and the control computer 9
Output to As an encoder device for detecting the angle of view of the zoom lens of the camera 1, for example, a lens position detector manufactured by Cosmate can be used.

Further, in the vicinity of the camera 1, there is provided a GPS device 6, which is a position detecting device using an artificial satellite, which is generally used in a car navigation system,
The S device 6 obtains the three-dimensional absolute position of the camera 1 according to the received signal (GPS data) from the antenna 7 and
The Z / DR / DT data (XYZ: position data of the camera 1, DR: moving direction data of the camera 1, DT: date and time data) are output to the control computer 9. At this time, the GPS error correction device 8 has received the correction data of the GPS data, and the GPS device 6 corrects the received signal from the antenna 7 based on the correction data.
As such a GPS device for detecting the three-dimensional position of the camera 1, for example, IPS-3000 manufactured by Sony Corporation, Sierra ^™ GPS chipset manufactured by Trimble Corporation can be used. G
The PS error correction device 8 can receive a periodic transmission service by a broadcasting service such as a mobile phone or FM or a wireless device, and can suppress an error of GPS data, which tends to be large due to weather conditions or the like, to 1 m or less. .

The gyro sensor 2, the angle of view encoder 3, the signal converter 4, the signal converter 5, and the GPS
The device 6, the antenna 7, and the GPS error correction device 8 constitute a camera position / posture measuring device. Then, each data from the camera position / posture measuring device is transferred to the control computer 9 and integrated there.

The control computer 9 calculates the parameters of the viewpoint in a virtual three-dimensional map space in the computer from the parameters (XYZ data, HPB data, FV data) obtained by the camera position / posture measuring device described above. . Like the position parameters of the camera 1, the calculated viewpoint position parameters include the viewpoint coordinates (X, Y, Z), the visual axis direction (head angle H, pitch angle P, bank angle B), and view angle (FV). Defined in seven dimensions. Control computer 9
Performs perspective transformation of the three-dimensional map information stored therein in accordance with the parameter result of the calculated viewpoint position, and displays the converted map information (direction, municipal name and boundary) on the monitor TV10.
Send to As software for performing a perspective transformation of the three-dimensional map data, for example, software “VPX” manufactured by Plus One can be used.

The monitor TV 10 has a built-in video editing device, and controls the computer 9 to control the video obtained by the camera 1.
Is displayed in a form in which the map information from is superimposed and presented to the occupant of the flying object. The time required for displaying the map information increases in proportion to the data amount of the processed map information. Therefore, monitor TV 10
The map information to be sent to is selected as necessary. For this reason, an editing function having excellent interactivity using a GUI (Graphical User Interface) or the like is required.

Further, the control computer 9 sends the parameter data (XYZ, HPB, FV) of the viewpoint position and the above-mentioned DR / DT data to the data encoder 11 in the form of serial data. The data encoder 11 encodes serial data from the control computer 9 and video data from the camera 1, and outputs the encoded data to the transmission device 12. The transmitting device 12 converts the encoded data into a radio signal and transmits it to a terrestrial broadcasting station.

The control computer 9 has calibration software and error correction software for each member mounted on the flying object, and performs initial setting and error correction of the gyro sensor 2, the angle of view detection encoder 3, and the like. Is going.

The receiving device 21 receives the radio signal transmitted from the transmitting device 12 and outputs it to the data decoder 22 and the recording device 25. The data decoder 22 converts the encoded data into video data and parameter data (XYZ, HP
B, FV) and DR / DT data, and sends the video data to the video editing device 26, and sends its parameter data (XYZ, HPB, FV) and DR / DT data to the host computer 23. This data encoder 11
/ Data decoder 22 is, for example, manufactured by Prosper Electronics.
VX-50T / R can be used.

The host computer 23 calculates a viewpoint parameter in a virtual three-dimensional map space in the computer from the input parameter data (XYZ, HPB, FV). The parameters of the viewpoint position to be calculated are the same as in the case of the control computer 9 described above. The host computer 23 performs perspective transformation of the three-dimensional map information stored in the external storage medium 24 such as a CD-ROM, HD, or MO disk according to the calculated viewpoint position parameter result, and obtains the obtained map information ( The direction, municipalities and boundaries, buildings, railways, roads, rivers, lakes, marshes, and mountain peaks) are sent to the video editing device 26 as CG video data. As software for performing a perspective transformation of the three-dimensional map data, similarly to the control computer 9, software "VPX" manufactured by Plus One can be used.

The video editing device 26 combines the video data actually shot from the data decoder 22 with the CG video data indicating the map information from the host computer 23, and outputs the synthesized video data to the video transmission device or VTR. . The recording device 25 records the signal received by the receiving device 21 on a video tape or the like.

Next, the operation will be described. First, initial settings of the gyro sensor 2, the angle-of-view detection encoder 3, and the like are executed by the calibration software of the control computer 9. The ground is photographed from the air by the camera 1, and the video signal is sent to the monitor TV 10 and the data encoder 11. During the shooting period, the attitude of the camera 1 is detected by the gyro sensor 2, the HPB data is output from the signal converter 4 to the control computer 9, and the angle of view of the zoom lens of the camera 1 is determined by the angle of view encoder 3. And the FV data is output from the signal converter 5 to the control computer 9, and further, XYZ data indicating the three-dimensional position of the camera 1, DR data indicating the moving direction of the camera 1, and DT indicating the date and time of shooting. The data is output from the GPS device 6 to the control computer 9.

In the control computer 9, based on the parameter data (XYZ data, HPB data, FV data) obtained from the camera position / posture measuring device, the control data in the virtual three-dimensional map space in the computer is obtained.
A viewpoint parameter defined by the dimension is calculated. 3 stored internally according to the calculated viewpoint parameters
The dimensional map information is perspectively transformed from the viewpoint, and the obtained map information (direction, municipal name, and boundary) is sent to the monitor TV 10. Then, an image obtained by superimposing the map information on the video taken by the camera 1 is displayed on the monitor TV10.

FIG. 3 is a schematic diagram showing an example of an image displayed on the monitor TV10. A direction sign indicating east, west, north and south, names of municipalities (A city and B town), and boundaries of municipalities (boundaries between A city and B town indicated by a dashed line) are superimposed and displayed on the captured video. Then, the pilot or the reporter of the broadcasting station relays the situation at the site while watching the superimposed image on the monitor TV10. Since the monitor TV 10 presents an image as if a transparent map is superimposed on the actual scenery to the pilot or reporter, they do not report wrong geographical information when relaying, even if they are not familiar with the geography of the site.

The parameter data (XY) of the viewpoint position
Z, HPB, FV) and DR / DT data are sent from the control computer 9 to the data encoder 11 and converted into encoded data. The encoded data is transmitted to the transmitting device
The signal is output to 12 and converted into a communicable radio signal, and the radio signal is transmitted to a terrestrial broadcast station.

In the terrestrial broadcasting station, the radio signal from the flying object is received by the receiver 21 and converted into the original coded data. The coded data is decoded by the data decoder 22 and converted into original video data captured and parameter data (XYZ, HPB, FV) and DR / DT data of the viewpoint position.

In the host computer 23, parameter data (XYZ, HPB, FV) is input, and based on this, the parameters of the viewpoint in the virtual three-dimensional map space in the computer are calculated. According to the calculated parameters, the three-dimensional map information stored in the external storage medium 24 is perspective-transformed from its viewpoint, and the obtained map information is sent to the video editing device 26.

The original video data is sent from the data decoder 22 to the video editing device 26, the video data is combined with the map information from the host computer 23, and the synthesized video data is output to the video sending device or VTR. You. V
This composite video data is recorded in the TR.

A video signal corresponding to the composite video data is output from the video transmission device as a broadcast signal of a home television. Then, an image corresponding to the composite video data is displayed on the television of each home. FIG. 4 is a schematic diagram showing an example of an image displayed on a television at home. In addition to the azimuth sign, the name of the municipalities, and the boundaries of the municipalities, information of the building (C bridge) and the names of the terrain (Mountain D, Mount E, F river, G bay) are also superimposed on the captured video. Is displayed.

A signal including video data, viewpoint parameter data and DR / DT data received by the receiving device 21 is also sent to the recording device 25 and recorded. The recorded data is used later when performing multi-faceted analysis processing. For example, a process of creating a map indicating the situation at the time of shooting from a video image of a real shot can be considered. In addition, since the data necessary for the analysis process is temporarily recorded on a video tape, etc.,
Of course, it is not necessary to perform such analysis processing in real time at the time of photographing.

In the above-described image presentation system, the three-dimensional position, orientation, and angle of view of the camera 1 having an unstable behavior for performing aerial photography are measured in real time, and a three-dimensional map that is perspective-transformed based on the result is measured. Since the image of the data is displayed in real time in accordance with the photographed video, the photographed situation can be transmitted without error. Also, since numerical information of the three-dimensional position / posture and angle of view of the camera 1 at the time of photographing is recorded in real time together with the photographed image, such information can be used effectively when analyzing the photographed image later.

This video presentation system not only provides viewers with more accurate information in real time for reporting various types of disasters such as earthquakes, forest fires, floods, and typhoons that have a regional spread. As a means of collecting accurate information at an early stage for a comprehensive disaster information system that is expected to be established in the future, or as a means of collecting information for assessing the status of damage and taking remedial measures, it is also very useful for non-report uses. Can contribute.

Hereinafter, the processing operation of software in the control computer 9 and the host computer 23 having high importance in the present invention will be described. 5 and 6 are flowcharts showing the flow of the processing operation. FIG. 5 shows the initial setting operation, and FIG. 6 shows the actual processing operation (camera 1).
From data input to map data drawing processing).

First, the initial operation will be described with reference to FIG. When the program starts, various variables are initialized (S1), and map data (city data, building data, etc.) of the area where the photographing is performed is read (S2).
Next, after initializing the data of the camera 1 (S3), the coordinates of the reference point are input (S4). Here, as the reference point, a point whose coordinates are known, such as a departure point or a specific point on the site, is adopted. Next, a timer for specifying a sampling interval of data input from the camera 1 is set (S5). In this state, the camera 1 enters a data input event waiting state, and the initial operation ends.

Next, the actual processing operation will be described with reference to FIG. When the data input event of the camera 1 occurs, it is detected (S11), and the expected coordinates are calculated from the stored input data one second before (S12). The predicted coordinates are the position coordinates of the camera 1 that are predicted when the time required for the subsequent map data drawing processing has elapsed. The obtained predicted coordinates are compared with the previous predicted coordinates, and it is determined whether or not the difference is within an error range (S13). If it is out of the error range, it is determined that there is a measurement abnormality of the camera position / orientation measuring device with respect to the camera 1, and the predicted coordinates are not used for the subsequent map data drawing processing, and the process returns.

On the other hand, if it is within the error range, the predicted coordinates are converted to CG coordinates (S14). Then, the current input data of the camera 1 is stored as the input data one second before the next time (S15). Next, an event of drawing CG video data is generated (S16), and map data closest to the viewpoint is searched to create a sort list (S17). After clearing the contents of the screen display and the Z buffer storing the information on the perspective relationship (S18), a drawing process is performed based on the map data (S19). The drawing process, for example, draws terrain with a wire frame from a position close to the viewpoint, draws the district name with characters, draws a building with a wire frame, draws a building with a polygon,
Perform terrain drawing in polygon order. However, when the data input event of the camera 1 occurs, the drawing process is forcibly terminated even if the drawing process is in progress (S20), and the process returns.

In the above-described processing operation, the map data is drawn based on the predicted coordinates of the camera 1, so that the time lag associated with the drawing processing can be compensated, and the image data exactly matches the image captured by the camera 1. CG map data can always be displayed. In addition, when the event of the data input of the camera 1 occurs, the drawing process is forcibly terminated immediately, so that the map information can be displayed at the speed of the animation.

(Second Embodiment) In the above-described first embodiment, a real image on which map information is superimposed can be viewed on the monitor TV 10 in the flying body. Then, there is a problem that the equipment mounted on the flying object increases. In addition, when a person familiar with the geography of the site is boarding the flying object, the situation of the site can be visually checked, and in many cases, it is not necessary to display a photographed image on which map information is superimposed. Therefore, an example in which the display of a real image on which map information is superimposed is a base station on the ground is the image presentation system according to the second embodiment of the present invention. FIG. 7 is a schematic diagram showing a configuration example of this video presentation system (second embodiment).

In FIG. 7, the same parts as those in FIG. 2 are denoted by the same reference numerals, and their description will be omitted. In the configuration of the second embodiment, the monitor is not provided in the flying body,
The actual video signal from the camera 1 is sent only to the data encoder 11. The control computer 9 includes a gyro sensor 2, an angle-of-view detection encoder 3, a signal converter 4, a signal converter 5, a GPS device 6, an antenna 7, and a GPS error correction device 8, It only integrates each data from the measuring device, and does not perform the operation of obtaining map information as in the first embodiment.

In FIG. 7, the host computer 23
Includes a video synthesizing device, CG video data indicating map information obtained based on parameter data of shooting conditions decoded by the data decoder 22, and a data decoder.
The actual video data decoded in step 22 is the host computer
23, the combined video data is output to the monitor TV 27, and the monitor TV 27 performs image display in a mode in which map information is superimposed on a video of a real photograph.

In the above-described first and second embodiments, the case where the camera 1 is mounted on the flying object has been described. However, the camera 1 is rotatably attached to a predetermined place of a building such as a high-rise building or a tower. A fixed point camera as described above may be used. In such a case, since the three-dimensional position of the camera is fixed, GPS is not necessary. In the case of a rotary encoder and a flying object instead of a gyro sensor as detection means for detecting the three-dimensional attitude and angle of view of the camera Requires the same encoder for image detection.

(Third Embodiment) FIG. 8 is a schematic diagram showing a configuration example of a video presenting system (third embodiment) of the present invention. In the third embodiment, humans, animals, vehicles, ships,
In this example, a moving object such as an airplane is set as a target, and the tracking of the target is performed.

In FIG. 8, reference numerals 18 and 19 denote a GPS device and a transmitter, and a locator 20 integrating the GPS device 18 and the transmitter 19 is a tracking target for a human, an animal, a vehicle, a ship, an airplane, or the like. The GPS device 18 determines the absolute position of the object, and transmits the position data (XYZ data) to the base station by the transmitter 19. The same components as those in the first and second embodiments described above are denoted by the same reference numerals, and description thereof will be omitted.

Next, the operation will be described. 1st, 2nd
As described in the first embodiment, the video taken by the camera 1 mounted on the flying object is displayed on the monitor TV 27 in a state where the map information is superimposed.
In such a state, first, when searching for an object,
The camera 1 is rotated 360 degrees with a wide-angle lens as much as possible in the air to take a picture. Since the position data is sent from the locator 20 attached to the target to the base station, if the target exists in the shooting area of the camera 1, the position and attribute information of the locator 20 attached to the target are stored in the base station. Station monitor TV
Appears on 27. Next, if the flying object is moved in the direction of the locator 20 displayed on the monitor TV 27 and zoomed up, the target object can be displayed in the photographed video on which the map information displayed on the monitor TV 27 is superimposed. Can be found.

After searching for and finding the object in this way, if the user moves the flying object and operates the camera so that the position of the locator 20 is not lost, a photographed image on which map information is superimposed can be obtained. It is possible to track a moving object while visually recognizing the object in the space.

Further, the actual photographed video data, the parameter data (XYZ, HPB, FV) and DR / DT data of the photographing conditions in the search and tracking processing for such an object are recorded on a video tape or the like by the recording device 25. . By doing so, it can be used as an information source for the analysis processing of the movement of the object.

FIG. 9 is a schematic diagram showing a configuration example of a modification of the video presentation system according to the third embodiment. In this modification, similarly to the first embodiment, the monitor T
The V10 is mounted, and the monitor TV 10 in the flying body can display the position and attribute information of the locator 20 and the target object similarly to the monitor TV 27 on the ground.

In the above description, one moving object is tracked. However, according to the third embodiment, a plurality of objects can be tracked simultaneously.

In each of the above-described embodiments, the map information is superimposed on the photographed video in the air and displayed.
If the camera 1, the gyro sensor 2, the angle-of-view detection encoder 3 and the GPS device 6 are waterproof, the map information is superimposed on the captured image obtained underwater, similarly to the above-described embodiments. Needless to say, it can be displayed.

Further, in each of the above-described embodiments, the map information is superimposed and displayed on the photographed image. However, the display screen is divided, and the photographed image is displayed on one screen area, and the other screen area is displayed. Map information may be displayed.

By the way, in the information presentation system of the present invention, the map information to be superimposed on the photographed image includes
Transportation network, public facilities, major buildings, landmarks,
In addition to basic data such as river names, etc., city facilities such as gas, water, electricity, highways, etc.
Sightseeing courses, marathon courses, construction plans, underground buried objects,
Optional data such as historical sites are prepared in accordance with the intended use of the user. By preparing various types of map information in this way, the range of use of the information presentation system of the present invention can be greatly expanded.

Hereinafter, the fields of application of the present invention will be described. The users of the system of the present invention are individuals, businesses, and public institutions. A first field of application of the system of the present invention is to use the configuration of the above-described first or second embodiment to increase the information value of a video taken in the air or underwater. Since various types of map information can be superimposed in real time on the photographed image of the actual photograph, the information value of the current airborne or underwater photographed image or fixed-point camera image can be dramatically increased. In other words, the screen on which the map information related to the video is superimposed together with the video can transmit valuable information to the user, and the TV (Augm.
ented TV).

As a specific example of such a first application field, in addition to information gathering, monitoring and analysis of disasters (particularly disasters such as earthquakes, floods, debris flows, and wildfires having a regional spread), TV broadcasting Display (maintenance broadcast program at the time of disaster, sports broadcast program such as marathon, yacht race, hobby program such as sightseeing spots etc.), display and maintenance of buried gas and water facilities, large-scale Use in construction planning (real-time presentation of planned facilities, etc.)
Wide use is possible.

A second field of use of the system of the present invention is to perform visual tracking using the configuration of the third embodiment. If the mobile object to be tracked has a locator, it is possible to visually search, track, and confirm the position of the mobile object on the TV video on which the map information is superimposed. Specific examples of the second field of use include operation management of vehicles and ships, rescue and rescue of humans in mountains and seas,
There are sport event management such as yacht racing and car rallies, as well as academic research on bird and beast behavior, etc., and even in this second field of application, wide use is possible.

[0073]

As described above, in the image display system of the present invention, an image in which map information is superimposed on an image of an actual photographed image by a camera can be presented, so that the information value of an image captured by the camera can be enhanced. Further, accurate tracking of a moving object can be performed. Further, since the photographed video data and the photographing conditions of the camera are recorded, these information can be used effectively when the photographed video is analyzed later.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing an outline of a video presentation system of the present invention.

FIG. 2 is a schematic diagram showing a configuration of an example of a video presentation system of the present invention.

FIG. 3 is a schematic diagram showing a display example of an image on a monitor TV mounted on a flying object.

FIG. 4 is a schematic diagram showing a display example of an image on a home television.

FIG. 5 is a flowchart showing a software processing operation (initial setting operation) in the video presentation system of the present invention.

FIG. 6 is a flowchart showing a software processing operation (map data drawing processing operation) in the video presentation system of the present invention.

FIG. 7 is a schematic diagram showing a configuration of another example of the video presentation system of the present invention.

FIG. 8 is a schematic diagram showing a configuration of still another example of the video presentation system of the present invention.

FIG. 9 is a schematic diagram showing a configuration of still another example of the video presentation system of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Camera 2 Gyro sensor 3 Encoder for angle of view detection 6 GPS device 9 Control computer 10 Monitor TV 12 Transmitting device 18 GPS device 19 Transmitter 20 Locator 21 Receiving device 23 Host computer 24 Storage medium 25 Recording device 26 Video editor 27 Monitor TV

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Ken Sawai 151-1515 Yamadakita, Suita-shi, Osaka

Claims (8)

[Claims] 1. A system for presenting an image captured by a camera, a detecting unit for detecting a three-dimensional position, a three-dimensional posture, and an angle of view of the camera to be captured, and a storage unit for storing three-dimensional map data. Means for perspective-transforming the three-dimensional map data stored in the storage means based on the detection result of the detection means to obtain map data corresponding to the photographed video; and obtaining the obtained map data by the camera. Means for displaying the captured image together with the captured image. 2. A camera provided on the flying object, detection means provided on the flying object for detecting a three-dimensional position, a three-dimensional attitude and an angle of view of the camera, and a three-dimensional camera provided on the flying object Storage means for storing map data, and perspective conversion of three-dimensional map data provided in the flying object and stored in the storage means based on the detection result of the detection means, and photographing with the camera An image presentation system, comprising: means for acquiring map data according to an image; and means provided on the flying object and displaying the acquired map data together with an image captured by the camera. 3. A camera provided on the flying object, detection means provided on the flying object and detecting a three-dimensional position, a three-dimensional attitude and an angle of view of the camera, and a camera provided on the flying object and provided with the camera Transmitting means for transmitting, to the ground, data of the photographed video and data of the detection result of the detecting means, receiving means for receiving transmission data from the transmitting means, and storing means for storing three-dimensional map data; Means for perspective-transforming the three-dimensional map data stored in the storage means based on the detection result of the detection means to obtain map data corresponding to a video taken by the camera; Means for displaying the image together with the image captured by the camera. 4. A camera rotatably provided at a fixed position, detection means for detecting a three-dimensional posture and an angle of view of the camera, storage means for storing three-dimensional map data, and Means for performing perspective transformation of the stored three-dimensional map data based on the detection result of the detection means to obtain map data corresponding to a video taken by the camera, and capturing the obtained map data by the camera Means for displaying the image together with the image. 5. A system for presenting an image captured by a camera including a moving object, a position identification means attached to the object to identify a three-dimensional position of the object, and the obtained position identification. Transmitting means for transmitting information;
Detecting means for detecting the three-dimensional position, three-dimensional attitude, and angle of view of a camera that performs shooting, storage means for storing three-dimensional map data, and three-dimensional map data stored in the storage means Means for obtaining map data according to the captured image by performing a perspective transformation based on the detection result of the detection means, and means for displaying the obtained map data together with the captured image including the object by the camera. A video presentation system characterized by the following. 6. The image presentation system according to claim 1, wherein the camera is a camera that performs aerial photography. 7. The video presentation system according to claim 1, wherein the camera is a camera that performs underwater photography. 8. The video presentation system according to claim 1, further comprising a unit that records data of a detection result of the detection unit and data of a video image captured by the camera.