JP2000152216A - Video output system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically provide image and audio data at a viewpoint required by the user or a user device. SOLUTION: A viewpoint conversion processing means 25 converts images from image input means 10 into other images based on a viewpoint position and a direction of a sight line received by a viewpoint input means 20 and an image composite means 26 synthesizes the other images and provides an output. Audio signals from audio input means 30 are converted by an audio processing means 32 based on the viewpoint information. In order to automate the system, an image input automatic selection means 23 and an audio input automatic selection means 31 that select an image input means 10 and an audio input means 30, an automatic viewpoint moving means 21 that changes automatically the viewpoint, and an adjustment mechanism automatic control means 24 that automatically controls a turning table and zooming are used. Data required for the processing are obtained from a database 22b.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video output system having a video processing function used for facility monitoring, facility guidance, simulated experience, image recognition, and the like.

[0002]

2. Description of the Related Art As a typical example of a video output system,
2. Description of the Related Art A CCTV (closed circuit television) system including a monitoring camera, a transmission path, a monitor, and the like is used for monitoring the status of facilities and surroundings of facilities, guiding facilities, and the like. In a CCTV system equipped with multiple surveillance cameras, in order to output images,
In addition to the method of providing a plurality of monitors, since the number of surveillance cameras is generally larger than the number of monitors, a method of selecting and displaying surveillance cameras, and dividing the display area of the monitor to display a plurality of captured images A method and a method of switching and displaying a plurality of captured images at appropriate time intervals have been used or used together.

FIG. 5 shows a CCT having a plurality of surveillance cameras.
It is a block diagram showing a schematic structure of an example of a V system.
In FIG. 5, reference numeral 10 denotes a plurality of image input units, which are monitoring cameras in this example. 11 is an adjustment mechanism, a zoom mechanism for changing the enlargement / reduction ratio of the image and a turntable for changing the shooting direction, 12 is a video switcher for switching image signals input from a plurality of image input means and outputting to the monitor, 13 is a monitor, 14 is an image input designator for designating image input means to be connected to the monitor, and 15 is a controller for controlling a zoom mechanism as an adjustment mechanism and a turntable. The operation procedure of this CCTV system will be described. The user determines an image input unit for taking a required image, and operates the image input designator 14 to notify the video switcher 12 of the corresponding image input unit. The video switcher 12 connects the notified image input means to the monitor 13 and
3 displays an image. Further, in order to obtain a desired photographing direction and viewing angle, the user operates the controller 15 to control the zoom mechanism and the turntable. There is also known an image processing system in which an image obtained by a CCTV system is input to an image processing apparatus, and an abnormality is determined by image recognition and various types of image analysis are performed. In FIG. 5, to perform such image processing, an image processing apparatus is connected to the monitor 13 in parallel or instead.
As in this example, to detect abnormalities quickly,
In addition to using the image at the same time as shooting,
There is also a form in which the information is recorded and then reproduced and used.

In such a conventional general video output system, in each stage of photographing, transmission, recording and reproduction, video signal output, display, etc., an object is accurately reproduced as a video, that is, the object is photographed precisely. The purpose is to transmit or record / reproduce without deterioration and to output and display or use a video signal that is faithful to the captured video.

[0005]

However, the prior art has the following problems. First, the viewpoint position, the line-of-sight direction, and the viewing angle of the output image are determined by the shooting position and the shooting direction of the image input unit, and the horizontal and vertical viewing angle performance (such as the focal length of a lens). Therefore, an image at a viewpoint position, a line-of-sight direction, and a viewing angle required by the user is not necessarily obtained, and it is difficult to grasp and recognize the target, and it is difficult to understand the situation.

A video output system having a plurality of image input means has the following additional problem. First, each time the user uses a different image input means, the user must re-recognize the shooting range, the shooting direction, and the like. For this reason, in order to monitor spatially continuous objects, it is necessary to be aware of the arrangement of the image input means and the overlapping shooting range / non-shooting range,
It is difficult to grasp the target comprehensively. Further, in order to perform image recognition processing or the like, it is necessary to adjust a difference in viewpoint caused by different image input means. Second, the user is conscious of the target by the position or part, but in order to obtain the image, the user must determine which image input means to use and perform a switching operation, etc. It is difficult to obtain images. Thirdly, it is necessary to perform an operation of switching a plurality of image input means and a control operation of a turntable / zoom mechanism in response to a request to see a different place or to view from a different angle, which has regularity. In the monitoring work, simple work is forced.

In order to solve the problems associated with a system having a plurality of video input means, Japanese Patent No. 2758432 "Image Processing Apparatus" discloses a method using a time-sharing method. "Means means such as a storage means. In addition, JP-A-9-1307
No. 83, "Distributed Video Monitoring Apparatus", describes improvement of convenience and the like by means of a combination of a wide-angle camera and a zoom camera. Further, in order to reduce the size of a monitoring system that performs image processing with a plurality of television cameras, Japanese Patent Application Laid-Open No. 9-83996
In the issue “Monitoring System”, means by superimposition and the like is described. However, both are different from the focus of the present invention.

[0008] The present invention aims at a conventional general video output system, which is to "output a target video accurately".
By switching to "outputting an image for easily grasping and recognizing an object with as few operations as possible", the above-mentioned problem of the conventional image output system is to be solved. In order to easily grasp and recognize the target, it is necessary to output a video that matches the spatial recognition of the user or the user's device.Therefore, it is necessary to output a video from the viewpoint required by the user or the user's device. did. In addition, in order to minimize the number of operations, the regularity of the operations performed by the user is extracted and automated. Further, by outputting a fine image and sound associated with the image, it is possible to enhance the sense of realism, and the system can be used more effectively.
Based on these, it is an object of the present invention to provide the following video output system corresponding to each claim. Here, the utilization device is an image processing device that performs image recognition and the like.

(1) To provide a video output system that outputs an image at a viewpoint position, a line-of-sight direction, and a viewing angle required by a user or a device to be used.

(2) One or a plurality of image input means are automatically selected in order to obtain an image at a viewpoint position, a line-of-sight direction and a viewing angle required by a user or a device to be used, and an image obtained by changing the viewpoint is automatically selected. To provide a video output system for composite output.

(3) To provide a video output system that automatically changes a viewpoint position, a line-of-sight direction, or a viewing angle to reduce the operation of a user or a device to be used.

(4) To provide a video output system which automatically controls a zoom and a turntable provided in an image input means and automatically outputs a fine image with less discomfort.

(5) To provide a video output system capable of outputting sound associated with an image at a viewpoint position and a line-of-sight direction required by a user or a device to obtain a high sense of realism.

[0014]

According to a first aspect of the present invention, there is provided a viewpoint input means for inputting a viewpoint position, a line-of-sight direction, and a viewing angle, an image being input, and an image at the viewpoint position, the line-of-sight direction, and the viewing angle. Video output system characterized by comprising a viewpoint conversion processing means for converting and outputting to
Provided is a video output system that outputs an image at a viewpoint position, a viewing direction, and a viewing angle required by a user or a usage device.

According to a second aspect of the present invention, there is provided an image output system having a plurality of image input means, wherein an image input data storage means for storing data relating to a photographing position, a photographing direction and a visual angle of the plurality of image input means. A viewpoint input unit for inputting a viewpoint position, a line-of-sight direction, and a viewing angle; and one or more images for outputting an image at the viewpoint position, the line-of-sight direction, and the viewing angle using the image input data storage unit. An image input automatic selection means for selecting an input means,
Viewpoint conversion processing means for inputting an image from the selected one or more image input means, converting the image into an image at the viewpoint position, the line-of-sight direction, and the viewing angle, and outputting the converted image; A video output system comprising: an image synthesizing unit for synthesizing and outputting, wherein one or a plurality of image input units are provided to obtain an image at a viewpoint position, a line-of-sight direction, and a viewing angle required by a user or a device to be used. Is automatically selected, and a video output system for automatically synthesizing and outputting an image whose viewpoint has been converted is provided.

According to a third aspect of the present invention, there is provided the video output system according to the first or second aspect, further comprising an automatic viewpoint moving means for automatically changing a viewpoint position, a line-of-sight direction, or a line of sight. Provided is a video output system that automatically changes a direction or a viewing angle to reduce the operation of a user or a user device.

According to a fourth aspect of the present invention, in the video output system in which the image input means includes an adjustment mechanism for changing an enlargement / reduction ratio of an image or a photographing direction, the movement of pixels before and after the viewpoint is changed with respect to the adjustment mechanism. The image processing apparatus further includes an adjustment mechanism automatic control unit that performs control for inputting an image necessary for image output at an input viewpoint position, a line-of-sight direction, and a viewing angle by reducing the amount and overlapping pixels in image synthesis. Item 4 is a video output system according to any one of Items 1 to 3, wherein the video output system automatically outputs a fine image with less discomfort by automatically controlling an adjustment mechanism such as a zoom and a turntable included in the image input means. .

According to a fifth aspect of the present invention, there are provided a plurality of sound input means, a sound input data storage means for storing data relating to a position of the sound input means, and a sound input data input means utilizing the sound input data storage means. Sound input automatic selection means for selecting one or more sound input means for outputting sound at the selected viewpoint position, and sound input from the selected one or more sound input means, and the input viewpoint position 3. The video output system according to claim 2, further comprising: an audio processing unit configured to convert the sound into a sound in accordance with the input gaze direction and output the sound in accordance with the input line-of-sight direction. Provided is a video output system that outputs sound associated with an image at a viewpoint position and a line-of-sight direction to provide high realism.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The elements constituting the present invention, except for those overlapping and other included, are enumerated below, and their embodiments will be described first. (1) image input means, (2) image input data storage means for storing data relating to shooting positions and shooting directions and viewing angles of a plurality of image input means, and (3) viewpoint input means for inputting viewpoint positions, viewing directions, and viewing angles. (4) viewpoint conversion processing means for inputting an image output from one or a plurality of image input means, converting the image into an image at a specified viewpoint position, a viewing direction, and a viewing angle, and outputting the converted image; Image synthesizing means for synthesizing and outputting the obtained plurality of images, and (6) one or a plurality of image inputs for outputting an image at a specified viewpoint position, a line-of-sight direction and a visual angle using the image input data storage means. Image input automatic selection means for selecting means, (7) automatic viewpoint moving means for automatically changing the viewpoint position or line-of-sight direction or viewing angle, (8) adjustment mechanism for changing the image enlargement / reduction rate or shooting direction,
(9) Control the adjustment mechanism to reduce the amount of pixel movement before and after viewpoint conversion and the overlapping pixels in image synthesis, and to input an image necessary for image output at the input viewpoint position, viewing direction, and viewing angle. (10) sound input means, (11) sound input data storage means for storing data relating to the position of the sound input means, and (12) sound input data storage means, A sound input automatic selection means for selecting one or more sound input means for outputting sound at a position, (13) sound from one or more selected sound input means is input, and a specified viewpoint position Sound processing means for converting the sound into the sound of the sound and outputting the sound corresponding to the line of sight.

(1) The image input means is a surveillance camera, video camera, digital camera or the like.

(2) The image input data storage means for storing data relating to the photographing positions, photographing directions, and viewing angles of the plurality of image input means is, for example, by storing the image input data in a computer equipped with a database management system. Can be implemented. The database management system simplifies the procedure for referring to data, and, if the data definition is extended, can store data relating to the adjustment mechanism of the image input means and data relating to the sound input means. The method of automatically selecting the image input means, the viewpoint conversion processing, and the image synthesis using the image input data storage means performs more stable processing irrespective of image quality, weather conditions, etc., as compared with the method based on image recognition or the like. It can be performed simply and at high speed.

(3) The viewpoint input means for inputting the viewpoint position, the line-of-sight direction and the viewing angle can be implemented as follows. The method of specifying the viewpoint position, that is, the three-dimensional coordinates of the viewpoint, includes three-dimensional directions (up, down, left, right,
Method of specifying the amount of movement (back and forth) (meters, kilometers, etc.) and the latitude and longitude and altitude as absolute values based on the external coordinate system, or the distance and altitude from the origin and center line of roads and rivers There is a method to specify. As a method of designating the line of sight, there are a method of designating the line of sight as a relative direction from the current display image, and a method of designating, as an absolute value, an azimuth from true north and a depression / elevation angle from a horizontal plane.
Methods for specifying the viewing angle include a method for specifying that the relative change with respect to the current display image is wide or narrow, a method for specifying the horizontal and vertical viewing angles, and a method for specifying the focal length. Relative values such as a moving direction and a moving amount can be input to a display image or a control panel using a pointing device such as a joystick or a mouse or a cursor key of a keyboard. The absolute value can be entered using a keyboard. Note that it is not necessary to always input all of the viewpoint position, line-of-sight direction, and visual angle. For example, the visual angle is internally stored as a standard value of a normally used image input unit as a default, and is processed using the values. By doing
Input operations can be reduced.

(4) An embodiment of the viewpoint conversion processing means for inputting an image output from one or a plurality of image input means, converting the image into an image at a specified viewpoint position, a viewing direction, and a viewing angle, and outputting the converted image. It is as follows.

First, an embodiment of a viewpoint conversion processing means for inputting one image, converting the image into an image at a specified viewpoint position, a viewing direction, and a viewing angle and outputting the converted image will be described.

A simple form is as follows. It is assumed that the image is represented by a coordinate system in which the X axis is horizontal, the Y axis is vertical, the Z axis is in the line of sight, and the coordinate origin is the center of the image. In that case, the viewpoint conversion processing can be performed by performing the following image processing. 1) Linking the enlargement / reduction of the image with the forward / backward movement of the viewpoint position. 2) Z to move left and right of viewpoint
Link the rotation of the image about the axis. 3) The rotation of the image about the X axis is linked with the vertical movement of the viewpoint position. 4) The rotation of the image around the Y axis is linked to the left and right in the line-of-sight direction. 5) Deformation in which the vertical magnification of the image is changed is linked in the vertical direction of the line of sight. 6) Linking the enlargement / reduction of the image with the change in the viewing angle.

By using the perspective drawing method, more accurate viewpoint conversion processing means can be implemented. Perspective imaging is generally used in the field of computer graphics to display three-dimensional coordinates on a screen. Here, an image captured and digitized by image input means is generated by computer graphics. And performs viewpoint conversion processing equivalent to the image.

In using the perspective drawing method, it is necessary to digitize an image and decompose it into pixels. Digitization of an image can be performed by a video capture board when inputting to the viewpoint conversion processing means. Also, because of the necessity of using a digital transmission line or the necessity of digitally recording an image, the image may be digitized by an image encoding method such as JPEG, Motion-JPEG, or MPEG and input to the viewpoint conversion processing unit. . In that case, the image may be converted to the bitmap format and the viewpoint conversion process may be performed.

Now, according to the perspective drawing method, an image photographed and digitized by the image input means (hereinafter referred to as an "original image").
That. ) Converts the world coordinates of each point of the monitored object into viewpoint coordinates on the screen between the monitored object and the viewpoint E of the image input means, and converts the viewpoint coordinates into two-dimensional screen coordinates. It is thought that it was done. Accordingly, the world coordinates are derived in reverse from the screen coordinates of each point of the original image, the world coordinates are multiplied by the input viewpoint and the viewing matrix of the viewing direction, and the perspective conversion is performed by the input viewing angle. Conversion process can be performed.

The above process is mathematically outlined with reference to FIG.
You. The viewpoint E of the image input means is a right-handed world polar coordinate system
, Given by coordinates (ρ θ φ)
Look at the coordinate origin O located near the center of the monitored object from E
Shall be. The viewpoint coordinate system uses the direction from E to O
Left-handed coordinate system with positive direction, X axis of viewpoint coordinate system is rightward
The Y axis is upward. The screen is orthogonal to the Z axis
2C in X axis direction_X, 2C in Y axis direction_YWhat is the size of
And World coordinates (X_WY_W 
Z_W) Is given, the viewpoint by the viewing transformation
Coordinates in the coordinate system (X _E Y_E Z_E) Is calculated by Equation 1.
You.

[0030]

[Number 1] _{(X E Y E Z E 1} ) = (X W Y W Z W 1) V

Where the viewport matrix V is
X4 matrix.

[0032]

(2) (−sinθ−cosφcosθ−sinφcosθ0) (cosθ−cosφsinθ−sinφsinθ0) (0 sinφ−cosφ0) (0 0 ρ1)

The resulting viewpoint coordinates (X _E Y _E Z _E) is
Are perspective transformed into screen coordinates (X _S Y _S) in Equation 3. In Equation 3, the size of the figure can be derived from the number of pixels that can be displayed on the monitor, and the size of the object can be derived from the viewing angle.

[0034]

Equation 3] _{X S = d × X E /} Z E + C X Y S = d × Y E / Z E + C Y d = size of the size / objects [rho × Figure

Assuming that the original image has been generated as described above, the world coordinates are derived from the screen coordinates of the original image displayed by equation (5).

[0036]

Equation 4] _{(X W Y W Z W 1} ) = ((X S -C X) Z E / d
_{(Y S -C Y) Z E} / d Z E 1) × V -1 V -1 is the inverse matrix of the viewing matrix V

Here, one dimension in the three-dimensional world coordinate system is extrapolated as known. For example, assuming that a road, a river, or a railway is on a plane where _Zw is zero, the world coordinates are obtained by Expression 4 as in Expression 5.

[0038]

X _W = V ₄₃ (AX _H −Y _H ) / ｛A (V ₁₁ −V ₁₃ X _H )
_{+ V 13 Y H -V 12}} Y W = V 43 (BX H -Y H) / {B (V 21 -V 23 X H) + V 23 Y
_{H -V 22} Z W = 0} A = (V 23 Y H -V 22) / (V 23 X H -V 21) B = (V 13 Y H -V 12) / (V 13 X H -V 11 ) V _ij is an element at row i and column j of the viewing matrix V X _H = (X _S −C _X ) / d Y _H = (Y _S −C _Y ) / d

If the viewing transformation and the perspective transformation based on the input viewpoint position and the like are performed again on the world coordinates obtained as described above, the viewpoint transformation processing can be performed.

In order to perform a plurality of viewpoint conversion processing means, there is a method in which a plurality of viewpoint conversion processing means for one image described above are juxtaposed, and a plurality of time-sheared images are input to the high-speed viewpoint conversion processing means. .

(5) An embodiment of an image synthesizing means for synthesizing and outputting a plurality of images subjected to viewpoint conversion processing will be described. A plurality of images that have been subjected to viewpoint transformation processing have undergone perspective transformation from the same viewpoint on the same screen coordinates, and can be combined by superposition. When pixels of different images overlap at the same position on the screen coordinates, for example, referring to the image input data, the image with the small absolute value of the difference between the imaging direction and the specified line-of-sight direction is given priority first. By synthesizing an image having a small distance between the position and the designated viewpoint position so as to be displayed with the second priority, it is possible to output a fine image with less user discomfort. In addition, the viewing transformation and the perspective transformation parameters are output from the viewpoint transformation processing means to the image synthesizing means,
It is also possible to flexibly set the overlapping order.

(6) Implementation of automatic image input selection means for selecting one or more image input means for outputting an image at a specified viewpoint position, line-of-sight direction and viewing angle with reference to image input data The form is as follows. A range of an image output after the viewpoint conversion processing (hereinafter, referred to as a “virtual image range”) is obtained from the specified viewpoint position, viewing direction, and viewing angle. As the virtual image range is obtained, the photographable range of the image input means is obtained from the data on the photographing position, the photographing direction, and the viewing angle of the image input means stored in the image input data storage means. Alternatively, the photographable range may be stored in the image input data storage means in advance. By comparing the photographable range and the virtual image range obtained as described above and selecting an image input unit having a matching part, the image input automatic selection unit can be implemented.

When the depth of the virtual image range becomes very large, the image obtained from the image input means at the back becomes extremely small and an effective image cannot be obtained. Since the amount increases, the depth is cut off at a constant value. In addition, when the line-of-sight direction of the image input unit is significantly different from the specified line-of-sight direction, by adding conditions such as not selecting, the viewpoint conversion process and the image synthesis process can be reduced.

FIG. 4 shows an example of the virtual image range. The photographing position of the image input means is given by coordinates (0 0 H) in the XYZ three-dimensional orthogonal coordinate system, and the photographing is performed at the depression angle α in the X-axis direction. The vertical viewing angle is 2β and the horizontal viewing angle is 2γ
If the imaging target is distributed on a horizontal plane at Z = 0, the virtual image range is a portion surrounded by the four coordinates of Expression 6.

[0045]

(Hcot (α + β) Htanγ / sin (α + β) 0) (Hcot (α + β) -Htanγ / sin (α + β) 0) (Hcot (α-β) Htanγ / sin (α-β) 0) (Hcot (α-β) -Htanγ / sin (α-β) 0)

(7) The automatic viewpoint moving means for automatically changing the viewpoint position, the viewing direction or the viewing angle can be realized by, for example, a calculation function for increasing the viewpoint position by a constant value every second in the viewing direction. By inputting the calculated value to the viewpoint input means, it is possible to automatically simulate observation by foot or by car. If the monitored object has curves or intersections and the monitored object cannot be continuously grasped without changing the line of sight, using a database or geographic information system on the shape or alignment of the monitored object The eye direction and the viewpoint position are linked. If you adopt a calculation function that increases only the line of sight by a constant value every second,
Because you can look around automatically, you can reduce the operation of the turntable. If only the viewing angle is increased or decreased, the zoom operation can be reduced.

(8) The adjusting mechanism for changing the enlargement / reduction ratio of the image is implemented by optical or electronic zoom, and the adjusting mechanism for changing the photographing direction is implemented by a rotating platform (sometimes called a pan head). You.

(9) To reduce the amount of movement of pixels before and after the viewpoint conversion and the number of overlapping pixels in image synthesis for the adjustment mechanism, and to input an image necessary for image output at the input viewpoint position, line-of-sight direction, and angle of view. The embodiment of the adjusting mechanism automatic control means for performing the above control is as follows. First, the case where there is one available image input means will be described. Reducing the amount of pixel movement before and after viewpoint conversion can reduce the user's discomfort. For this purpose, the photographing direction is made closer to the line of sight. Next, since the viewpoint position and the line-of-sight direction are determined, the zoom is controlled so as to capture the virtual image range by using the trigonometric function described in the embodiment of the image input automatic selection means. By the above operation, the moving amount of the pixel before and after the viewpoint conversion can be reduced, and the difference between the images can be reduced. When a plurality of image input units are used, a virtual image range is set to an appropriate region, for example, a viewpoint position and a photographing position, in order to reduce overlapping pixels generated after the viewpoint conversion process and to achieve efficient image acquisition and synthesis. And the above-described control is performed for each of the divided areas. In calculating the control value, in addition to obtaining the current value and the adjustable range through communication with the adjustment mechanism, it is possible to store and use data relating to the control performance of the adjustment mechanism in the image input data storage means.

(10) The sound input means is a sound collecting microphone or the like.

(11) The sound input data storage means for storing data relating to the position of the sound input means can be implemented using a database, similarly to the image input data storage means.

(12) Using the sound input data storage means,
The automatic sound input selection means for selecting one or a plurality of sound input means for outputting sound at the designated viewpoint position can be implemented as follows. Taking the case where four speakers are arranged around the user in front, rear, left and right as an example, the closest sound input means within the range of 90 ° in each of the front, rear, left and right of the designated viewpoint position is stored in the sound data storage means. Inquire and select.

(13) Embodiment of sound processing means for inputting sound from one or a plurality of selected sound input means, converting the sound into sound at a designated viewpoint position, and outputting the sound corresponding to the line of sight. Is as follows. First, the distance between the designated viewpoint position and the selected sound input means is obtained by using the sound input data storage means. The sound volume of the selected sound input means around the viewpoint is inversely distributed to the square of the distance obtained earlier, and a delay time calculated from the sound speed and the distance is added. Is output to the speaker around the user corresponding to. As a result, the sound of the sound input means far from the viewpoint is small and delayed, and the close sound is quickly large and output from the loudspeaker in the direction matching the line of sight.

[0053]

Embodiments of the present invention will be described below with reference to the drawings. In all the drawings for describing the embodiments, components having the same function are denoted by the same reference numerals, and a repeated description thereof will be omitted.

FIG. 6 is a diagram showing a schematic configuration of a video output system according to a first embodiment of the present invention. FIG.
In the above, 20 is a viewpoint input means, and 25 is a viewpoint conversion processing means. The user uses the viewpoint input unit 20 to input the viewpoint position, the line-of-sight direction, and the viewing angle of the image. The viewpoint is sent from the viewpoint input unit 20 to the viewpoint conversion processing unit 25. The viewpoint conversion processing unit 25 performs a viewpoint conversion process on the input image based on the data regarding the viewpoint, and outputs the image after the viewpoint conversion process. Output.

FIG. 7 shows a conventional CCTV shown in FIG.
FIG. 11 is a diagram illustrating a schematic configuration of a second embodiment in which the present invention is applied to a system. In FIG. 6, reference numerals 1 to 3 and 5 denote communication paths, reference numeral 10 denotes a plurality of image input means, and in this example, a monitoring camera. Numeral 11 denotes an adjusting mechanism, which is a zoom mechanism for changing the enlargement / reduction ratio of the image and a turntable for changing the photographing direction. Reference numeral 12 denotes a video switcher for switching image signals input from a plurality of image input means and outputting the same to a monitor, 13 a monitor, 14 an image input designator for designating an image input means connected to the monitor, and 15 an adjustment mechanism. A controller for controlling the zoom mechanism and the rotary table; 21, an automatic viewpoint moving means; 22 a, a database storing data relating to the image input means and the adjusting mechanism; 23, an automatic image input selecting means; , 26
Is an image combining means.

The operation procedure of the CCTV system will be described. The user uses the viewpoint input means 20 to input necessary viewpoint positions, viewing directions, and viewing angles of images and sounds.
For example, a part to be viewed is specified on a map, and the height of the viewpoint is input as 1.5 meters above ground. Further, in order to view a continuous object over a wide area, the automatic viewpoint moving means 21 is used to set the viewpoint position to be automatically moved along the object. It is possible to simulate walking or patrol by car according to the moving speed at that time. The input viewpoint position, line-of-sight direction, and line-of-sight angle are transmitted to the image input automatic selection means 23 through the communication path 1.

The image input automatic selection means 23 inquires one or a plurality of image input means 10 capable of photographing a virtual image range to the database 22a through the communication path 2 to output an image at a specified viewpoint, and selects the image input means. I do. If the selected image input means has an adjusting mechanism, the automatic image input selecting means 23 sends the data relating to the designated viewpoint and the adjusting mechanism of the image input means to the adjusting mechanism automatic control means 24 through the communication path 5. Send out.

The adjusting mechanism automatic control means 24 calculates a value to be controlled and controls the adjusting mechanism 11 through the controller 15. This eliminates the need for the user to operate the controller 15 in the related art. Adjustment mechanism automatic control means 24
Returns the photographing position, photographing direction, and viewing angle of the image input means after control to the automatic image input selecting means 23 through the communication path 5. The image input automatic selection means 23 updates the value of the database 22a with the data after control via the communication path 2.

On the other hand, the image input automatic selection means 23 controls the video switcher 12 through the image input designator 14,
The image from the selected image input means is input to the viewpoint conversion processing means 25. This eliminates the need for the user to operate the image input designator 14 in the prior art. The specified viewpoint, the selected image input unit, and the data related to the image input unit, which are held by the image input automatic selection unit 23, are transmitted to the viewpoint conversion processing unit 25 through the communication path 3.
Sent to

The viewpoint conversion processing means 25 performs a viewpoint conversion process on the input image based on the data on the designated viewpoint and the image input means.

The viewpoint-converted image output from the viewpoint conversion processing means 25 is input to the image synthesizing means 26 together with the data relating to the designated viewpoint and the image input means and the parameters used for the viewpoint conversion processing. Are input, and are output to the monitor 13.

When the viewpoint is continuously changed by the automatic viewpoint moving unit 21, the above-described image input unit selection, adjustment mechanism control, viewpoint conversion processing, and synthesis processing are continuously continued. Therefore, the user can automatically advance the viewpoint position and grasp the target without performing any operation. Further, if the functions of the automatic viewpoint moving means 21 and the viewpoint input means 20 are released as necessary, and the image input designator 14 and the controller 15 are manually operated, the conventional CCTV system can be operated. In such a case, the user can operate the image input device while paying attention to the image input means, and can output an image that is not converted to a captured image.

FIG. 1 is a diagram showing a schematic configuration of a video output system according to a third embodiment of the present invention. In FIG. 1, 1
9 is a communication path, 16 is a digital image network transmitting images from the image input means, 17 is a digital audio network transmitting sounds from the audio input means,
Reference numeral 22b denotes a database storing data relating to the image input means, the adjustment mechanism, and the sound input means, 30 denotes a sound input means, 31 denotes an automatic sound input selection means, 32 denotes a sound processing means, and 33 denotes a speaker.

The user uses the viewpoint input means 20 to specify the required viewpoint position, line-of-sight direction, and visual angle of the image and sound. If necessary, the viewpoint position is set to be automatically moved along the target by using the automatic viewpoint moving means 21. The designated viewpoint position, line-of-sight direction, and line-of-sight angle are sent to the image input automatic selection means 23 through the communication path 1. The image input automatic selection unit 23 selects one or a plurality of image input units 10 capable of capturing a virtual image range by querying the database 22b through the communication path 2 in order to output an image at a specified viewpoint. If the selected image input means has an adjustment mechanism, the automatic image input selection means 23 sends data relating to the designated viewpoint and the adjustment mechanism to the adjustment mechanism automatic control means 24 through the communication path 5.

The adjusting mechanism automatic control means 24 calculates the value to be controlled and controls the adjusting mechanism 11, and determines the photographing position, the photographing direction and the visual angle of the image input means after control through the communication path 5. Return to 23. The image input automatic selection means 23 updates the value of the database 22b with the controlled image input data through the communication path 2.

The designated viewpoint, the selected image input unit number, and the data relating to the image input unit, which are held by the automatic image input selection unit 23, are sent to the viewpoint conversion processing unit 25 through the communication path 3.

Based on the transmitted image input means number, the viewpoint conversion processing means 25 extracts an image taken by the corresponding image input means from the digital image network 16 through the communication path 4, and specifies the designated viewpoint position and the like and the image input means. A viewpoint conversion process is performed based on the data on the means.

The image converted from the viewpoint and the like output from the viewpoint conversion processing means 25 is input to the image synthesizing means 26 together with the designated viewpoint position and the like, data relating to the image input means, and the parameters used for the viewpoint conversion processing. If a plurality of images are input, they are combined and output to the monitor 13.

FIG. 2 shows an example of image output. In FIG. 2, two surveillance cameras, a and b, are taken. The images follow viewpoints a to d, which are input, by outputting the viewpoints, converting and combining the images.

On the other hand, sound is output as follows. The viewpoint position and the line-of-sight direction designated by using the automatic viewpoint moving unit 21 and the viewpoint input unit 20 are transmitted to the sound input automatic selection unit 31 through the communication path 6. The automatic sound input selecting means 31 transmits one or more sound input means 30 near the specified viewpoint position to the database 2 through the communication path 7 in order to output sound at the specified viewpoint position and the line of sight.
Inquire to 2b to select. Sound input automatic selection means 3
The sound input means number selected by the user 1 and the designated viewpoint position and line-of-sight direction are transmitted through the communication path 8 to the sound processing means 32.
Sent to The sound processing means 32 extracts sound input by the corresponding sound input means from the digital sound network 17 through the communication path 9 based on the sent sound input means number, and performs sound processing based on the specified viewpoint position and line-of-sight direction. Do.

If the sound is transmitted as a conventional analog signal, a sound switcher is provided between the sound processing means 32 and the sound input means 30, and the sound input automatic selection means 31 controls the sound switcher. A required acoustic signal can be obtained.

The sound processing means 32 adjusts the volume and delay based on the distance between the designated viewpoint position and the sound input means, and outputs sound to the speakers 33 around the user corresponding to the line of sight.

As described above, according to the third embodiment, it is possible to output a sound linked to an image, so that a high sense of realism can be obtained. This makes it easier to grasp the target.

As described above, the present invention has been specifically described based on the embodiments. However, it is needless to say that the present invention is not limited to the above-described embodiments, and can be variously changed without departing from the gist thereof.

[0075]

The video output system according to the present invention has the following effects for each invention. According to the first aspect, it is possible to output an image at a viewpoint position, a line-of-sight direction, and a viewing angle required by the user, so that the target can be easily grasped and recognized. According to the second aspect, since a plurality of images can be synthesized, it is not necessary to re-understand the images due to switching of the image input means, and it is particularly easy to grasp and recognize continuous objects.
Further, since the selection of the image input means and the image synthesis can be automated, the operation of the user can be reduced. According to the third aspect, a fine image with less discomfort can be obtained by using the adjustment mechanism provided in the image input means, and the operation can be reduced by automation. According to the fourth aspect, since the viewpoint position, the line-of-sight direction, or the viewing angle is automatically changed, the number of user operations can be reduced. According to the fifth aspect, since the sound associated with the image can be output, the presence can be enhanced and the target can be effectively grasped.

[Brief description of the drawings]

FIG. 1 is a diagram showing a schematic configuration of Embodiment 3 of a video output system according to the present invention.

FIG. 2 is a diagram illustrating an example of an image obtained by using the system of FIG. 1;

FIG. 3 is a diagram illustrating a viewpoint conversion process.

FIG. 4 is a diagram for explaining a virtual image range.

FIG. 5 is a diagram showing a schematic configuration of an example of a conventional CCTV system.

FIG. 6 is a diagram showing a schematic configuration of Embodiment 1 of a video output system according to the present invention.

FIG. 7 is a diagram showing a schematic configuration of a second embodiment in which the present invention is applied to FIG. 5;

[Explanation of symbols]

1-9 Communication path 10 Image input means 11 Adjustment mechanism 12 Video switcher 13 Monitor 14 Image input designator 15 Controller 16 Digital image network to which image of image input means is transmitted 17 Sound of sound input means is transmitted Digital acoustic network 20 View point input means 21 Automatic viewpoint moving means 22a Database on image input means and adjustment mechanism 22b Database on image input means, adjustment mechanism and sound input means 23 Image input automatic selection means 24 Adjustment mechanism automatic control means 25 View point conversion processing Means 26 Image synthesis means 30 Sound input means 31 Sound input automatic selection means 32 Sound processing means 33 Speaker

Claims (5)

[Claims] A viewpoint input means for inputting a viewpoint position, a line-of-sight direction and a viewing angle, and a viewpoint conversion processing unit for inputting an image, converting the image into an image at the viewpoint position, the line-of-sight direction and the viewing angle, and outputting the image. Video output system characterized by the following. 2. An image output system comprising a plurality of image input means, an image input data storage means for storing data relating to a shooting position, a shooting direction, and a viewing angle of the plurality of image input means; Viewpoint input means for inputting a viewing angle, and image input for selecting one or a plurality of image input means for outputting an image at the viewpoint position, the viewing direction, and the viewing angle using the image input data storage means. Automatic selection means, viewpoint conversion processing means for inputting an image from one or a plurality of selected image input means, converting the image into an image at the viewpoint position, the viewing direction, and the viewing angle, and outputting the converted image; An image synthesizing means for synthesizing and outputting a plurality of images. 3. An automatic viewpoint moving means for automatically changing a viewpoint position, a line-of-sight direction or a visual angle.
The video output system according to 1. 4. An image output system in which an image input means includes an adjustment mechanism for changing an enlargement / reduction ratio or a photographing direction of an image, wherein a moving amount of a pixel before and after a viewpoint conversion and an overlapping pixel in image synthesis are provided to the adjustment mechanism. 4. An adjusting mechanism automatic control means for performing control for inputting an image necessary for image output at the input viewpoint position, line-of-sight direction, and visual angle at the viewpoint. Video output system. 5. A plurality of sound input means, a sound input data storage means for storing data relating to a position of the sound input means, and a sound at an input viewpoint position using the sound input data storage means. Sound input automatic selection means for selecting one or more sound input means for outputting, and inputting sound from the selected one or more sound input means, converting the sound into the sound at the input viewpoint position, 3. The video output system according to claim 2, further comprising: an audio processing unit that outputs an output corresponding to the input line-of-sight direction.