JPH1188767A - Video processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video processing system capable of having a user instinctively recognize correspondence relation between a standard video and a wide angle video. SOLUTION: When a wide angle image is processed by this system, the standard video 1a is picked up with optional photographing magnification by a photographing means 1. In addition, image data 3a segmented from the standard video 1a is stored by a storage means 3. The wide angle image 4a is generated from the image data 3a by a wide angle image generation means 4. The wide angle image 4a and the standard image 1a are displayed by an image display means 5. Input of instruction of a photographing condition including the photographing magnification is accepted for the standard video 1a by a photographing condition instruction means 6. The photographing means 1 is activated according to the photographing condition by a photographing condition execution means 7. And a wide angle image updating instruction is outputted so that display magnification of the wide angle image 4a is changed by being synchronized with the standard video 1a by a wide angle image updating means 8. Therefore, the wide angle image 4a is also changed by being synchronized with a change of the standard video 1a.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a video processing system, and more particularly to a video processing system for processing a wide-angle image.

[0002]

2. Description of the Related Art Conventionally, for the purpose of observation, monitoring, guidance, etc.,
There is a demand that a photographed image be obtained from a video camera or the like installed in a remote place only by operating a computer.

In response to such a demand, there is a technique for connecting a photographing device such as a video camera to a computer via a network or the like. The user does not need to go to the place where the video camera is installed to perform the operation, and may operate the computer that has been connected to the camera and set.

Generally, in such a video processing system,
It is necessary to acquire not only standard images but also wide-angle images (panoramic images). In addition, a function for enlarging and displaying a position designated by the user is also required.

By the way, in order to obtain a panoramic image with a video camera or the like, an expensive wide-angle lens had to be used before. At present, however, a method has been considered in which a panoramic image is created by performing image processing on a standard video photographed with a standard lens to reduce the cost. In this specification, “video” means a moving image, and “image” means a still image.

In a video processing system utilizing this method, a plurality of standard images previously photographed with a standard lens are linked to create and display a panoramic image. In addition to the panoramic image, a live video image captured by the imaging device at that time is also displayed. When there is a portion in the panoramic image to be monitored at a different magnification, if the portion and the desired magnification are designated, the video of the designated portion at the designated magnification is displayed as a live image.

[0007]

However, in the above-described video processing system, the display magnification of the panoramic image cannot be changed, and it is difficult for the user to grasp the correspondence between the standard video and the panoramic image. .

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and has as its object to provide a video processing system that changes a wide-angle image in synchronization with a change in a standard image.

[0009]

According to the present invention, in order to solve the above-mentioned problems, in a video processing system for processing a wide-angle image, a photographing means for photographing a standard image at an arbitrary photographing magnification; Storage means for storing the captured image data, wide-angle image creation means for creating a wide-angle image from the image data, image display means for displaying the wide-angle image and the standard video, and shooting magnification for the standard video. A photographing condition instructing unit for receiving an instruction input of a photographing condition; a photographing condition executing unit for operating the photographing unit in accordance with the photographing condition; and a wide-angle image updating so as to change a display magnification of the wide-angle image in synchronization with the standard image. And a wide-angle image updating means for outputting a command.

When a wide-angle image is processed by such an image processing system, the photographing means photographs a standard image at an arbitrary photographing magnification. The storage means stores image data cut out from the standard video. The wide-angle image creating means creates a wide-angle image from the image data. The image display means displays a wide-angle image and a standard image. The photographing condition instructing means receives an instruction input of a photographing condition including a photographing magnification for a standard image. The photographing condition executing means operates the photographing means according to the photographing conditions. Then, the wide-angle image updating unit outputs a wide-angle image update command so as to change the display magnification of the wide-angle image in synchronization with the standard image.

As described above, in the image processing system of the present invention, a wide-angle image is generated from a standard image, and the displayed wide-angle image is updated in synchronization with a change in shooting conditions for the standard image.
The user can intuitively recognize the correspondence between the standard image and the wide-angle image.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the principle configuration of the video processing system of the present invention.

The image processing system according to the present invention comprises:
, Driving means 2, storage means 3, and wide-angle image creating means 4
, Image display means 5, photographing condition instructing means 6, photographing condition executing means 7, and wide-angle image updating means 8.

The photographing means 1 is a video camera or the like having a standard lens capable of photographing at an arbitrary photographing magnification.
The standard image 1a is photographed. Further, the driving unit 2 moves or rotates the photographing unit 1. The storage means 3
The image data 3a cut out from the standard video 1a is stored. The wide-angle image creating means 4 creates a wide-angle image 4a from the image data 3a. The image display means 5 outputs the wide-angle image 4
a and the standard image 1a are displayed. Shooting condition indicating means 6
Receives an instruction input of a shooting condition including a shooting location and a magnification for the standard video 1a. Photographing condition executing means 7
Drives the driving means 2 and the photographing means 1 according to photographing conditions. Then, the wide-angle image updating means 8 outputs a wide-angle image update command so as to change the center position and the magnification of the wide-angle image 4a in synchronization with the standard image 1a.

In the video processing system of the present invention, the standard video 1
When the photographing condition for a is received, the wide-angle image 4a is updated in synchronization, so that the user can intuitively recognize the correspondence between the standard image 1a and the wide-angle image 4a.

FIG. 2 is an overall configuration diagram when the video processing system of the present invention is realized by a computer. The video processing system according to the present invention is roughly composed of a photographing unit 10 as a video camera, a camera driving unit 20, a camera control unit 30, a network 40, and a computer 50.

The photographing unit 10 corresponds to the photographing means 1 shown in FIG. 1, and includes a lens block 11, a CCD (Charge C).
oupled Device) 12, SH (Sample Hold) / AGC
(Automatic Gain Control) 13, A / D converter 14,
It has a signal processing unit 15 and a motor 16. Then, a video signal is generated.

The camera drive section 20 corresponds to the drive means 2 shown in FIG. 1, and has a rotary head 21, a pan motor 22, and a tilt motor 23. Then, the photographing unit 10 is moved or rotated.

The camera control unit 30 corresponds to the photographing condition execution means 7 shown in FIG.
, A pan tilter controller 32 and a mode controller 33. The camera control unit 30 is connected to the computer 50 via the network 40, and controls the photographing unit 10 and the camera driving unit 20 according to a control command transmitted from the computer 50.

The computer 50 has a video capture 5
1, a data storage unit 5 corresponding to the storage unit 3 shown in FIG.
2, an image processing unit 53 corresponding to the wide-angle image creating unit 4 shown in FIG. 1, a CPU 54 corresponding to the photographing condition instruction unit 6 and the wide-angle image updating unit 8 shown in FIG. 1, and an image display unit 5 shown in FIG. Is provided. In addition,
The user inputs an instruction to the video processing system from input means (not shown) such as a keyboard and a mouse connected to the computer 50. Also, the monitor 55
The live video display unit 55a and the panoramic image display unit 55b are separately prepared. Note that the live video described below refers to the standard video described in FIG. 1, and the panoramic image refers to a wide-angle image.

Here, the lens block 11 of the photographing unit 10
Is constituted by a lens and an aperture, and forms an image of the imaging light reaching from the subject on the CCD 12. The lens block 11 will be described later in detail. CCD12
Converts a visual field image into a video signal and outputs the SH / AGC13
To enter. The SH / AGC 13 samples and holds the input video signal, and controls the gain so as to have a predetermined gain by the control signal of the art iris. The A / D converter 14 converts analog data into digital data. Then, the signal processing unit 15 converts the input digital video data into a luminance signal, a color difference signal, and a video signal, and transmits the signals to the video capture 51 of the computer 50.

Describing the lens block 11 again, the lens block 11 is a zoom lens that can change the angle of view by driving a built-in zoom lens. The motor 16 is a stepping motor that drives the zoom lens according to a driving command from a camera controller 31 described later.

The rotating head 21 of the camera driving unit 20 has two degrees of freedom in the directions of rotation such as pan and tilt, and moves and rotates the photographing unit 10 installed on the rotating head 21. Further, the pan motor 22 and the tilt motor 23 are respectively provided with a pan tilter controller 32 described later.
Is a stepping motor that drives the rotary head 21 in accordance with a drive command from the camera.

The camera controller 3 of the camera control unit 30
Reference numeral 1 denotes a controller that constantly performs lens control (focus, zoom), exposure control (aperture, gain, electronic shutter speed), white balance control, image quality control, and the like of the imaging unit 10. The camera controller 31 interfaces with the mode controller 33 and the image capturing unit 10 for controlling the zoom lens of the lens block 11 of the image capturing unit 10. That is, while constantly notifying the mode controller 33 of the lens position of the zoom lens for zooming, and receiving a drive request for the zoom lens sent from the mode controller 33, the zoom lens is driven to the requested position. Thus, a control signal is output to the driver of the motor 16.

The pan / tilt controller 32 of the camera control unit 30 also interfaces with the mode controller 33 and the camera drive unit 20. That is, the position and angle of the rotating head 21 are constantly notified to the mode controller 33, and the rotating head 21 is driven in the pan and tilt directions sent from the mode controller 33. A control signal is output to the motor driver of the pan motor 22 and the tilt motor 23 so as to be driven to the requested position.

The mode controller 33 is connected to the computer 50 via the network 40, and controls the photographing unit 10 via the camera controller 31 as described above. Further, it controls the camera drive unit 20 via the pan tilter controller 32. That is, the position of the zoom lens of the lens block 11 is transmitted from the camera controller 31, and the rotation
The position and angle of 1 are always notified.

By the way, what is input from the computer 50 to the mode controller 33 is a request for driving the absolute position of the video signal. Therefore, the mode controller 33
This absolute position drive request is provided by a drive command for the rotary head 21,
The command is distributed to the driving command of the zoom lens of the lens block 11 and input to the camera controller 31 and the pan tilter controller 32, respectively.

As the network 40, here, the RS-
232C is used. The video capture 51 of the computer 50 converts the video signal input from the signal processing unit 15 of the photographing unit 10 to the monitor 5 with an arbitrary quality.
5 is displayed in the standard video display section 55a. In addition, the image data is converted into an arbitrary image format (a still image in a bitmap format, a JPEG format, or the like) with an arbitrary quality by a capture signal, and is recorded in the data storage unit 52. The data storage unit 52 stores a video signal and an image signal input from the video capture 51.

The image processing section 53 acquires an image from the data storage section 52 and generates a panoramic image from the data. Then, this is displayed on the panoramic image display unit 5 of the monitor 55.
5b. When generating a panoramic image, an image may be directly obtained from the video capture 51 and used.

The CPU 54 controls all the internal components and sends instructions input by the user to the image processing unit 5.
Notify 3. The communication with the mode controller 33 via the network 40 is performed.

FIG. 3 is a diagram showing an example of a display screen displayed on the monitor 55 in the video processing system shown in FIG.
On the monitor 55, a live video display unit 55a and a panoramic image display unit 55b are separately arranged. An operation button group 55c is provided on the display screen so that the photographing position and the photographing magnification can be operated with a pointing device such as a mouse. Further, a frame 55d indicating the position of the live video is shown in the panoramic image display section 55b.

FIG. 4 is a diagram showing another example of a display screen displayed on the monitor 55 in the video processing system shown in FIG. The monitor 55 has a panoramic image display unit 55b
And a live video display unit 55a is provided therein. Further, an operation button group 55c is prepared.
Here, a procedure for processing a video signal in the video processing system shown in FIG. 2 and providing a user with a panoramic image and a standard video will be described.

FIG. 5 is a flowchart showing a procedure for processing a video signal in the video processing system shown in FIG. Hereinafter, description will be given along the step numbers. [S1] The display screen displayed on the monitor 55 is initialized.
The image capturing unit 10 captures a video that can be captured at that time, and the captured video is displayed on the live video display unit 55a. [S2] The CPU 54 of the computer 50 sets a timer and a timer event in order to periodically communicate with the mode controller 33. [S3] Wait for an event to occur, and if an event occurs, go to step S4. [S4] The type of the generated event is checked. If the generated event is a timer event, the process proceeds to step S5. If the generated event is a panorama creation request event, the process proceeds to step S6. If the generated event is a panorama zoom event, the process proceeds to step S7. [S5] Timer event processing is performed. The timer event processing will be described later. [S6] T is set in the panorama creation request flag. [S7] A panorama zoom event process is performed. The panorama zoom event processing will be described later. [S8] It is determined whether or not the operation on the video processing system has been completed. If the operation is completed, the processing of this flowchart is also completed. If the operation is not completed, the process proceeds to step S3 again.

FIG. 6 is a flowchart showing the procedure of the timer event process shown in FIG. The timer event is an event that occurs every set time. Hereinafter, description will be given along the step numbers. [S11] The CPU 54 sets the communication port RS232
It is determined whether the setting of C has been completed. If the communication port setting has not been completed, the process proceeds to step S12, and if the communication port setting has been completed, the process proceeds to step S13. [S12] The CPU 54 sets communication ports. The first timer event in the case where the setting of the communication port has not been completed, for example, immediately after the power is turned on to the video processing system, ends here. [S13] The CPU 54 determines whether or not the video capture 51 has received data. If there is received data, go to step S14. If there is no received data, go to step S14.
Proceed to 15. [S14] The CPU 54 analyzes the received data. That is, the position information of the zoom lens in the lens block 11 and the position information of each of the pan and tilt directions of the rotary head 21 included in the received data are acquired. Then, zoom magnification information and pan / tilt angle information are calculated from the information. [S15] The CPU 54 determines whether or not the transmission request flag that is T when the data to be transmitted is stored in the transmission buffer is T. If the transmission request flag is T, the flow advances to step S16 to set the transmission request flag to T.
If not, the process proceeds to step S17. [S16] The CPU 54 transmits the data stored in the transmission buffer to the mode controller 33. When the transmission is completed, the transmission request flag is set to F, and the timer event ends. [S17] The CPU 54 determines whether the value of the internal counter C is 0, 1, or 2. If the value of the internal counter C is 0, the process proceeds to step S18, if it is 1, the process proceeds to step S19, and if it is 2, the process proceeds to step S21. [S18] The CPU 54 causes the mode controller 33 to
Communication data including an absolute position drive request for driving the pan motor 22 and the tilt motor 23 of the camera drive unit 20 is transferred. [S19] The CPU 54 causes the mode controller 33 to
Communication data including an absolute position drive request for driving a motor 16 for driving a zoom lens included in the lens block 11 is transferred. [S20] The CPU 54 adds 1 to the value of the internal counter C and ends the timer event. [S21] The CPU 54 sets the panorama creation request flag to T
It is determined whether or not is standing. If T is set in the panorama creation request flag, step S22
If T is not standing, go to step S23.
move on. [S22] The CPU 54 controls the image processing unit 53 to
Perform panorama creation processing. The panorama creation processing will be described later. [S23] The CPU 54 sets the value of the internal counter C to 0 and ends the timer event.

Next, the procedure of the panorama creation process will be described. FIG. 7 is a flowchart illustrating the procedure of the panorama creation process. [S31] The CPU 54 determines whether the value of the internal counter P is 1, 2, or 3 or more. If the value of the internal counter P is 1, the process proceeds to step S32, where
If so, go to step S34. If it is 3 or more, go to step S36.
Go to [S32] The CPU 54 causes the mode controller 33 to
A driving command is transmitted to the motor 16 for driving the zoom lens included in the lens block 11 so that the imaging magnification of the image captured by the imaging unit 10 becomes the set value. [S33] The CPU 54 adds 1 to the value of the internal counter P and proceeds to step S31 again. [S34] The CPU 54 causes the mode controller 33 to
A drive command is transmitted to the pan motor 22 and the tilt motor 23 of the camera drive unit 20 so that the shooting range of the video shot by the shooting unit 10 is set to a position where image processing is performed in the panorama image creation process. I do. [S35] The CPU 54 adds 1 to the value of the internal counter, and proceeds to step S31 again. [S36] The CPU 54 confirms from the transmission data from the mode controller 33 that the shooting magnification matches the set value. If the shooting magnification does not match the set value, the process proceeds to step S31 again. [S37] The CPU 54 confirms from the transmission data from the mode controller 33 that the shooting range is set at a position where image processing must be performed in the panorama image creation processing. If the shooting range has not been set correctly, the process proceeds to step S31 again. [S38] The image processing unit 53 cuts out an image from the video signal being received by the video capture 51 at this time. [S39] The CPU 54 causes the mode controller 33 to
A driving command for the pan motor 22 and the tilt motor 23 of the camera driving unit 20 is set so that the photographing range of the video photographed by the photographing unit 10 is set to the position where the next image processing is performed in the panoramic image creation processing. Send [S40] The image processing unit 53 performs horizontal and vertical reduction processing on the image cut out in step S39. Also,
If there is an image that has already been reduced,
The newly reduced image is linked to the already reduced image. The image reduction processing and connection will be described later in detail. [S41] The image processing unit 53 displays the reduced and connected image on the panoramic image display unit 55b of the monitor 55. [S42] The CPU 54 adds 1 to the value of the internal counter P. [S43] The CPU 54 determines whether or not the image displayed in step S41 is a full screen, that is, whether or not a full-screen panoramic image has been completed at this time. If it is determined that the full-screen panorama image has been completed, step S44
Proceed to. If it is determined that the full-screen panoramic image has not been completed, the process proceeds to step S31 again. [S44] The CPU 54 sets the panorama creation request flag to F
To [S45] The data storage unit 52 stores the generated full-screen panoramic image.

Here, a method of reducing and connecting images will be described. In addition, enlargement, not reduction, is possible in the same manner. Magnification and alpha ₁ for reducing the original image before, beta ₁
= 1 / α ₁ , γ ₁ = x × β ₁ (1 ≦ x ≦ the number of pixels in the X direction of the original image, x is a natural number), δ ₁ = γ ₁ −int (γ ₁ )
(Int (A) is an integer less than or equal to A), each pixel g in the X direction of the reduced image from each pixel f (x) of the original image
(X) is calculated by the following equation (1). In the video processing system of the present invention, the image is reduced in the X direction using the equation (1).

[0037]

(Equation 1)

Next, the same processing is performed on the pixel f (y) in the Y direction of the image collapsed in the X direction. That is, α
₂ = Yn / Yu, β ₂ = 1 / α ₂ , γ ₂ = y × β ₂ (1
≤ y ≤ the number of pixels in the Y direction of the original image, y is a natural number), δ ₂ =
When γ ₂ -int (γ ₂ ), each pixel f of the original image
Each pixel g (y) in the Y direction after reduction from (y) is calculated by the following equation (2).

[0039]

(Equation 2)

Here, the state of zooming in a panoramic image created by the procedure described above will be described.
In the following description, the following panoramic image K is targeted.

FIG. 8 is a diagram showing a state in which a target panoramic image is displayed on the monitor 55 in explaining the zooming of the panoramic image. A panoramic image H is formed by connecting p original images G1 to Gn (X0 × Y0 pixels) photographed at a photographing magnification m0 in the X direction and q sheets in the Y direction. This panoramic image H is scaled by a (0 <a ≦
The image reduced in 1) is a panoramic image K (a ² × X0 × p
× Y0 × q pixels) and the panoramic image display unit 55b
Will be displayed on the entire screen.

At this time, the live video display 55a
, The live video L is displayed at a reduced magnification b (0 <b ≦ 1). When the live video L is embedded in the panoramic image K, a = b.

FIG. 9 is a diagram showing a state of panoramic image enlargement. Panoramic image region R1 (Xu × Yu pixels)
The user who has designated the area R1 with a mouse or the like
Attempting to enlarge to 2 (Xn × Yn pixels) causes a panorama zoom event. In the panorama zoom event, the entire panorama image is enlarged in conjunction with the enlargement from the region R1 to the region R2.

If the panoramic image K shown in FIG. 8 is enlarged, the panoramic image K1 ((a ² × X0 × p × Y0)
× q) × (Xn / Xu) × (Yn / Yu) pixels) are created. However, since a portion of the panoramic image K1 that does not fit in the panoramic image display unit 55b on the monitor 55 cannot be displayed, a scroll bar or the like may be provided as shown in the figure. Note that, at this time, each pixel has a magnification α ₁ = X in Expression (1).
Determined by applying n / Xu.

It should be noted that only the original images G1 to Gn and the coordinate information necessary for the connection are stored, and the panoramic images H, K, K
1 may be generated as needed. Also, here, a case has been described where a panorama zoom event occurs along with the enlargement of the designated area in the panorama image,
The panorama zoom event may be generated when the magnification of the panorama image is set by the operation button group 55c as shown in FIG.

Further, when the panoramic image is scrolled, the rotary head 21 may be driven in conjunction with the movement. FIG. 10 is a diagram showing another aspect of the panoramic image enlargement.

When the operation buttons are prepared as shown in the figure, the magnification of the live video L can be changed from m0 to m1 with a pointing device such as a mouse. When the magnification changes, the angle of view changes, but the number of photographed pixels X0 × Y0 does not change. Therefore, in the panorama zoom event that occurs at this time, a panorama image K2 having (m1 / m0) ² times the number of pixels of the panorama image H is generated.

Conversely, a magnification change may be calculated from a panoramic image enlargement operation to drive the zoom lens. As described above, when the conditions regarding the shooting magnification and the shooting position for the live video are received, the panoramic image is updated in synchronization with the conditions. Therefore, the user can intuitively recognize the correspondence between the live video and the panoramic image.

[0049]

As described above, in the video processing system of the present invention, when a shooting condition for a standard image is received, a wide-angle image is updated in synchronization with the standard image. When an image is displayed, a wide-angle image with a high magnification centered on the standard image is displayed, and the user can intuitively recognize the correspondence.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the principle configuration of a video processing system according to the present invention.

FIG. 2 is an overall configuration diagram when the video processing system of the present invention is realized by a computer.

FIG. 3 is a diagram illustrating an example of a display screen displayed on a monitor in the video processing system illustrated in FIG. 2;

4 is a diagram showing another example of a display screen displayed on a monitor in the video processing system shown in FIG.

FIG. 5 is a flowchart showing a procedure for processing a video signal in the video processing system shown in FIG. 2;

FIG. 6 is a flowchart illustrating a procedure of a timer event process illustrated in FIG. 5;

FIG. 7 is a flowchart illustrating a procedure of a panorama creation process.

FIG. 8 is a diagram illustrating a state in which a target panoramic image is displayed on a monitor when explaining zooming of a panoramic image.

FIG. 9 is a diagram showing a state of a panoramic image enlargement.

FIG. 10 is a diagram showing another aspect of the panoramic image enlargement.

[Explanation of symbols]

1 ... photographing means, 1a ... standard video, 2 ... driving means,
3 ... storage means, 3a ... image data, 4 ... wide-angle image creation means, 4a ... wide-angle images, 5 ... image display means, 6
...... photographing condition instructing means 7, photographing condition executing means 8,
... Wide-angle image updating means.

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Naoyasu Hosunuma 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Ken Tamayama 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo No. Sony Corporation

Claims (4)

[Claims] 1. An image processing system for processing a wide-angle image, a photographing means for photographing a standard image at an arbitrary photographing magnification, a storage means for storing image data cut out from the standard image, and a wide-angle image from the image data. A wide-angle image creating unit that creates an image; an image display unit that displays the wide-angle image and the standard image; a shooting condition instruction unit that receives an instruction input of a shooting condition including a shooting magnification for the standard image; A photographing condition executing means for operating the means in accordance with the photographing conditions; and a wide-angle image updating means for outputting a wide-angle image updating command to change the display magnification of the wide-angle image in synchronization with the standard image. Video processing system. 2. The image processing apparatus according to claim 1, further comprising a display condition instructing unit configured to receive an instruction input of a display condition including a display magnification for the wide-angle image, wherein the photographing condition executing unit operates the photographing unit in accordance with the display condition. The video processing system according to claim 1, wherein: 3. A driving unit for moving and rotating the photographing unit, and when the photographing condition includes a photographing place designation, the photographing condition executing unit operates the driving unit according to the photographing condition. The video processing system according to claim 1, wherein: 4. A display condition instructing unit for receiving an instruction input of a display condition including a display location designation for the wide-angle image, wherein the photographing condition executing unit operates the driving unit in accordance with the display unit. Claim 3
The video processing system as described.