JPH10200807A - Camera control method using graphical user interface - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the camera control method with good operability for a videoconferencing system, a remote monitor system, etc. SOLUTION: A window for displaying video on a display is divided into five areas 41, 42, 43, 44, and 45 and when each area, e.g. the camera right movement area 41 is clicked, a right movement command is sent to a camera. When the button of the mouse is released, a camera stop command is sent. Similarly, the area 42 is a camera upward movement area, the area 43 is a camera right movement area, the area 44 is a downward movement area, and the area 45 a zoom up/down area. When the mouse cursor 22 is in the camera right movement area, the cursor changes in shape to a right arrow to let the user know distinctly which area the mouse cursor 22 is in. Camera movement, zoom-in/out operation, etc., become possible by one operation of the mouse, etc., to obtain a man-machine interface with good operability.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention displays an image captured by a camera on a display such as a personal computer monitor, and performs communication, dialogue, monitoring, and the like based on the image. It is used in a system that performs camera control such as tilt, zoom, and the like. For example, the present invention relates to an operation method and a control method of camera control of a local or remote party of a TV conference system. In addition, the present invention relates to an operation method and a control method of camera control of a surveillance system that performs surveillance using an image of a remote camera.

[0002]

2. Description of the Related Art In a TV conference system, a remote monitoring system, etc., the direction (pan, pan,
The ability to control tilt (tilt) and zoom (telephoto, wide-angle) is a useful function. 2. Description of the Related Art Conventionally, as an operation method for controlling a camera on a remote side, there are four operation methods of an upward direction, a downward direction, a right direction, and a left direction corresponding to a moving direction of a camera on an operation device or a console of a TV conference system or a remote monitoring system. A button or a control stick tilted in each direction is provided and operated to control pan and tilt of the remote camera. In general, a method is provided in which a button corresponding to wide-angle and telephoto is provided on an operation device, and zoom control of a remote camera is performed by operating the button. An example of this is disclosed in Japanese Patent Application Laid-Open No. 1-258576.

As a method for improving the operability of the remote camera, there is Japanese Patent Application Laid-Open No. HEI 3-129899. In this example, remote control is performed by transmitting a full view image of the conference room on the remote side and displaying the camera direction and how the zoom was specified to the remote camera in a window shape on the full view image. Operability is improved. further,
Japanese Patent Laying-Open No. 7-135594 discloses a technique for controlling a camera using a pointing device.

[0004]

The conventional method has the following two problems. First, as a first problem, when zooming up a certain portion of the image currently being viewed, in the conventional method, first, the user operates the buttons in the up, down, left, and right directions or the control stick to first zoom in on the portion to be zoomed in. Display in the center of the screen. (That is, the camera is pointed at the point to be zoomed in.) Then, by operating the telephoto button to zoom up the camera, a two-step operation of zooming in on a desired portion must be performed. Operability was not good.

[0005] Next, as a second problem, in the former example of the prior art, the relative direction to which the camera is currently facing, There is a problem that the relative range being photographed is not known. In particular, in a monitoring system or the like, the operability is not good because it is not clear which direction in the range that can be monitored.

In the latter example of the prior art, as a countermeasure against this problem, a method is employed in which a panoramic image of the opposing conference room is transmitted and information indicating where the image is being viewed is displayed. However, in this method, first, it is necessary to transmit the whole scene video of the subject (monitored) side, and a cost for that is required. In addition, a display device and a display area for displaying a panoramic view image and an image indicating the camera direction are required separately from an area for displaying an image being monitored and monitored, which results in an increase in the size of the apparatus and an increase in cost.

In Japanese Patent Application Laid-Open No. Hei 7-135594, a camera is controlled by a pointing device. For example, when a specific area is photographed, the camera is controlled so that a specified angle of view and a display screen match. Control. The present invention provides a camera control method using a graphical user interface with better operability.

A first object of the present invention is to directly click on a desired portion of a displayed image with a pointing device such as a mouse cursor or the like, thereby zooming in or zooming out the desired portion by one-touch operation. The goal is to provide an intuitive and easy-to-understand man-machine interface that goes down.

A second object of the present invention is to indicate a direction and a range currently being viewed in a whole range that can be monitored by a camera by using a narrow range along a region for displaying a video being communicated. The goal is to provide a man-machine interface that can.

[0010]

SUMMARY OF THE INVENTION According to the present invention, a dialogue and a monitoring video are displayed on a screen of a personal computer or the like equipped with an operating system supporting a graphical user interface (GUI), and the user wants to view the video in the video. When the user clicks on a part with a mouse cursor or the like, the direction of moving the camera is calculated from the coordinates on the screen at the click, and camera control for panning, tilting, and zooming of the remote camera is performed based on the calculated value. Is provided.

In addition, a portion (horizontal and vertical scroll bars) indicating a relative direction with respect to the rotation range of the camera is added to an area along the upper or lower side of the window for displaying an image and along the left or right side. The direction and shooting range are shown to the user. Conversely, when the user operates the scroll bar with a mouse or the like, the camera rotation angle is calculated from the operation, and pan and tilt control of the remote camera is performed based on the calculated value.

The operation using the scroll bar described above is a general operation method in an application using a GUI, and the camera can be controlled by an operation similar to the operation. Therefore, the operation is a general and easy-to-understand camera control method. You can zoom up and down by directly clicking the part you want to see with the mouse,
One-touch operation enables zoom operation. In addition, an intuitive and easy-to-understand man-machine interface can be provided. Since the direction of the camera and the area being photographed can be known from the scroll bar at the end of the window, the camera direction and the photographing range can be indicated without using a particularly large display area. In addition, since the scroll bar can be controlled by performing the same operation as a general application, a general and easy-to-understand man-machine interface can be provided.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below in detail. FIG. 1 is a configuration diagram of a typical personal computer TV conference system that can be considered when the present invention is applied to a personal computer TV conference system.

In FIG. 1, reference numeral 4 denotes a personal computer (hereinafter, referred to as a personal computer). In this configuration example, 9 TV conference boards and 5 video overlay boards are mounted on the personal computer. The personal computer 4 has an operating system having a graphical user interface (GUI).

Reference numeral 1 denotes a self-side camera, which takes a picture of yourself. Reference numeral 10 denotes a local microphone for inputting a local voice. The audio and video input by the own camera 1 and the own microphone 10 are encoded by the TV conference board 9 and multiplexed with a data channel for controlling the other camera. The other party's TV via the ISDN network
It is transmitted to the conference system.

Similarly, the other party camera 1a and the other party microphone 1
Audio and video input from Oa are also encoded, multiplexed with camera control data and the like, and transmitted to the local TV conference board 9. Voice received from the other party's TV conference board 9a,
The video and camera control data are separated and decoded by the local TV conference board 9, and the audio is output and displayed on the local speaker 3 and the local monitor 3. Conference board 9a
The received camera control data is transmitted to the local camera platform 2 and performs camera control such as pan, tilt, and zoom of the local camera 1. Reference numeral 6 denotes a mouse, and 7 denotes a keyboard. By operating these input devices, a user can perform operations of the TV conference system, such as outgoing calls, incoming calls, and camera control, using a GUI. It is also possible to operate other application software that can operate on the personal computer 4.

In the configuration example shown in FIG. 1, the local system and the remote system have a symmetrical configuration, and the remote component corresponding to the local component is indicated by adding a suffix a to the number. , Each component has the same function.

FIG. 2 is a diagram showing a screen display of the monitor 3 on the own side. The screen 20 is a display screen of the monitor 3. On the screen 20, a video window 21 for displaying a communication video (video of the other party) of the TV conference, a mouse cursor 22 for pointing a point corresponding to an operation of the mouse 6, and a window for displaying an application other than the TV conference 23 is displayed.

FIG. 3 shows details of the video window 21 which is a feature of the present invention. Inside the window 21, an image 32 of the person A who is currently communicating is displayed. Also, this window includes a vertical scroll bar (24, 25, 26, 27) for controlling the camera on the other side or the own side and the camera platform 1, 2, 1a, 2a.
) And horizontal scroll bars (28, 29,
30, 31). Reference numerals 24 and 25 denote vertical camera movement buttons, which can be clicked with the mouse cursor 22 to move the camera on the own side or the other side by a very small amount.

Reference numeral 27 denotes a vertical camera moving unit, which can also move the camera by clicking the mouse cursor 22 here. However, when the user clicks 27, the camera movement is larger than when the user clicks 24 and 25. Reference numeral 26 denotes a vertical scroll box, which is moved by dragging the mouse cursor 22 with the mouse.
Camera can be moved. In this case, the camera movement amount moves in proportion to the drag movement amount. Further, the scroll box 26 indicates which range of the currently displayed image is within the entire range of the image that can be viewed by adjusting the pan, tilt, zoom, and the like. The horizontal scroll bars (28, 29, 30, 31) are the same as the vertical scroll bars (24, 25, 26, 27) except that the camera movement direction is horizontal.

FIG. 4 is a diagram showing the positional relationship between the other camera 1a, the camera platform 2a, and the person A32a who is the subject corresponding to the display image of FIG. FIG. 4 is a diagram in which the camera 1a and the subjects 32a, 33a, and 34a are viewed from above. Angle wdh is camera head 2
a is the maximum rotation range in the left-right direction. The central axis of the image of the camera 1a rotates right and left within the range of the angle wdh. Angle w
c is the range of the video currently being captured by the camera 1a.
(The person A32a is currently photographed up, and the image 32 of the person A is shown in the window 21.) The angle wah is when the camera is rotated to the left and right to the maximum extent with the zoom of the camera being the widest. Is the range to be photographed. That is, it is a range in which photographing is possible. Angle wbh
Is an angle indicating the current camera direction. Angle wbh
Is in the range of (wah-wdh) / 2 <= wbh <=
wah- (wah-wdh) / 2.

Although the above description relates to the horizontal camera movement, the same concept can be applied to the vertical camera movement. Therefore, in the following description, the description will be made only in the horizontal direction.

The relationship between the direction of the camera 1a, the overall length Lah of the scroll bar on the screen, the position Lbh of the scroll box 30, and the size Lch of the scroll box will be described below with reference to FIGS. Lah is the angle wah
It corresponds to. That is, it indicates the maximum range in the left-right direction that can be viewed by the camera 1a. Lch corresponds to the angle wc. That is, the range in the left-right direction in which the camera 1a is currently shooting is shown. Lah, Lch, wa
h and wc have a relationship of wc / wah = Lch / Lah. Lbh corresponds to the angle wbh. That is, it indicates the current direction of the camera 1a. Lbh,
For Lah, wbh, wah, wbh / wah = Lbh
/ Lah.

FIG. 5 is a flow chart showing the initialization processing when the present system actually operates and the processing flow when the pan and tilt camera movements are performed.

FIG. 6 is a diagram showing a case where the user operates the mouse 6 to click the right small camera movement button 28 and moves the camera rightward. FIG. 7 shows the directions of the cameras 1a, 2a and the subjects 32a, 33a, 3 corresponding to FIG.
It is the figure which showed the positional relationship of 4a.

The operation in the case where panning and tilting camera control is performed in accordance with the flow of FIG. 5 will be described below with reference to FIGS. 6 and 7. In the standby state (100), when the communication start event (101) occurs (when communication is started), each variable and constant are initialized (102),
The state transits to the operation event waiting state during communication (103).

First, the contents to be initialized in the initialization processing of 102 will be described. SmallChange is 24,
This is the amount by which Lbh and Lbv are changed when the minute camera movement buttons 25, 28 and 29 are clicked. La
rgeChange is an amount by which Lbh and Lbv are changed when the camera moving units 27 and 31 are clicked. The details of SmallZm and LargeZm will be described later. However, when the zoom is changed, the change amount is a small change amount and the change amount is a large change amount. Lah, Lav, wah, wdh, wb
h, wc, Lbh, Lch, wbv, etc. are the values already described, and these initial values are set.

Next, in the operation event waiting state (103), the user operates the mouse 6 as shown in FIG.
4, 25, 26, 27, 28, 29, 30, 31, etc., horizontal / vertical camera movement scroll bars are clicked, dragged, etc., and scroll box positions Lbh, Lb are performed.
When v changes, an Lbh change event (104) or an Lbv change event (105) occurs.

As described above, the Lbh, Lbh, and Lvh are determined by clicking the micro camera movement buttons 24, 25, 28, and 29, clicking the camera movement units 27 and 31, or dragging the scroll boxes 26 and 30. Lb
Although the way of changing the value of v is different, SmallChange
By initializing variables such as e, LargeChange, etc., the GUI operating system updates Lbh, Lbv to values corresponding to mouse operations, and
Generate a bv change event.

In the process (106), the camera angle wbh corresponding to the changed Lbh is calculated. Processing (10
In 8), the command “MOVEH wbh” for moving the camera to the camera angle wbh obtained in (106) is transmitted to the local T
It is transmitted to the V conference board 9 and transits to the operation event waiting state (103) again. The conference board 9 multiplexes the received camera control command together with audio and video data and transmits the multiplexed command to the partner TV conference board 9a. The other party's TV conference board 9a separates the camera control data from the multiplexed received data together with the audio and video data, and transmits the received camera control command to the camera platform 2a. The camera head 2a receives the command “MOVEH wbh”,
When the camera 1a is rotated to the angle wbh, the state shown in FIG. 7 is obtained.

Next, the operation of the present system for controlling the zoom of the other camera will be described. FIG. 8 is a processing flow in the case where zoom control of the camera of the user or the other party is performed. FIG. 9 shows an image 3 of the leftmost person B of the other party with whom the user is currently communicating by operating the mouse 6.
FIG. 4 is a diagram in which the mouse cursor 22 points to 4.
By clicking the left button of the mouse 6 in this state, the image 34 of the person B can be zoomed up (telephoto image) around the point indicated. Click once to zoom up a small amount, and double click to zoom up more. Also,
By clicking the right button of the mouse 6, it is possible to return to zoom down (wide-angle image). FIG. 9 is an image before zooming up.

FIG. 10 shows the other party camera 1 corresponding to FIG.
FIG. 3 is a diagram showing a positional relationship between directions of a and 2a and subjects 32a, 33a and 34a.

The operation in the case of performing the zoom control of the other camera will now be described with reference to the flow of FIG. When the screen inside the video window 21 is clicked by the mouse 6 in the operation event waiting state (103),
Mouse click events (110) and (112) occur. For the left button click event (110),
In the process (111), the photographing range (zoom angle) w of the camera
c is subtracted by the minute zoom change angle SmallZm. To subtract the shooting range angle wc of the camera means to zoom in (telephoto image). In the case of the right click event (112), the processing (11
In step 3), the angle wc is changed to a small zoom change angle SmallZm.
Add minutes. That is, zoom down (wide-angle image)
And

The double click event (114), (1
In the case of 16), in steps (115) and (117), subtraction and addition are performed by the zoom change angle LargeZm, and the zoom-up and down are performed more greatly than in the case of the right-click event. Next, in a process (118), a camera zoom-up command “ZOOM wc” is transmitted to the local TV conference board 9. As in the case of the camera movement command, the command is transmitted to the other camera 1a and the camera platform 2a to be zoomed up or down.

Next, in the process (119), a camera angle wbh for moving the camera in the direction centered on the point clicked in FIG. 9 is calculated, and a camera movement command is transmitted (120). In the process (121), the camera angle wbv in the vertical direction is similarly calculated, and a camera movement command is transmitted (122). As described above, the camera movement command is transmitted to the other camera 1a and the camera platform 2a to rotate the camera 1a to the angles wbh and wbv.
As a result, the image 34 of the person B is zoomed up and displayed as shown in FIG. FIG. 12 shows the relationship between the direction of the opponent camera 1a and the subjects 34a, 32a, and 33a at that time.

Next, another embodiment of the present invention will be described. FIG. 13 differs from the video window 21 in that horizontal and vertical scroll bars (24, 25, 26, 27, 28, 29, 30, 3, 3) are used.
FIG. 2 is a diagram of a video window 40 without 1). In this embodiment, video windows 41, 42, 43, 44,
When the mouse cursor 22 is clicked on each of the five areas, camera control corresponding to the clicked area is performed.

That is, when the camera right movement area 41 is clicked, a right movement command is transmitted to the camera. When the mouse button is released, a camera stop command is sent. As a result, the camera rotates to the right as shown in FIG. 14, and the user can see the image in the direction he wants to see. Similarly, reference numeral 42 denotes a camera upward movement area, 43 denotes a camera left movement area, 44 denotes a downward movement area, and 45 denotes a zoom up / down area. The zoom-up / down area 45 can be switched between zoom-in and zoom-down by, for example, zooming up when the left button of the mouse is pressed and zooming down when the right button is pressed.

FIG. 15 shows that the mouse cursor is zoomed up.
It is a figure in the case of being in a down area. When the mouse is left-clicked at this position, a zoom up command is transmitted to the camera, and when the mouse button is released, a zoom stop command is transmitted to the camera. As a result, the image is zoomed up as shown in FIG.

Dotted lines in FIGS. 13, 14, 15, and 16 are only provided for convenience in order to indicate the distinction between the areas, and need not be displayed on an actual video window. However,
When the mouse cursor is in the camera right movement area as shown in FIG. 13, the shape becomes a right arrow or the like. When the mouse cursor is in the zoom up or down area as shown in FIG.
By adopting a shape that represents this, it is possible to clearly notify the user whether the mouse cursor is in the current area.

[0040]

As described above, according to the present invention, the camera movement direction is calculated based on the coordinates indicated by the pointing device such as a mouse cursor, and the camera movement, zoom-in, and zoom-out are performed. The zoom operation can be performed by one-touch operation. Also,
An intuitive and easy-to-understand man-machine interface can be provided. The horizontal and vertical scroll bars at the edges of the video display window can indicate the relative position of the camera within the entire moving range of the camera and the relative range of the image being captured.
In addition, by operating this scroll bar, pan,
The camera can be moved to an arbitrary position by dragging the scroll box and dragging the scroll box. In addition, since a camera can be controlled by a scroll bar operation similar to a general application, a general and easy-to-understand man-machine interface can be provided.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a typical personal computer TV conference system that can be considered when the present invention is applied to a personal computer TV conference system.

FIG. 2 is a diagram showing a screen display displayed on a local monitor 3 of FIG. 1;

FIG. 3 is a video window 2 displayed on a local monitor 3
FIG.

FIG. 4 shows the other party's camera 1a corresponding to the display image of FIG.
And the camera head 2a and the subject A3
The figure which showed the positional relationship of 2a.

FIG. 5 is an initialization process when the present system actually operates, and a process flow when performing pan and tilt camera movement.

FIG. 6 is a diagram showing a case where the user operates the mouse 6 and clicks the right micro camera movement button 28 to move the camera to the right.

FIG. 7 is a view showing the positional relationship between the directions of the other-side cameras 1a and 2a and the subjects 32a, 33a and 34a corresponding to FIG.

FIG. 8 is a processing flow in the case of performing zoom control of the camera of the user or the other party.

FIG. 9 is a diagram in which the user operates the mouse 6 and points the video 34 of the leftmost person B of the other party with whom the user is currently communicating with the mouse cursor 22;

FIG. 10 is a view showing the positional relationship between the directions of the other-side cameras 1a and 2a and the subjects 32a, 33a and 34a corresponding to FIG.

FIG. 11 is a diagram in which a person B is zoomed up as a result of the operation shown in FIG. 10;

FIG. 12 is a diagram showing the positional relationship between the directions of the other-side cameras 1a and 2a and the subjects 32a, 33a and 34a corresponding to FIG.

FIG. 13 is different from the video window 21 in that horizontal and vertical scroll bars (24, 25, 26, 27, 28, 2);
9, 30, 31). FIG.

14 is an example of an image displayed as a result of rotating the camera to the right by clicking at the mouse cursor position shown in FIG.

FIG. 15 is a diagram when the mouse cursor is in a zoom-up / down area.

16 is an example of an image displayed as a result of zooming up the camera by clicking at the mouse cursor position shown in FIG.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Local camera 1a Remote camera 2 Local camera platform 2a Remote camera platform 3 Local monitor 4 Personal computer 5 Video overlay board 6 Mouse 7 Keyboard 9 TV conference board 10 Microphone 11 Speaker 20 Monitor screen 21 Communication video window 22 Mouse cursor 24, 25 Vertical small camera moving button 26 Vertical scroll box 27 Vertical camera moving unit 28, 29 Left / right small camera moving button 30 Left / right scroll box 31 Left / right camera moving unit 32 Image of person A 32a Person A 33 Image of person C 33a Image of person C 34 Image of person B 34a Person B 40 Communication image window without scroll bar 41, 42, 43, 44 Camera right, up, left, down movement area 45 Camera zoom up, Down area

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁶ Identification code FI H04N 7/15 H04N 7/15 7/18 7/18 E

Claims (6)

[Claims] 1. A camera control method for a video communication device having a graphical user interface, wherein a video captured by a camera or a video captured by a remote camera and transmitted over a communication line is displayed on a monitor. The image displayed on the upper side is divided into four areas of the upper, lower, right, and left directions, or the image display area is divided into areas corresponding to some of these directions. A camera control method for controlling a camera in a direction corresponding to a designated area by giving an instruction with a pointing device. 2. The image display area is divided into five areas including an area corresponding to four directions (up, down, left, and right) and an area corresponding to zoom control of an image, or some of these areas. When an area corresponding to is specified, the camera is zoomed up or
2. The camera control method according to claim 1, wherein zoom down is performed. 3. When the mouse pointer moves to an area corresponding to up / down / left / right directions, zoom control, etc., the shape of the mouse pointer corresponds to the area including the mouse pointer.
3. The camera control method according to claim 1, wherein a color, a display method, and the like are changed. 4. A method for controlling a camera in a video communication apparatus having a graphical user interface, wherein the video captured by a camera or the video captured by a remote camera and transmitted via a communication line is displayed on a monitor. A camera control method in which an arbitrary point of an image displayed above is indicated by a pointing device such as a mouse cursor, and the camera is controlled such that the indicated point is the center of the image. 5. A camera control method in a video communication apparatus having a graphical user interface, wherein a video captured by a camera or a video captured by a remote camera and transmitted over a communication line is displayed on a monitor, the monitor comprising: Point an arbitrary point on the video displayed above with a pointing device such as a mouse cursor, control the camera so that the point indicated is the center of the video, and zoom in on the camera according to the pointing method 5. The camera control method according to claim 4, wherein zooming down is performed. 6. A camera control method in a video communication apparatus having a graphical user interface, wherein a video captured by a camera or a video captured by a remote camera and transmitted over a communication line is displayed on a monitor, the monitor comprising: The top, bottom, left and right sides of the rectangular area that displays the video on top, or
In a video communication device that displays the direction of the camera in a rectangular area along some of these sides and controls the camera direction by operating the rectangular area with a pointing device such as a mouse cursor, Inside the rectangular area, there is a rectangular area that indicates the shooting range of the camera, and the ratio of the currently shot angle of view to the total angle of view that can be shot by controlling the camera, and the inner rectangle relative to the length of the outer rectangular area A camera control method for displaying the camera status so that the ratio of the lengths of the regions is the same.