JP3320128B2 - Image processing apparatus and image processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method for displaying an input image on a monitor at an arbitrary angle and size, such as a video conference system.

[0002]

2. Description of the Related Art Conventionally, in a video conference system as an image processing apparatus, for example, as shown in FIG.
1. A self-portrait and a partner image are displayed in different windows, respectively, like a camera control window 82, and the camera control window 82 is operated with a pointing device such as a mouse to control the self-portrait camera and the partner camera (panning). , Tilt, zoom) or a keyboard or dedicated keys are provided to control the camera.

In a conventional image processing apparatus, when a plurality of applications such as a personal computer and a workstation operate and have a videophone application for inputting an image with a television camera or the like, the programs of these applications are written in a dot format. In the image etc., display with the icon stored in advance,
By clicking the icon with a mouse, etc.
The application is running.

[0004]

However, in the above-described conventional video conference system, the sizes of the self-image display window and the partner image display window cannot be freely changed according to the user's preference. In addition, if the size of the self-image display window and the partner image display window can be freely changed according to preference, the camera control window overlaps with the self-image display window or the partner image display window to operate. There is a problem that it becomes difficult.

Further, when a keyboard or a dedicated key is provided to control the camera, there is an inconvenience that the user has to operate while looking at both the operation unit and the monitor. In addition, when an application program is displayed as an icon stored in advance and the application is started by clicking the icon with a mouse or the like, it is necessary to set an easy-to-understand icon for each application.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus and an image processing method which facilitate operations such as scrolling and enlargement of a window screen, and which can easily grasp a running application.

[0007]

Means for Solving the Problems To achieve the above object,
Therefore, the present invention has the following configuration. Window screen
With an instruction part for giving an instruction to change the size of
The output image from the imaging device is displayed on the window screen.
Display control means to make, based on an instruction to the instruction unit,
A window for controlling a change in size of the window screen
C control means for responding to a change in the size of the window screen.
First control for controlling a zoom function of the photographing device
Means.

[0008] Preferably, the window is further provided.
C At the position close to the screen,
A screen for specifying the scroll direction of the image to be displayed
Display the roll instruction section, and
Control the shooting operation of the shooting device
And second control means.

Further, preferably, the window screen
When the first control means is expanded,
To zoom out. More preferably, the
When the window screen is reduced, the first control means
The imaging means is zoomed in.

Further, according to the present invention, there is provided an instruction unit for giving an instruction to change the size of the window screen, display control means for displaying an image output from the photographing device on the window screen, and the instruction unit. A window control unit that controls a change in the size of the window screen based on an instruction to the control unit; and a control unit that controls a magnification of an output image from the imaging device in accordance with the change in the size of the window screen. Having. Further, according to an eighth aspect of the present invention, there is provided a display control step of displaying an output image from a photographing device on a window screen together with an instruction unit for giving an instruction to change the size of the window screen, and an instruction to the instruction unit. And a control step of controlling a magnification of an image output from the photographing apparatus in accordance with the change of the size of the window screen. Further, claim 1
The invention described in claim 0 is an image processing method for operating an application for displaying an output image from a photographing means for photographing an arbitrary object on a monitor screen, wherein the output image is used as an icon corresponding to the application on the monitor screen. An icon display control step to be displayed on the top,
A receiving step of receiving an instruction for the icon; and, in the receiving step, a starting step of activating the application in response to receiving the instruction.

In the above-described configuration, the function is such that the operability is improved and the monitor screen is effectively used.

[0011]

Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. [First Embodiment] FIG. 1 is a block diagram showing a configuration of a still picture video conference apparatus (hereinafter, referred to as a system) according to a first embodiment of the present invention. In the figure, reference numeral 1 denotes a CPU (central control unit) for controlling the entire system, 2 denotes a monitor, 3 denotes a monitor display image memory, and 4 denotes a zooming mechanism for capturing a self-image of an operator of the system. A video camera with a camera 5, a camera platform for panning and tilting the video camera 4, a camera image memory 6 for storing an image from the video camera 4 and reading it out at an arbitrary magnification, A camera control unit that controls zooming of the video camera 4 and pan and tilt of the camera platform 5.

Reference numeral 8 denotes a communication control unit for controlling communication between the system and a remote place via a line such as ISDN or LAN, and reference numeral 9 denotes a partner image received by the communication control unit 8. , A receiving image memory that can read it out at an arbitrary magnification, an operation unit 10 including a keyboard, a mouse, and the like; 11, a ROM for storing an operation program of the CPU 1; RAM for temporarily storing data required for operation, and 13,
This is a system bus common to the above components.

FIG. 2 shows an example of a monitor screen in the system of this embodiment. In the figure, 20 is a monitor screen,
21 is a self-image display window, 22 is a partner image display window, 23 is an upper scroll button, 24 is a lower scroll button, 25 is a right scroll button, 26
Is a left scroll button. Reference numeral 27 denotes an enlarged or reduced area of the window.

The scrolling, enlargement, or reduction of the window is performed by clicking the scroll buttons 23 to 26 or by dragging the window enlargement / reduction area 27 with the mouse constituting the operation unit 10. Next, the operation of the system according to the present embodiment will be described. FIG. 3 is a flowchart illustrating the operation of the system (video conference apparatus) according to the present embodiment. As shown in the figure, when the present system is started, the CPU 1 creates self-image display window screen data and transfers it to the monitor display image memory 3 (step S301).
Further, the data in the camera image memory 6 storing the image from the video camera 4 is transferred to the monitor display image memory 3 so as to be displayed in the self image display window, and the self image is displayed on the monitor 2 (step S302).

Next, a line is connected to a partner with whom a conversation is desired (step S303). When the line is connected, the self-image data is read from the camera image memory 6 and is read via the communication control unit 8. The message is transmitted to the other party (step S304). In addition, window screen data for displaying the other party's image is created and stored in the monitor display image memory 3.
(Step S305). Then, it requests a partner image via the communication control unit 8, and waits for reception of image data from the partner.

If image data is received from the communication partner (step S306), the image data is transferred to the monitor display image memory 3 to display the partner image data in the partner image display window, and the partner image is monitored. 2 (step S307). Then, CPU 1 monitors whether a window operation (for example, screen scrolling, enlargement, reduction) has been performed (step S308), and if a window operation has been performed, performs a window operation process corresponding to the window operation (step S308). S309).

Also, the reception of a camera control command from a communication partner is monitored (step S310). If a command from the communication partner is received, processing for the command is performed (step S311). These steps S30
8 and the processing of step S310 are performed until the conversation using the present system ends (step S31).
2) When the conversation is over, the line is disconnected (step S3).
13). Then, the window for displaying the partner image is deleted (step S314), and finally, it is confirmed whether or not the operation of the system is completed (step S315). Here, if the end of the system operation is detected, the window for displaying the self-portrait is also erased, and the operation of the system is ended. If not, the process returns to step S303 to connect to the next conversation partner. Perform

FIG. 4 is a flowchart showing the detailed operation of step S309 in FIG. In FIG. 4, when a window operation is performed, the CPU 1 determines which window has been operated, and performs a process corresponding thereto (steps S401 and S403). For example, if the operated window is a self-image display window, self-image display window operation processing is performed (step S402).
If it is a window for displaying a partner image, a window operation process for displaying a partner image is performed (step S4).
04).

Next, the operation of step S402, that is, the self-image window operation process will be described with reference to the flowchart shown in FIG. In FIG. 5, C
PU1 determines whether or not the operation of the self-image display window is scrolling (step S501). If it is scrolling, the camera pans according to the scrolling direction.
Or perform tilt. If the operation on the window is upward scrolling (step S502), the camera controller 7 is instructed to tilt the camera platform 5 upward, and the camera is moved upward (step S503). When the movement of the camera is completed, the self-image data is read out from the camera image memory 6 (step S504), transferred to the monitor display image memory 3, and displayed on the monitor 2 (step S505).

On the other hand, if it is determined that the scroll is the downward scroll (step S506), the camera controller 7 is instructed to tilt the camera platform 5 downward, and the camera is moved downward (step S506). Step S507). If the scroll is right scroll (step S50)
8) Instruct the camera controller 7 to pan the camera platform 5 in the right direction to move the camera to the right (step S50).
9) If the scroll is left (step S510), the camera controller 7 is instructed to pan the pan head 5 in the left direction to move the camera to the left (step S511), and if these movements are completed. Steps S504 and S505 are performed as in the case of the upward scroll.

If the window operation is to enlarge the window (step S512), the CPU 1 enlarges the window for displaying the self-portrait (step S513), and instructs the camera controller 7 to zoom out the video camera 4. Then, the video camera 4 is zoomed out (step S514). When this zoom-out is completed, to fit the size of the enlarged self-portrait display window,
The read magnification from the camera image memory 6 is set (step S515).

When the read magnification is set, the processes of steps S504 and S505 are performed as in the case of the upward scroll. If the window operation is to reduce the window (step S516), the window for displaying the self image is reduced (step S517).
Is instructed to zoom in on the video camera 4, and the video camera 4 is zoomed in (step S51).
8). When the zoom-in is completed, the read magnification from the camera image memory 6 is set to match the size of the reduced self-image display window (step S515). The processes of steps S504 and S505 are performed in the same manner as in the case of scrolling.

Next, the operation of step S404 in FIG. 4 (window operation processing for the partner image) will be described with reference to the flowchart shown in FIG. In FIG.
U1 determines whether the operation of the window for displaying the other party's image is scrolling (step S601), and if it is scrolling, a command for instructing the other party's camera pan or tilt of the other party's camera depending on the scrolling direction. Is transmitted. In other words, if the operation of the window is upward scrolling (step S602), a command for instructing the other-side camera to tilt upward is transmitted via the communication control unit 8 (step S603), and the reception of the other-party image data is performed. wait. Then, when the partner image data is received (step S604), the image data is read from the receiving image memory 9 storing the partner image data (step S605), and is transferred to the monitor display image memory 3. It is displayed on the monitor 2 (step S606).

If it is determined in step S601 that the scroll is a downward scroll (step S60)
7) A command for instructing the other party's camera to tilt downward is transmitted to the other party via the communication control unit 8 (step S608). If the scroll is right scrolling (step S609), the other party is A command for instructing the side camera to pan right is transmitted via the communication control unit 10 (step S610). On the other hand, if the scroll is left scrolling (step S611), a command for instructing the other-side camera to pan left is transmitted via the communication control unit 8 (step S612), and the reception of the other-party image data is waited. . Then, if the other party's image data is received after transmitting these commands, the processes of steps S604 to S606 are performed as in the case of the upward scroll.

If the window operation is to enlarge the window (step S613), the CPU 1 enlarges the window for displaying the other party's image (step S614), and instructs the other party's camera to zoom out via the communication control unit 8. Then, a read command is transmitted (step S615), and the read magnification from the receiving image memory 9 is set so as to match the size of the enlarged window for displaying the partner image (step S616). After setting the reading magnification, the processes of steps S604 to S606 are performed as in the case of the upward scroll.

If the window operation is to reduce the window (step S617), the CPU 1 reduces the window for displaying the partner image (step S618), and sends a command for instructing the partner camera to zoom in to the communication control unit 8. (Step S619). And
After setting the read magnification from the receiving image memory 9 so as to match the size of the reduced window for displaying the partner image (step S616), the processes of steps S604 to S606 are performed as in the case of the upward scroll.

FIG. 7 is a flowchart for explaining the command processing in step S311 of FIG. FIG.
In step S701, the CPU 1 determines whether or not the command received from the communication partner is a command for instructing zooming (step S701).
702), the window for displaying the self-portrait is enlarged (step S703), and the camera controller 7 is instructed to zoom out the video camera 4, and the video camera 4 is zoomed out (step S704). When the zoom-out is completed, the read magnification from the camera image memory 6 is set to match the size of the enlarged self-image display window (step S705).

Subsequently, the self-image data is read out from the camera image memory 6 (step S706), transferred to the monitor display image memory 3 and displayed on the monitor 2 (step S707). Then, the self image data is read out from the camera image memory 6 and transmitted to the other party via the communication control unit 8 (step S708). If the command received from the other party is zoom-in (step S70)
9) Reduce the window for displaying the self-portrait (Step S)
710), the camera controller 7 is instructed to zoom in on the video camera 4, and the video camera 4 is zoomed in (step S711). When the zoom-in is completed, the read magnification from the camera image memory 6 is set to match the size of the reduced self-image display window (step S705). When the read magnification is set, the processes of steps S706 to S708 are performed as in the case of zooming out.

On the other hand, if the command received from the communication partner is the upper tilt (step S712), the CPU 1 instructs the camera controller 7 to tilt the camera platform 5 upward and causes the camera platform 5 to tilt upward. (Step S713). When the upper tilt is completed, steps S706 to S708 are performed as in the case of zooming out. If the command received from the other party is downward tilt (step S71)
4) Instruct the camera controller 7 to tilt the camera platform 5 downward, and cause the camera platform 5 to tilt downward (step S715). Then, if the lower tilt is not completed, the processes of steps S706 to S708 are performed as in the case of zooming out.

On the other hand, if the command received from the other party is the right pan (step S716), the CPU 1 instructs the camera controller 7 to pan the video camera 4 right, and causes the camera platform 5 to pan right (step S717). ). When the right pan is completed, step S7 is performed in the same manner as when zooming out.
When the command received from the other party is left panning (step S718), the CPU 1
Instructs the camera controller 7 to pan the video camera 4 left, and causes the camera platform 5 to pan left (step S719). When the left pan is completed, the processes of steps S706 to S708 are performed as in the case of the zoom-out.

As described above, according to the present embodiment,
By linking the operation of scrolling and enlarging / reducing the image display window with the camera control such as pan, tilt, zoom, etc., the actual camera control matches the image displayed on the monitor, and There is an effect that the system can be easily operated. Further, by providing the camera control window close to the image display window, it is possible to prevent the effective display portion of the monitor screen from being reduced by the camera control window. Second Embodiment Hereinafter, a second embodiment according to the present invention will be described.

FIG. 9 is a block diagram showing the overall configuration of an information processing apparatus (hereinafter, referred to as a system) according to a second embodiment of the present invention. In the figure, reference numeral 201 denotes a CPU that controls the entire system, 202 denotes a monitor as a display device, 203 denotes an image memory for monitor display, 204
Displays the image in the monitor display image memory 203 on the monitor 20.
Reference numeral 2 denotes an output signal processing circuit for displaying on the display unit; 205, a video camera as an image input device; and 206, a camera image memory for storing an image from the video camera 205.

Reference numeral 207 denotes an input signal processing circuit for detecting the presence / absence of an input signal from the video camera 205 and storing an image from the video camera 205 in the camera image memory 206. Reference numeral 208 denotes a camera image memory 206. Monitor data in the monitor display image memory 203 in an arbitrary size.
209 is a memory control circuit for reading data to
ROM for storing the operation program 01, etc., 21
0 is a RAM for temporarily storing data necessary for the operation of the CPU 201, 211 is a keyboard,
An operation unit including a mouse and the like.

FIG. 10 is a diagram showing an example of a monitor screen in the system according to the present embodiment. In FIG.
20 is a monitor screen, 221 is a videophone application program icon, 222 is a word processor application program icon, and
Reference numeral 223 denotes an icon of the spreadsheet application program. Here, each application is started by clicking these icons using a mouse constituting the operation unit 211 shown in FIG.

Next, the operation of the system according to this embodiment will be described. FIG. 11 is a flowchart illustrating the operation of the system according to the present embodiment. In the figure, when the system power of the present system is turned on (step S301)
'), The CPU 201 checks whether there is an input signal from the video camera 205 to the input signal processing circuit 207 (step S302'). Then, in this step S302 ′,
If it is determined that there is an input signal from the video camera 205, the image from the video camera 205 is input to the input signal circuit 20.
7 is stored in the camera image memory 206 (step S303 ').

When the above determination is made, the CPU 201
Is instructed to read the image data stored in step S303 'from the camera image memory 206 so as to have the icon size, and to transfer the image data as an icon to the monitor display image memory 203 (step S304).
´). If image data is transferred, the output signal processing circuit 2
At 04, a process for outputting this image data to the monitor 202 is performed (step S305 ').
02 (step S306 ').

On the other hand, in step S302 ', if there is no input signal to the camera after the power of the video camera 205 is turned on, the CPU 201 reads out the icon data stored in advance from the RAM 210 and displays it as an icon on the monitor. It is transferred to the image memory 203 for use (step S307 '). And the following step S
In steps 305 and S306, the same processing as when there is an input signal from the video camera is performed.

As described above, in this embodiment, in the information processing apparatus on which a plurality of applications operate,
Displaying the input video as an icon of a videophone application has the effect of making it easier to understand the icon display of the videophone application program. Also, when there is no image input to the information processing device, by displaying an icon prepared in advance,
It is possible to confirm whether or not there is an input image only by looking at the icon display without starting the program.

In the above embodiment, the presence or absence of an input signal from the video camera 205 is always detected, image data is read out from the camera image memory 206 at certain intervals, and is transferred to the monitor display image memory 203. Then, the icon data of the videophone application may be rewritten and displayed. The present invention may be applied to a system including a plurality of devices or an apparatus including a single device. Further, it goes without saying that the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or an apparatus.

[0040]

As described above, according to the present invention,
Scroll and enlarge / reduce operation of image display window
Operation and camera control such as pan, tilt and zoom
The actual camera control and image display on the monitor.
Images match and the system can be easily viewed while looking at the monitor screen.
The effect is that the system can be operated.

In particular, according to the third and seventh aspects of the present invention, even if the display area of the window screen is expanded, it is possible to perform processing such as control so that the size of the subject is not changed. Further, according to the inventions of claims 4 and 8, even if the display area of the window screen is reduced, it is possible to perform processing such as controlling so as not to change the size of the subject.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a still image video conference device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of a monitor screen in the video conference device according to the first embodiment.

FIG. 3 is a flowchart showing an operation of the device according to the first embodiment.

FIG. 4 is a flowchart showing a detailed operation of step S309 in FIG. 3;

FIG. 5 is a detailed flowchart of a self-portrait window operation process in step S402 shown in FIG. 4;

FIG. 6 is a detailed flowchart of a window operation process for a partner image in step S404 shown in FIG. 4;

FIG. 7 is a flowchart illustrating a command process shown in step S311 of FIG. 3;

FIG. 8 is a diagram showing a monitor screen of a conventional video conference system.

FIG. 9 is a block diagram illustrating a configuration of an entire information processing apparatus according to a second embodiment of the present invention.

FIG. 10 is a diagram showing an example of a monitor screen on the device in the second embodiment.

FIG. 11 is a flowchart showing an operation of the device according to the second embodiment.

[Explanation of symbols]

 1,201 CPU 2,202 Monitor 3,203 Monitor display image memory 4,205 Video camera 5 Pan head 6,206 Camera image memory 7 Camera control unit 8 Communication control unit 9 Reception image memory 10, 211 Operation unit 11 , 209 ROM 12, 210 RAM 13 System bus 20 Monitor screen 21 Self-image display window 22 Other image display window 23 Up scroll button 24 Down scroll button 25 Right scroll button 26 Left scroll button 27 Window enlargement / reduction area 80 Monitor Screen 81 Self-image display window 82 Opponent image display window 83 Camera control window

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G09G 5/00 G09G 5/34 G09G 5/36

Claims (8)

(57) [Claims] A display control unit configured to display an output image from a photographing device on a window screen together with an instruction unit for issuing an instruction to change a size of the window screen; An image, comprising: a window control unit configured to control a change in the size of a window screen; and a first control unit configured to control a zoom function of the image capturing apparatus according to the change in the size of the window screen. Processing equipment. 2. A scroll instructing unit for instructing a scroll direction of an image to be displayed on the window screen is displayed at a position close to the window screen, and linked to an instruction to the scroll instructing unit. The image processing apparatus according to claim 1, further comprising a second control unit configured to control a photographing operation of the photographing device. 3. When the window screen is enlarged,
The first control means zooms out the imaging means.
3. The method according to claim 1, wherein
The image processing apparatus according to any one of the preceding claims. 4. When the window screen is reduced,
The first control means causes the imaging means to zoom in
The method according to any one of claims 1 to 3, wherein
Image processing device. 5. A display control step of displaying an output image from a photographing device on a window screen together with an instruction unit for giving an instruction to change the size of the window screen, and based on an instruction to the instruction unit, An image, comprising: a window control step of controlling a change in the size of a window screen; and a first control step of controlling a zoom function of the imaging device in accordance with the change in the size of the window screen. Processing method. 6. A scroll instruction portion for instructing a scroll direction of an image to be displayed on the window screen is displayed at a position close to the window screen, and linked to an instruction to the scroll instruction portion. The image processing method according to claim 5, further comprising a second control step of controlling a photographing operation of the photographing device. 7. When the window screen is enlarged,
The first control step zooms the zoom function of the imaging device.
7. The method according to claim 5, wherein
2. The image processing method according to claim 1. 8. When the window screen is reduced,
The control step zooms in on a zoom function of the imaging device.
8. The method according to claim 5, wherein
The image processing method according to the item.