JPH099137A - Image processing method and device therefor - Google Patents

Abstract

PURPOSE: To provide an image processing method by which an image obtained from image pickup is displayed dynamically in following to a change in an image pickup position of a camera and flexible and wide variety of image processing is conducted by using the displayed image and to provide a device therefor. CONSTITUTION: An image picked up by a camera is displayed on a window 200 and a window 201 commanding horizontal/vertical movement of the camera and zoom up/down is displayed on a screen together with the window 200. On the other hand, a transparent control panel 213 including GUI components 210-212 for various image processing commands is displayed while being superimposed onto the window 200. When the display image of the window 200 is changed by the movement of the image pickup position of the camera, the control panel 213 is moved in matching with the movement to keep the relation of position between the control panel 213 and the window 200 constant.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method and an apparatus therefor, and more particularly to an image processing method and an apparatus for handling an image captured from an image capturing device such as a camera as a moving image.

[0002]

2. Description of the Related Art In a conventional image processing system, an image capturing device such as a camera is connected, and an image captured from the device is treated as a moving image and displayed in an image display window on a screen such as a bitmap display. . At this time, in order to process and display such an image or a moving image, an icon that operates as one or a plurality of buttons, a toggle switch, or the like is displayed on the display screen.
(Graphic User Interface) Displayed as a component superimposed on the display image. Then, when a specific GUI component is picked up with a mouse or a cursor to instruct, a specific process is executed on the image being displayed according to the instruction.

[0003]

However, in the above-mentioned conventional example, since the direction and position of the image capturing device such as a camera remains fixed, the range of the image that can be captured by the device is limited, and the image displayed is limited. The range of images that can be operated by the range and GUI parts is also limited. As a result, there is a problem that sufficient image processing cannot be performed on an image representing an object to be photographed.

The present invention has been made in view of the above-mentioned conventional example, and provides an image processing method and apparatus capable of expanding a processable range for a target object and more flexibly performing sufficient image processing. To aim.

[0005]

In order to achieve the above object, the image processing method of the present invention comprises the following steps. That is, an input step of inputting image information representing a subject photographed by a camera, a first display step of displaying an image on a screen based on the input image information, and a predetermined process for the displayed image. On the screen to do,
A second display step of superimposing and displaying a specific image for instructing processing on the display image, and changing the display image based on the image information in accordance with a change in the shooting range of the camera. An image processing method is characterized in that the display of the specific image is controlled to change in accordance with the change of the image based on the image information.

According to another aspect of the invention, input means for inputting image information representing a subject photographed by a camera,
A first display unit for displaying an image on a screen based on the input image information; and a specific image for giving a processing instruction to the screen for performing a predetermined process on the displayed image. Second display means for superimposing and displaying on the display image, camera control means for controlling the camera so as to change the photographing range of the camera, and based on the image information according to the change of the photographing range of the camera. First display control means for controlling the first display means to change the image, and to control the second display means to change the display of the specific image in accordance with the change of the image based on the image information. And an image processing apparatus having a second display control means.

[0007]

With the above arrangement, the present invention inputs the image information representing the object photographed by the camera and displays the image on the screen based on the image information, while the predetermined image is displayed for the displayed image. In order to perform the processing of, the specific image for instructing the processing is displayed on the same screen so as to be superimposed on the display image, but the display image based on the image information changes with the change of the shooting range of the camera. And let
With the change of the display image, the display of the specific image is also changed.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings. FIG. 1 is a block diagram showing the configuration of an image processing system 100 capable of moving image processing in a typical embodiment of the present invention. In FIG. 1, reference numeral 101 is a camera that includes an image pickup tube, a sensor such as a CCD, and shoots an object.
Reference numeral 02 is a camera control unit for controlling the camera 101, 103 is a bitmap display (hereinafter referred to as a display), 104 is a CPU that controls the entire system and executes image processing and image display control, and 104a is a CPU 10
A ROM 105 for storing control programs and processing programs for various processes executed by the CPU 4 is used as a main area for storing image data, or as a work area when the CPU 104 executes the control programs and processing programs. A memory, 106 is for turning on / off the power supply of an external device such as an electric light or an OHP connected via an external bus.
An external device control unit for controlling and controlling the brightness of an electric lamp, 107 is a GU displayed on a display screen
A file that describes the position information and operation of the I part (hereinafter, G
A file storage unit configured to store a UI file), for example, a magnetic hard disk, a magneto-optical disk, or the like, and 108 indicates from the display screen to change the position of the camera 101 vertically and horizontally and to perform zoom up / down. Pointing device (below,
Reference numeral 108a is a keyboard (KB) for inputting various commands to the system 100 and response instructions from the GUI, and 109 is a CPU bus for connecting the respective components to each other.

The camera control unit 102 moves the position of the camera 101 vertically and horizontally or zooms up / down based on a control signal sent from the CPU 104 according to an instruction from the mouse 108. Also, the CPU 104
Is associated with a video (hereinafter referred to as a background image) captured by the camera 101 and a window (hereinafter referred to as a control panel) in which GUI components such as icons operating as one or more buttons or toggle switches are arranged. It also executes processing.

In the system 100 having the above configuration, the image captured by the camera 101 is input to the system 100 via the camera control unit 102, and the image is displayed on the display 1.
It is displayed as a moving image in the video display window displayed on the screen 03. FIG. 2 is a diagram showing various windows displayed on the screen of the display 103. Here, FIG. 2A shows a video display window 20 for displaying a moving image.
0 is a screen example in which a camera control window 201 used for a control instruction of 0 and the camera 101 is displayed. FIG.
(B) is a diagram showing a detailed configuration of the camera control window 201. The camera 101 is controlled by operating the mouse 108 using the camera control window 201. For example, if the arrow 202 is selected with the mouse 108,
A signal indicating the direction is transmitted to the camera control unit 102, and the camera 101 moves up, down, left, or right. When the enlargement (T) / reduction (W) bar 203 is selected, a signal indicating the zoom direction (enlargement (T), reduction (W)) is transmitted to the camera control device 102, and the camera 101 zooms up ( Enlarged image) or zoom down (reduced image).

Further, FIG. 2C shows the display 103.
GUI components 210, 211 such as icons that operate as buttons or toggle switches at appropriate positions on the screen of
It is a figure showing the control panel 213 which arranged 212. GUI parts 210, 211, 212 to mouse 1
If 08 is selected, the corresponding process is executed. The control panel 213 is generated as a transparent display panel, and its panel size is set larger than that of the video display window 200. In the present embodiment, the camera 10
The size is set so as to cover the entire range that can be captured as a camera image by the zoom, pan, and tilt operations of 1. Of course, when it is not necessary to cover the entire range, it goes without saying that the size should be set to the minimum required size.

Usually, the control panel 213 and the video display window 200 and the camera control window 201 for displaying the captured moving image are superimposed and displayed on the screen of the display 103, as shown in FIG. 2 (d). become. When the camera 101 is moved in the vertical / horizontal directions by the operation from the camera control window 201, the background image changes correspondingly, and the control panel 213 is similarly scrolled. The linking operation will be described later.

Next, image processing using the image processing system having the above configuration will be described with reference to FIGS.
FIG. 3 is a flowchart showing an outline of image processing executed by the image processing system 100. This image processing is activated by inputting a predetermined command to the system 100 from the keyboard (KB) 108a.

When the image processing is started, first, in step S601, the GUI file is stored in the file storage unit 107.
Check if it is stored in. Here, if the existence of the GUI file is confirmed, the process proceeds to step S602 to check whether a new control panel needs to be generated, and if it is determined that the new control panel needs to be generated, the process proceeds to step S604. If it is determined that it is not necessary, the process proceeds to step S603 and the GUI
The file is read into the main memory 105, and then the process proceeds to step S701. Also, step S601
Also, in the case where the existence of the GUI file is not confirmed, the process proceeds to step S604.

In step S604, a control panel is generated. The details will be described with reference to the flowchart shown in FIG. First, in step S651, the position of the origin e in the background image (all background images) in the maximum range that can be captured by the camera 101 is determined as follows. 5 shows the control panel 213 (FIG. 5B) and the background image 301 (FIG. 5) before the GUI parts are arranged.
It is a figure which shows the correspondence with (a). As shown in FIG. 5, on the display screen, the control panel 21
The positions of 3 and the background image 301 are common x with the origin being e.
It is defined by a y-orthogonal coordinate system, in which the x-axis is in the horizontal direction and the Y-axis is in the vertical direction, and the origin e is as shown in FIG.
As shown in (a), it is set at the lower left corner of the range of the entire background image.

Next, in step S652, its origin e
Corresponding point e on the control panel 213 so that
Determine the position of. That is, the control panel 213
Considering the correspondence between (FIG. 5B) and the background image 301 (FIG. 5A), the lower left corner of the control panel 213 is the origin of the xy orthogonal coordinate system, as shown in FIG. 5B. e
To be

When the setting of two points (the origin of the background image 301 and the origin of the control panel 213) is completed in this way, the process proceeds to step S653, and the control panel 213 before the GUI parts are arranged is displayed on the display 103. Display on screen. In step S654, the camera 101 is controlled to capture a background image and the video signal is input, and in step S655, the video display window 200
Display this background image on.

FIG. 6 is a diagram for explaining the direction / position control of the camera 101 and the correspondence between the background image and the control panel 213 using the same xy orthogonal coordinate system as used in FIG. In FIG. 6, when the horizontal deflection angle of the camera 100 is θ, the vertical deflection angle is φ, and the distance from the camera to the virtual screen is h with reference to the straight line connecting the first camera position and the origin e, When the camera is swung by θ in the left / right direction, the scroll distance (l) in the left / right direction of the background image / control panel 213 is approximated to 1≈h · tan θ (1). Also, the vertical scroll distance (m)
Is similarly approximated as m≈h · tan φ (2).

In this display, the control panel 213
And the whole background image are superimposed and displayed by aligning the origin e and aligning the x axis and the y axis respectively. Next, in step S656, the GUI is displayed on the control panel 213.
Ask the user whether or not to place the part using the dialog panel. Here, the control panel 21
In the case of arranging GUI parts on 3, the process is step S6.
In step 57, the GUI component is arranged at an appropriate position by using the mouse 108 while checking it against the displayed background image. Further, in step S658, the arranged GUI is arranged.
Set the process corresponding to the part. For example, camera 101
If there is an OHP (overhead projector) in the background image 301 taken in step # 3 and you want to turn ON / OFF the power of the OHP, place the GUI component 210 on the control panel 213 at a position corresponding to the OHP and turn ON / OFF the OHP power. The process for controlling the OFF is set in the GUI component 210. After the processing of step S658, the processing returns to step S656.

If it is determined in step S656 that there is no suitable place for arranging the GUI component based on the display condition of the background image, the process proceeds to step S659, and the photographing position is controlled by the control of the camera 101. Whether the position can be changed is checked, and if the position can be changed, the process returns to step S654. On the other hand, if it is determined that the camera control has been performed as much as possible, the process proceeds to step S660, and the control panel generated up to that point is stored in the file storage unit 10.
7 is stored.

After that, when the control panel 213 becomes available, the process proceeds to step S701,
Display the background image. Then, in step S702,
The background image and the control panel 213 are associated with each other, and in the next step S703, the control panel 2
13 is displayed so as to be superimposed on the video display window 200, and the process waits for an operation from the user in step S704. After that, control from the control panel 213 is performed.

For example, in step S705, the GUI components 210, 211, 2 of the control panel 213 are set.
If it is determined that 12 is selected, the process is step S7.
06, the selected GUI component 210, 21
A signal for executing the process of controlling the device corresponding to the No. 1, 212 is transmitted to the external device control unit 106, and the unit controls the power ON / OFF of the OHP, for example. After the execution of the control, the process returns to step S704 to enter the operation waiting state.

From the operation waiting state, step S707
Signal for controlling the camera 101 in the camera control unit 1
If it is transmitted to 02, the process proceeds to step S708, and the camera control unit 102 changes the direction of the camera 101 up / down / left / right or zooms up / down. Then, the camera 101 as described above
The background image and the control panel 213 scroll so that the control of the direction and position of the background image and the correspondence between the background image and the control panel 213 are maintained. When such camera control ends, the process returns to step S704,
The operation waits.

Further, in step S709, if it is confirmed that the command input for instructing the end of the process is input, the process ends. Therefore, according to the present embodiment, while remotely controlling the camera position from the display screen, the change in the shooting range that changes according to the change in the camera position is dynamically reflected in the image displayed on the display screen.
The GUI components necessary for processing the image can also be displayed in a superimposed manner while maintaining the positional correspondence with the image display screen. This makes it possible to more flexibly perform a wide range of image processing while changing the camera position to change the range of the display image.

The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of one device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or an apparatus.

[0026]

As described above, according to the present invention, image information representing a subject photographed by a camera is input and an image is displayed on the screen based on the image information, while the image is displayed. In order to perform a predetermined processing on an image, a specific image for instructing processing is displayed on the same screen so as to be superimposed on the display image, but based on the image information as the shooting range of the camera changes. The display image is changed and the display of a specific image is also changed according to the change of the display image, so that the shooting position of the camera can be changed while displaying the image of the subject over a wider range. However, there is an effect that it becomes possible to perform image processing.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an image processing system that is a typical embodiment of the present invention.

FIG. 2 is a diagram showing various windows displayed on a screen of a display 103.

FIG. 3 is a flowchart showing an outline of image processing executed by the image processing system 100.

FIG. 4 is a flowchart showing in detail a control panel creation process.

FIG. 5 is a diagram showing a manner of associating a control panel with a background image.

FIG. 6 is a diagram showing correspondence between camera positions and video display windows.

[Explanation of symbols]

 101 Camera 102 Camera Control Unit 103 Bitmap Display 104 CPU 104a ROM 105 Main Memory 106 External Device Control Unit 107 File Storage Unit 108 Mouse 108a Keyboard (KB) 109 CPU Bus

Claims (10)

[Claims] 1. An input unit for inputting image information representing a subject photographed by a camera, a first display unit for displaying an image on a screen based on the input image information, and for the displayed image. Second display means for superimposing and displaying a specific image for instructing a process on the screen in order to perform a predetermined process by controlling the camera so as to change the photographing range of the camera. A camera control means, a first display control means for controlling the first display means so as to change the image based on the image information in accordance with a change in a shooting range of the camera, and an image based on the image information. An image processing apparatus comprising: a second display control unit that controls the second display unit to change the display of the specific image in accordance with the change. 2. The image processing apparatus according to claim 1, wherein the camera control unit controls the camera to perform zooming or panning. 3. The image processing apparatus according to claim 2, wherein the zooming or panning instruction is given on the screen. 4. The image processing apparatus according to claim 3, wherein one window is displayed on the screen for instructing the zooming or panning. 5. The apparatus according to claim 1, further comprising an external device control means for connecting the device and controlling the device, the device including an electrically operated device included in a part of the subject. The image processing device described. 6. The image processing apparatus according to claim 5, wherein the control of the external device control means includes control of powering on / off of the device. 7. The power-on / off instruction of the device is given via the external device control means by an instruction to a specific image displayed by the second display means. The image processing device according to item 1. 8. The image processing apparatus according to claim 1, wherein the specific image displayed in a superimposed manner by the second display means is transparent. 9. The image processing apparatus according to claim 1, wherein the image displayed by the first display means is displayed in one window on the screen. 10. An input step of inputting image information representing a subject photographed by a camera, and a first step of displaying an image on a screen based on the input image information.
A display step, and a second display step of displaying, on the screen, a specific image for instructing processing to be superimposed on the display image for displaying a predetermined process on the displayed image, The display image based on the image information is changed according to the change of the photographing range of the camera, and the display of the specific image is changed according to the change of the image based on the image information. An image processing method characterized by the above.