JP2004215298A - Image processing apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus in which video with more presence desired by a user is generated even without changing camera setting. <P>SOLUTION: A plurality of photographed images obtained from first and second camera heads and photographic conditions of the camera heads are read and in a coordinate transformation circuit 208 or an image compositing circuit 211, the read photographed images and photographic conditions are used to generate an image watched from a point of view designated in accordance with user's operation of a joy stick 210. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus connected to a camera having a plurality of camera heads via a network and an image processing method thereof.

  2. Description of the Related Art Conventionally, video cameras have been used as moving image input devices in video conferences, video phones, and devices that make presentations using video images. Here, there is a document camera in which a television camera, lighting, a camera stand, and the like are integrated so as to be particularly convenient for a presentation or a meeting.

  Patent documents 1 and 2 are related to the present invention.

JP 05-110935 A JP-A-04-292086

  The conventional document camera is based on the premise that a presenter handles the camera while directly operating the camera. Particularly, when performing video communication with a distant place using a high-speed network capable of communicating video signals, the following problems occur in the conventional document camera.

  In order for the presenter to directly operate the camera, switches for camera settings (for example, zoom and focus) are attached to the document camera body, and the camera installation angle and illumination position cannot be changed or manually set. For this reason, in the network conference, when the video receiver wants to focus on a part of the subject, the presenter is requested to zoom in on the camera. Also, in a conference where a three-dimensional object with a complicated shape is shown, it is difficult to understand the shape only from the video viewed from one side, so it is inevitable that the receiver needs to change the camera position, angle, focus, etc. Often occur.

  However, depending on the performance and physical arrangement of the camera, a desired image may not be obtained even if the camera setting is changed. Further, the video after the setting change is performed by the specific recipient is not necessarily the video desired by another recipient.

  The present invention has been made in order to solve such a problem, and provides an image processing apparatus and an image processing method capable of generating a more realistic image desired by a user without changing camera settings. The purpose is to do.

  In order to achieve the above object, an image processing apparatus according to the present invention is an image processing apparatus connected via a network to a camera having a plurality of camera heads, wherein a plurality of photographing images obtained from the plurality of camera heads are provided. Reading means for reading an image and shooting conditions of each camera head, operating means for designating a viewpoint of a subject included in the plurality of shot images according to a user operation, and using the plurality of shot images and the shooting conditions And image processing means for executing a process of generating an image viewed from the viewpoint designated by the operation means.

  Further, the image processing method of the image processing device of the present invention is an image processing method of an image processing device connected via a network to a camera having a plurality of camera heads, wherein the plurality of camera heads obtained from the plurality of camera heads are provided. A reading step of reading the photographed image of each of the camera heads and the photographing conditions of each camera head, and a process of generating an image viewed from a specified viewpoint according to a user operation using the plurality of photographed images and the photographing conditions. And an image processing step to be executed.

  According to the present invention, it is possible to generate a more realistic image desired by a user without changing the camera settings.

  FIG. 1 shows a configuration diagram of a network-compatible camera according to the present invention. In the figure, 101 is a subject, 102 is a subject stage, 103 is a first camera head, 104 is a second camera head, 105 is a first illumination, 106 is a second illumination, 107 is a video signal bus, and 108 is a video signal bus. A control signal bus, 109 is a system controller, 110 is a video signal encoder, and 111 is a network interface. Reference numeral 112 denotes a stage driving unit, 113 denotes a first camera head driving unit, 114 denotes a first camera head focal length driving unit, 115 denotes a first camera head focus driving unit, and 116 denotes a first camera head aperture. A drive section 117 is a second camera head angle drive section, 118 is a second camera head focal length drive section, 119 is a second camera head focus drive section, 120 is a second camera head aperture amount drive section, 121 Is a first illumination position drive unit, 122 is a first illumination intensity control unit, 123 is a second illumination position drive unit, 124 is a second illumination intensity control unit, and 125 is a ROM table. The attachment angle drive / detection units 131 and 132 handle at least two directions, and can be operated, for example, in the horizontal direction in the plane of the paper and in the direction perpendicular to the plane of the paper. This apparatus is connected to a network through an interface 111 and communicates video signals and control signals.

  Next, the operation of the present apparatus will be described. First, the video signal flows as follows. Video signals from the first camera head and the second camera head are input to a video signal encoder 110 through a video signal bus 107. This encoder performs conversion processing and coding into a data format for communication over a network. Further, this signal is transmitted to the network through the interface 111 that performs protocol processing and the like. Here, in normal usage, the video signals of the two camera heads are each coded and communicated as independent signals. However, as shown in the figure, the encoder 110 is configured to be controlled by the controller 109. For example, the encoder 110 transmits a signal only from the first camera head and does not transmit a signal from the second camera head, or vice versa. Can also be changed.

  A control command for the camera head sent from the network is received by the interface 111 and passed to the controller 109. The controller 109 stores the actuator control amount of each unit at that time, and when a control command is sent from the net, outputs a difference from the command value as an actuator drive amount. In this example, all actuators are assumed to be of the type that is controlled by an open loop, such as a stepping motor. It is configured to control afterwards. From the net, the two mounting angles of the two camera heads, the focal length, the focus, the aperture, and the positions and intensities of the first and second illuminations can be controlled.

  This apparatus can read out the control amounts, set values, and device specifications of the apparatus, that is, the photographing conditions of each camera head from the network. For example, the current control amount of the actuator is stored in the controller 109, and is read and transmitted from here. Information unique to the device, such as the variable focal length range of the lens, the mounting interval of each camera, the characteristics of the light receiving element, the minimum unit for driving the mounting angle, and the color temperature of the illumination system, are stored in the ROM table 125. Data is read from the table 125 and transmitted to the network.

  Since the camera head of the present invention has such a configuration, it is possible to realize the following new functions. FIG. 2 shows its use. In the figure, 200 is a trunk network, 201 is a document camera of the present invention, 202 is a video camera, 203 is a presenter, 204, 206, 209, and 212 are first, second, third, and fourth displays, 205, respectively. , 207, 210, and 213 are first, second, third, and fourth joysticks for operating the display on the first, second, third, and fourth displays, respectively. Reference numeral 208 denotes an image coordinate conversion circuit; and 211, an image synthesis circuit. In this figure, the video signal and the control signal are shown as being communicated on the same net, but may be connected on different networks.

  First, the first and second displays will be described. The first display 204 displays the image of the first camera head of the document camera 201, and the second display 206 displays the image of the second camera head as they are. Each display is equipped with a joystick, and by this operation, the mounting angle, focal length, and focusing of the camera head can be controlled. Therefore, the recipient of the image can operate the joystick and view the subject of the document camera at any angle and size as desired. In this example, since only two camera heads are mounted in the document camera 201, up to two receivers can directly operate the camera head independently. Here, there are cases where there are three or more recipients and the user wants to view an image from a position different from the first and second displays. When this apparatus is used, a third image can be obtained without changing the states of the first and second camera heads. For example, an image coordinate conversion circuit 208 is attached to the third display 209, and the joystick is connected to this circuit. The coordinate conversion circuit 208 reads the state of each camera head and a video signal by communication, and detects stereoscopic information of the subject. Then, the receiver converts the image into an image from the direction designated by the joystick, outputs the image, and displays the image on the third display. When there are two or more such camera heads, a technique for pseudo-changing the shooting direction of a subject is already known and has been realized. In this network, no matter how many displays on the receiving side are provided, if a coordinate conversion circuit is provided, they can be displayed at the respective photographing angles. Further, depending on the recipient, there is a case where it is desired to obtain detailed information by focusing on a certain point of interest. In a network using the present apparatus, when two camera heads are photographing the same portion repeatedly, the portion can be projected with high definition. For example, the fourth display includes an image synthesizing circuit 211, calculates a corresponding point of an overlapping portion of the images of the two camera heads, and superimposes the pixels of the corresponding two images, thereby obtaining an S / N ratio. Image can be obtained. Further, by giving an instruction to shift the angle of view of the two camera heads by 0.5 pixel, it is possible to obtain image information with an apparently fine pixel pitch. As described above, in the document camera of the present invention, by providing the processing circuit on the receiver side, it is possible for a plurality of receivers to obtain one subject in any direction, size, and image quality.

  However, when performing the above-described coordinate conversion processing, the processing for increasing the definition of the overlapping portion, and the processing for increasing the S / N ratio, and when the subject is moving, the following points must be noted. For example, it is assumed that the image sensor of each camera is a CCD image sensor of a current television system such as NTSC and the like, and continuously operates at a field frequency of 60 Hz. At this time, if the exposure start times of the cameras are shifted, that is, if the V-direction synchronization of the video signals does not match, if the subject moves at high speed, the cameras capture different states. At this time, the correspondence between the two images cannot be taken, and the coordinate conversion process cannot be performed accurately. Further, when the images of the overlapping portions are superimposed, the images become loose or blurred. Therefore, when performing such processing, the present apparatus can control the exposure time of each camera unit by an external command. When the exposure is performed periodically as in the current television system, the exposure time may be controlled once by resetting. When the exposure periods of the respective camera units are different, the respective exposure start times, exposure end times, and readout start times are controlled independently.

  Furthermore, when each camera unit is exposing at the same field frequency, it is possible to intentionally shift the V cycle of each other. By doing so, the number of exposures per unit time can be apparently doubled. Therefore, when the subject has a fast movement, the displacement of the image for each field can be suppressed to be small, and the state of the movement can be accurately grasped. In this case, the difference between the exposure times of the camera units is read out or controlled from a network.

  An error may occur in the detection value of the photographing condition of each camera unit in the present apparatus due to mechanical mechanical play or electric noise. For example, even if the focal length information read from each camera unit has the same numerical value, the respective video sizes may be slightly different. When the video is synthesized as it is, the video is also blurred or blurred. Therefore, in a processing device that performs image synthesis, for example, 208 and 211 in FIG. 2, the command value and the detection value of the imaging condition are calibrated using the actually obtained image.

  In the above example, two camera heads are shown. However, when three or more camera heads are used, stereoscopic information can be measured more accurately, and the range of the imaging direction to be converted is widened. Also, when a high-definition image is desired to be obtained as in the case of the fourth display, a finer image can be displayed if the number of camera heads is three or more. However, a document camera having only one camera head or a camera that cannot be operated from a network even if two or more camera heads are provided cannot obtain any of the above functions. Therefore, these functions are unique to the camera of the present invention.

  This device further has the following functions. In FIG. 2, if the second, third, and fourth displays desire the same video and only display that video, for example, if only the first camera head is required, The video signal of the second camera head uses a useless channel (band) on the network. Therefore, when there is no signal requesting the image of the second camera head, the document camera 201 does not transmit the signal of the second camera head to the network. Therefore, the traffic of the network can be reduced to the minimum.

FIG. 1 is a diagram illustrating a configuration of a document camera of the present invention. FIG. 3 is a diagram illustrating an example of imaging of the document camera of the present invention in a video network.

Explanation of reference numerals

101 subject 102 subject stage 103 first camera head 104 second camera head 105 first lighting 106 second lighting 107 video signal bus 108 control signal bus 109 system controller 110 video signal encoder 111 network interface 112 stage driving unit 113 First camera head angle drive unit 114 First camera head focal length drive unit 115 First camera head focus drive unit 116 First camera head aperture amount drive unit 117 Second camera head angle drive unit 118 Second Camera head focal length drive unit 119 Second camera head focus drive unit 120 Second camera head aperture stop drive unit 121 First illumination position drive unit 122 First illumination intensity control unit 123 Second illumination position drive unit 124 Second illumination intensity control unit 1 5 ROM table 200 Trunk network 201 Document camera 202 Video camera 203 Presenter 204 First display 205 Joystick 206 for first display operation 206 Second display 207 Joystick 208 for second display operation 208 Image coordinate conversion circuit 209 Third display 210 Third display operation joystick 211 Image combining circuit 212 Fourth display 213 Fourth display operation joystick

Claims (8)

An image processing device connected to a camera having a plurality of camera heads via a network,
Reading means for reading a plurality of captured images obtained from the plurality of camera heads and the shooting conditions of each camera head,
Operating means for designating a viewpoint of a subject included in the plurality of captured images according to a user operation;
An image processing apparatus comprising: an image processing unit that executes a process of generating an image viewed from a viewpoint specified by the operation unit using the plurality of captured images and the imaging conditions.   2. The image processing device according to claim 1, wherein the image processing unit synthesizes images obtained from the plurality of camera heads using the shooting conditions to generate an image of the subject viewed from an arbitrary viewpoint. The image processing apparatus according to claim 1.   2. The image processing device according to claim 1, wherein the image processing unit performs coordinate transformation processing on the images obtained from the plurality of camera heads using the shooting conditions, and generates an image of the subject viewed from an arbitrary viewpoint. An image processing apparatus according to claim 1.   The shooting conditions include a distance between camera heads, a shooting angle of each camera head, a lens focal length, a focus position of each camera head, an aperture amount, a shutter speed of a light receiving element, a position of a subject stage, an illumination position, and an illumination amount. The image processing apparatus according to claim 1, further comprising:   The photographing conditions include information on signal characteristics of a light receiving element, characteristics of a color filter, a variable range of a focal length / focus / aperture of a lens, a variable minimum unit, a response characteristic to a command, a color temperature of illumination, and flicker characteristics. The image processing device according to claim 1, wherein:   2. The image processing apparatus according to claim 1, wherein the photographing conditions include information on start and end of exposure of an image sensor of each camera head, readout time, and a time difference therebetween.   The image processing apparatus according to claim 1, further comprising a display unit that displays an image obtained by the image processing unit. An image processing method of an image processing apparatus connected to a camera having a plurality of camera heads via a network,
A reading step of reading a plurality of photographed images obtained from the plurality of camera heads and photographing conditions of each camera head,
An image processing method for executing a process of generating an image viewed from a specified viewpoint in accordance with a user operation using the plurality of captured images and the capturing conditions. .

Images