JP3744995B2 - Imaging method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging method and an apparatus therefor.
[0002]
[Prior art]
Recently, remote video conferences and desktop conferences have been held via public lines and private lines. Some video cameras at this time are fixed-point cameras, and others perform operations such as zooming and panning from a remote location.
When operating the camera from a remote location, send the corresponding data by sending the command of up / down / left / right, zoom, focus while watching the projected image, or touching the icon corresponding to the command and selecting it .
[0003]
Furthermore, as disclosed in JP-A-6-205409, by pointing a graphic bar or the like provided at the bottom or right of the display screen, an angle corresponding to the graphic bar can be given as a command. The calculation at this time is obtained from the focal length and the angle of view received from the camera.
In addition, it has been proposed to perform panning and zooming on a document by specifying a center image or an area with a mouse.
[0004]
[Problems to be solved by the invention]
However, pointing on the screen and specifying the area is easy to specify with a mouse, etc., but there are also those that involve commands or those that are specified outside the screen or move for unknown images. I had to use other commands and key operations together.
[0005]
Conventionally, the angle of view has been calculated by receiving information such as the focal length from the other side, but it is difficult to automatically adjust the angle of view with a simple system without specific information from the remote camera. Met.
The present invention has been made in view of the above-described conventional example, and provides an imaging method and apparatus capable of easily and appropriately performing control of the imaging apparatus without knowing physical parameters on the imaging apparatus side. With the goal.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, the imaging of the present invention has the following configuration. That is, a display step for displaying an image sent from the imaging device, and a command interpretation step for interpreting a command for instructing the operation of the imaging device based on a gesture input to the image displayed in the display step A feature amount extracting step of extracting a feature amount of a predetermined feature of an image portion designated based on a gesture input to the image, a feature amount extracted in the feature amount extracting step, and the command interpreting step And a control step of controlling the imaging device to change the displayed image based on the command interpreted in
The gestures that can be input as the gesture are:
A gesture for controlling the imaging device to change the displayed image with a portion designated based on the gesture as a new image center ;
The control step calculates a distance and an angle of view between the designated portion and the current image center based on the feature amount of the designated portion and the feature amount of the current image center, and calculates the calculated distance and image Based on the corner, a temporary movement command for moving the basic unit by a movement amount is calculated, the calculated temporary movement command is issued to the imaging device, and the center of the image after the imaging device is moved by the temporary movement command is calculated. It is determined whether or not the feature amount and the feature amount of the designated portion match, and if the feature amount at the center of the image and the feature amount of the designated portion do not match, it is determined that the feature amount is matched. Repeatedly issuing a correction command for correcting the temporary movement command based on the feature amount of the image center and the feature amount of the designated portion to the imaging device, and the feature amount of the image center after the move and the designated portion of If it is determined that the symptoms amount and match, issuing a movement of the stop command to the imaging apparatus.
[0007]
Another invention interprets a command for instructing the operation of the imaging device based on a display step for displaying an image sent from the imaging device and a gesture input to the image displayed in the display step. A command interpretation step, a feature amount extraction step of extracting a feature amount of a predetermined feature of an image portion designated based on a gesture input to the image, and a feature amount extracted in the feature amount extraction step And a control step for controlling the imaging device to change a displayed image based on the command interpreted in the command interpretation step, and the predetermined feature is designated based on the gesture And a positional information of a region represented by the hue in the image portion .
[0008]
DETAILED DESCRIPTION OF THE INVENTION
In accordance with a gesture input from the pen, a camera operation command is issued while comparing the original image and the current image so that the camera moves in the designated direction and size.
(First embodiment)
FIG. 1 shows a configuration diagram of a camera control apparatus according to an embodiment of the present invention.
[0009]
The display control unit 1 performs display control for displaying the contents of the application, the video of the built-in camera, and the video of a remote place sent via various communication means.
The display device 2 is composed of a CRT, liquid crystal, or the like, and 3 is a pen input means such as a digitizer that is integrated with the display device.
[0010]
Reference numeral 4 denotes gesture command analysis means for interpreting a command inputted from the shape, position, size, direction, etc. of the line locus inputted by the pen input means.
Reference numeral 5 denotes feature extraction means for extracting features from video received by communication, and reference numeral 6 denotes a reference buffer for storing the features of the target image when a gesture command is designated from the pen input means.
[0011]
A current image buffer 7 stores image characteristics of the current video.
8 is a comparison means for comparing the reference and the current image in accordance with the gesture command to obtain the degree of achievement of the command, and 9 is a camera that controls the camera according to the displayed screen according to the comparison result. Camera operation command control means for giving a command for feedback. Commands to the camera are sent to a built-in camera or a remote camera.
[0012]
FIG. 2 shows the configuration of the camera device.
Reference numeral 10 denotes a camera unit which receives an image passing through a lens by a light receiving element such as a CCD, and further sends it to the main body in a standard signal format such as NTSC or a digital signal.
Reference numeral 11 denotes an external I / O that receives a control command sent from the main body or a remote place.
[0013]
A camera control unit 12 interprets commands and controls each device.
Reference numeral 13 denotes a horizontal movement motor for panning the camera, and reference numeral 14 denotes a tilt movement motor for performing vertical movement.
Next, FIG. 3A shows an example of an internal block of an information processing system in which a camera unit and a camera control body are integrated.
[0014]
A 32- or 16-bit CPU 15 controls the entire device and executes applications.
Reference numeral 16 denotes a peripheral I / O controller for controlling CPU and peripheral I / O, that is, interrupt control, serial / parallel communication, RTC, and the like.
Reference numeral 17 denotes a RAM / ROM used for a control execution program and a work area.
[0015]
Reference numeral 18 denotes an HDD (hard disk) for storing applications and user data.
A digitizer 19 designates an area on the image shown on the display, and converts a pen locus used for menu selection and character input into coordinate values.
Reference numeral 1 denotes a display control controller for displaying a screen on the display device (liquid crystal) 2.
[0016]
Reference numeral 20 denotes an audio control controller that controls audio reproduction via the microphone / speaker 21.
A camera control unit 12 interprets a camera command instructed by the outside or the main body and controls the camera as described above.
A communication control unit 22 controls data exchange with a device such as a computer or a FAX from a telephone line via the modem 23. This may be communication via a LAN or the like. Further, the communication medium at that time may be wired or wireless, and the data may be in an analog format or a digital format.
[0017]
Next, the external appearance of the entire apparatus of FIG. 3A is shown in FIG. 3B.
FIG. 4 is a list of gesture commands that are actually used.
(1) is a command to move the camera so that the point becomes the center of the image, and the gesture is to tap the point once.
Note that a round dot in the operation column indicates the tap position, and a type arrow means that the round dot position should be noted. The type arrows are not actually entered.
[0018]
(2) is a command for moving the first pen-down point to the pen-up point, and the gesture is tapping and dragging.
The type arrows are for indicating tap points and are not actually input. (3) is a command for focusing on the point, and the gesture is a double tap.
[0019]
In addition, although two round dots are placed a little apart, this is for the sake of clarity in order to clearly show that there are two, and actually means that they are tapped twice at the same position. The type arrow means that the point position should be noted, and is not actually input.
(4) is a command for moving the camera in the direction of an arrow written with a pen and stopping the movement by inputting a tap. The gesture is an arrow and a tap.
[0020]
(5) is a command for zooming in to a region surrounded by a double arrow, and the gesture is a double arrow.
(6) is a command for zooming in, and zooming in on the size of the next large circle centered on the first small circle written, and the gesture is a small circle, an arrow, and a large circle.
[0021]
(7) is a command that zooms out around the first pointing point and stops at the tap, and the gesture is a pigtail and a tap.
(8) is a command that zooms out until the size of the next small circle is written, centering on the first large circle written, and the gesture is a large circle, an arrow, and a small circle.
[0022]
Since these command interpretation methods are extensions of gesture commands used in existing pen computers, they are not specifically described here.
Next, a description will be given with reference to the flowchart of camera control in FIG.
First, in step S1, pen input coordinates from the digitizer are read.
In step S2, it is interpreted which command is instructed how.
[0023]
In step S3, it is determined whether or not a command (4, 7 in FIG. 4) that is interrupted by a tap is currently being executed. If so, a stop command is issued in step S4 and the process ends.
Conversely, if not, the process proceeds to step S5.
In step S5, an initial setting calculation for obtaining a reference described later is performed.
[0024]
Here, the feature of the image at the designated target point is extracted.
Here, as features , for example, paying attention to a 3 × 3 block near the designated target point, for example, two representative hues are extracted from a plurality of pixels included in the block. Then, the relative position information in each block where each hue and each hue are located becomes the above-mentioned reference.
[0025]
Here, the hue is raised as a feature of the block, but the present invention is not limited to this, and it goes without saying that a feature that characterizes the color such as lightness and saturation may be combined.
Then, in order to actually achieve the gesture command, the first control command to be given to the camera is calculated. For example, in order to bring the tap point in (1) to the center of the image, the relationship between the tap point and the image center is obtained and a temporary movement command is issued.
[0026]
Here, the provisional movement command means a command to move by the movement amount of the basic unit because there is no information regarding the angle of view in the camera unit and the actual movement distance to the tap point. Basically, it is possible to finally move to a target point by issuing a command for moving by the movement amount in the basic unit a plurality of times.
That is, an accurate camera movement amount cannot be calculated unless the current camera status (distance to the target, lens angle of view) is known. In this embodiment, feedback is applied while viewing the target point of the image. .
[0027]
The temporary movement command is calculated based on, for example, the X and Y distances, the temporary angle of view, and the temporary target distance from the tap point to the image center.
Further, in the command (4), the direction of the arrow is calculated, and in the command (6, 8), the ratio of large and small circles is calculated.
In step S6, the calculated command is actually issued to the remote camera.
[0028]
In step S7, the target point of the current image is evaluated. In step S8, it is determined whether the image requested by the gesture command is obtained. If the image corresponds to the command, a stop command is issued in step S9 and the process ends. Otherwise, a correction command is issued in step S10. With the above hypothetical numbers, feedback will always be required because errors occur during movement. If the direction is correct but has not yet reached, no correction command is issued.
[0029]
As described above, since commands can be given directly on the display while viewing the display with pen input, the operation method is very easy to understand.
Furthermore, in the case of remote operation, since control is performed based on an image sent from the other party, if there is a conventional camera capable of remote operation, control can be performed without adding anything.
(Second Embodiment)
In the above-described embodiment, the operation of the camera finds a coincidence point or the operation is terminated by a pen tap.
[0030]
On the other hand, in recent years, when collaborating between remote locations, research on awareness systems to grasp each other's situation has been conducted. For example, when you actually visit the other party, you look around the room when you are away, like looking around with a camera when the other party is not in front of the display. To do.
[0031]
At this time, it cannot be found in one direction as in the above-described embodiment, and cannot be used for monitoring purposes.
In the present embodiment, the gesture can be further expanded and the movement can be continued until a tap stop such as a reciprocating movement or a circular movement is made.
Examples of gestures are shown in commands (9) and (10) in FIG.
[0032]
First, (9) is a command for reciprocating the range indicated by the trajectory at a constant speed, and the gesture is a horizontal eight character. (10) is for writing a double circle at the same place, and is a command for moving the camera circularly at a constant speed.
A flowchart in this embodiment is shown in FIG.
In FIG. 6, the same step numbers as those in FIG. 5 are the same control. Step S11 is different from FIG. This time, since it continues to move until the tap is entered, in the case of the reciprocating motion of (9), when it reaches one end point, the reference target point is switched to the other end point so that it moves there. the can exchange cut. In the case of circular motion, the direction must always be moving, and a movement target is set for each short section, and the circular motion is realized while frequently switching the direction.
[0033]
As described above, by switching the target point as needed, the user can continuously move in the intended direction indicated by the pen locus, such as a reciprocating motion or a circular motion. The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. Needless to say, the present invention can also be applied to a case where the present invention is implemented by supplying a program to a system or apparatus. In this case, the storage medium storing the program according to the present invention constitutes the present invention. Then, by reading the program from the storage medium to the system or apparatus, the system or apparatus operates in a predetermined manner.
[0034]
As described above, according to the embodiment of the present invention,
1) While viewing the display with pen input, commands can be given directly on the display, so the operation method is very easy to understand and the camera can be operated easily.
2) Furthermore, in the case of remote operation, since control is performed based on the image sent from the other party, if there is a conventional camera capable of remote operation, control can be performed without adding anything. .
3) Furthermore, control that keeps the camera moving continuously, such as circular motion and reciprocating motion, is also possible, and the user's intention can be easily converted into camera motion.
[0035]
When the imaging target moves, the moving target point may not be searched by the adjustment method based on the image feature described above. In this case, the maximum value of movement is set in advance, and if the movement related to search exceeds the maximum value, a message indicating that search is impossible is displayed on the display screen, and the next command from the user is input. That's fine.
[0036]
【The invention's effect】
As described above, according to the present invention, the image pickup apparatus can be controlled easily and appropriately without knowing the physical parameters on the image pickup apparatus side.
[Brief description of the drawings]
FIG. 1 is a diagram showing functional blocks of a camera control unit according to an embodiment of the present invention.
FIG. 2 is a block diagram of a camera portion controlled from the camera control unit of FIG.
FIG. 3A is a block diagram of an information processing system in which a camera unit and a camera control unit are integrated.
FIG. 3B is an external view of an information processing system in which a camera unit and a camera control unit are integrated.
FIG. 4 is a diagram showing a list of gesture commands.
FIG. 5 is a process flowchart of a camera control unit according to the first embodiment.
FIG. 6 is a process flowchart of a camera control unit according to the second embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Display control 2 Display apparatus 3 Pointing device 4 Gesture command analysis means 5 Feature extraction 6 Reference buffer 7 Current image buffer 8 Comparison means 9 Camera operation command control means 10 Camera part 11 External I / O
12 Camera control 13, 14 Camera attitude control motor 15 CPU
16 Peripheral I / O
17 Memory 18 Secondary storage device 19 Digitizer 20 Voice control 21 Voice input / output 22 Communication control 23 Modem

Claims (8)

A display step of displaying an image sent from the imaging device;
A command interpreting step for interpreting a command for instructing an operation of the imaging device based on a gesture input with respect to the image displayed in the display step;
A feature amount extraction step of extracting a feature amount of a predetermined feature of an image portion designated based on a gesture input to the image;
A control step of controlling the imaging device so as to change a displayed image based on the feature amount extracted in the feature amount extraction step and the command interpreted in the command interpretation step;
The gestures that can be input as the gesture are:
A gesture for controlling the imaging device to change the displayed image with a portion designated based on the gesture as a new image center ;
The control step calculates a distance and an angle of view between the designated portion and the current image center based on the feature amount of the designated portion and the feature amount of the current image center, and calculates the calculated distance and image Based on the corner, a temporary movement command for moving the basic unit by a movement amount is calculated, the calculated temporary movement command is issued to the imaging device, and the image center after the imaging device is moved by the temporary movement command is calculated. It is determined whether or not the feature amount and the feature amount of the designated portion match, and if the feature amount at the center of the image and the feature amount of the designated portion do not match, it is determined that the feature amount matches. Repeatedly issuing a correction command for correcting the temporary movement command based on the feature amount of the image center and the feature amount of the designated portion to the imaging device, and the feature amount of the image center after the move and the designated portion of If it is determined that the symptoms amount and matches, imaging method and issues a movement of the stop command to the imaging apparatus. A display step of displaying an image sent from the imaging device;
A command interpretation step for interpreting a command for instructing an operation of the imaging device based on a gesture input with respect to the image displayed in the display step;
A feature amount extraction step of extracting a feature amount of a predetermined feature of an image portion designated based on a gesture input to the image;
A control step of controlling the imaging device to change a displayed image based on the feature amount extracted in the feature amount extraction step and the command interpreted in the command interpretation step;
The imaging method according to claim 1, wherein the predetermined feature includes a hue representing an image portion designated based on the gesture and position information of an area represented by the hue in the image portion.   The predetermined feature includes a color type representing an image portion designated based on the gesture and position information of an area represented by the color type in the image portion. Item 2. The imaging method according to Item 1.   The imaging method according to claim 3, wherein the color type is a hue. The gestures that can be input as the gesture are:
The imaging method according to claim 2, further comprising a gesture for controlling the imaging apparatus so as to change the displayed image with a portion designated based on the gesture as a new image center. . Display means for displaying an image sent from the imaging device;
Command interpreting means for interpreting a command for instructing the operation of the imaging device based on a gesture input to the image represented by the display means;
Feature amount extraction means for extracting a predetermined feature amount of an image portion designated based on a gesture input to the image;
Control means for controlling the imaging device to change a displayed image based on the feature amount extracted by the feature amount extraction unit and the command interpreted by the command interpretation unit;
The gestures that can be input as the gesture are:
A gesture for controlling the imaging device to change the displayed image with a portion designated based on the gesture as a new image center ;
The control means calculates a distance and an angle of view between the designated portion and the current image center based on the feature amount of the designated portion and the feature amount of the current image center, and the calculated distance and image Based on the corner, a temporary movement command for moving the basic unit by a movement amount is calculated, the calculated temporary movement command is issued to the imaging device, and the image center after the imaging device is moved by the temporary movement command is calculated. It is determined whether or not the feature amount and the feature amount of the designated portion match, and if the feature amount at the center of the image and the feature amount of the designated portion do not match, it is determined that the feature amount matches. Repeatedly issuing a correction command for correcting the temporary movement command based on the feature amount of the image center and the feature amount of the designated portion to the imaging device, and the feature amount of the image center after the move and the designated portion of If it is determined that the symptoms amount and matches, the imaging control unit and issues a movement of the stop command to the imaging apparatus. Display means for displaying an image sent from the imaging device;
Command interpreting means for interpreting a command for instructing the operation of the imaging device based on a gesture input to the image displayed by the display means;
Feature quantity extraction means for extracting a feature quantity of a predetermined feature of an image portion designated based on a gesture input to the image;
Control means for controlling the imaging device to change a displayed image based on the feature amount extracted by the feature amount extraction unit and the command interpreted by the command interpretation unit;
The imaging control apparatus according to claim 1, wherein the predetermined feature includes a hue representing an image portion designated based on the gesture and position information of an area represented by the hue in the image portion.   The imaging control apparatus according to claim 6, wherein the imaging apparatus and the imaging control apparatus are separate apparatuses.

Images