JP3426997B2 - Editing video shooting method and apparatus with subject information acquisition function, and storage medium storing program describing the method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for photographing a scene including a subject, and when photographing the scene, the subject in the scene designated by a user is tracked during or after the photographing. The present invention relates to a video shooting technique that enables editing work using the designated and tracked subject information at the time of editing.

[0002]

2. Description of the Related Art Due to the widespread use of small video cameras and VCRs, the number of people who record and shoot images has increased significantly.

However, editing a video requires a grasp of its contents and requires repetitive work such as repeatedly watching the video, which is a difficult task, and most people perform the editing work. There is a circumstance that there is no.

This seems to have some problems, but one of them is that it is difficult to obtain a good material. Also, even if good materials are available,
Since the currently popular video editing equipment is complicated to handle, only an expert can handle it, and most unskilled people cannot perform editing work. For example, finding a part of the material to be used for editing from the material is a complicated task, and it is not an easy task with the current editing equipment, and improvement is required.

Therefore, attempts have been made to enhance the functions of video cameras for the purpose of photographing good materials. For example,
(Japanese Patent Laid-Open No. 7-115584 "Image Shake Correction Device")
Aims to correct camera shake information that tends to occur when handling a small video camera, and can provide an image captured by stable camera work even if there is slight vibration due to camera shake.

[0006] This method is an effective method capable of photographing an appropriate material image by a relatively stable camera work rather than an unsightly image in which camera shake occurs. However, these videos are such that it is easy to shoot videos of higher quality, and although it can be expected to have the effect of making it easier to obtain video materials that are useful for editing, they cannot necessarily be said to promote editing work.

[0007] On the other hand, it is necessary to grasp the contents and search for a necessary video from the video material in order to use it for editing. Methods to support such work have also been tried.
For example, (Yukinobu Taniguti, Yo
Shinobu Tonomura, "Panoram
aExcerpts: Extracting andp
acking Panormas for Video
"Browning" 1996 ACM Multim
edia 97).

This method is for turning on / off the camera for video information.
The information of the shot change corresponding to the FF is extracted and displayed, and in the case of a shot with camera work, the representative image that created the image of the entire camera is displayed on the screen by shifting the image by the amount of camera work. Several methods have been proposed, such as arranging them on paper or displaying them. This method is an effective method in which an image given as a material is classified into sections that are likely to be used, and an effect that the contents of the divided sections can be easily grasped can be expected. In this method, it is assumed that the image captured by the video camera is processed on another computer.

Further, there has been proposed an attempt to realize a method for summarizing the contents of such an image, dividing the intervals so that it can be easily used as a material, and making it easy to grasp the contents in a video camera for photographing. For example, (Japanese Patent Laid-Open No. 10-28
No. 250 "Intelligent Video Camera and Intelligent Still Camera"). This method
The video, which is one section obtained during ON / OFF of the camera for shooting, is organized and classified, and a representative image, which is an image representative of each section, is displayed so that the contents can be grasped.

According to this method, since the photographed contents can be confirmed immediately after photographing, it is possible to photograph again and again until the image which can be used for editing is photographed.
It can be expected to have the effect of increasing the possibility of obtaining higher quality material images.

Further, this method enables a simple editing by combining the displayed representative images, and has a function of adding character information to the representative image. Such added text information can be expected to have an effect of grasping what kind of video is included in the material video when performing editing work based on the captured video, and is effective in acquiring the video for editing. It appears to be.

[0012]

By the way, images taken by a general person with a video camera are roughly classified into two. One is a case where a specific subject is focused and is tracked and photographed, and the other is a case where an entire scene unrelated to the specific subject is photographed. The former example is often seen when photographing a friend or friend at an event. As a typical example, images of children at the athletic meet,
It is an image of a friend at a wedding or the like. On the other hand, an example of the latter is to use camerawork to capture the beautiful scenery of the trip. Examples of each are shown in FIGS.

When shooting is performed with such a specific subject as the center, the subject is often used even in editing using the image. Of course, the meaning of video is very ambiguous, so the idea that the photographer wants to shoot a specific subject is very often reflected in the editing.

However, in the conventional method (intelligent video camera, intelligent still camera, etc.), no reference is made to such an object. Therefore, it is necessary to search for images containing a specific subject one by one during editing from images captured centering on the specific subject. It is hard to say that this realizes prompt material selection.

That is, in the above-mentioned conventional invention, the case where a specific subject is mainly photographed and edited is
The problem is that we cannot provide sufficient support.

As described above, in order to make it easy to edit a video imaged around a specific subject, centering around the specific subject,
There is a need to develop a method, device and system for organizing and summarizing videos and reducing the work burden on editors, and it is an object of the present invention to provide such a method and device.

[0017]

The present invention solves the above problems by the following means.

That is, one means thereof is a photographing step of photographing a subject, a subject designation step of designating the photographed subject, a video recording step of recording the photographed video, and the recorded video. Video playback process for reading
Includes a subject reflecting step of reflecting the subject information obtained for the designated object in said read image, and a video operation step of operating an image in which the subject information is reflected, the subject reflecting step Then, the subject
Information read out by the video playback process
To show the presence of the subject in the video
An editing video image capturing method with a subject information acquisition function, which is characterized by including steps .

According to this method, when the subject is photographed, the subject in the image is designated, and when the editing work is performed, the recorded image and information about the designated subject are reflected in the image. Since these can be handled, the information of the subject intended by the photographer can be clearly grasped during the editing work including the handling work. Therefore, it is possible to support shooting and editing a video centering on the subject.

Alternatively, a photographing step for photographing a subject,
Subject designation step of designating the subject being photographed
And a first subject information obtaining step of obtaining and recording first subject information representing the area including the designated subject, and analyzing the image information photographed by the photographing step.
Is the physical feature amount of the specified subject.
The second subject to acquire and record the second subject information
An information acquisition step, the image SL <br/> recording process of recording the captured image, and the recorded first or second of the Utsushitai information reproduction process to read out <br/> subject information , The recorded video
The video playback process to read and the previous video in the read video.
Subject reflection process that reflects the retrieved subject information
And the video operation for operating the video in which the subject information is reflected.
A method for photographing an image for editing with a subject information acquisition function, which is characterized by having a work process .

According to this method, when a subject is photographed, it is possible to specify the subject in the image, acquire the information while photographing, and record the obtained subject information together with the photographed image. Then, when performing editing work, etc., it is possible to read the recorded video and subject information, and handle them while the subject information is reflected in the video, so that the subject information intended by the photographer is It is possible to clearly understand during editing work including this handling work. Therefore, it is possible to support shooting and editing a video centering on the subject.

Alternatively, a photographing step for photographing a subject,
Subject designation step of designating the subject being photographed
And a first object representing an area including the specified object
It acquires body information, a first object information obtaining step of recording, a video recording process of recording the captured image, before
And video reproduction step of reading the serial recording video, the video reproduction by analyzing the image information read by the process, the second is a physical feature value for the specified object
A second subject information obtaining step of obtaining and recording subject information of the subject, and the obtained subject information in the read video.
A subject reflecting step of reflecting the second subject information, and
And a video operation step of operating a video in which the subject information is reflected .

According to this method, the subject information can be extracted from the information when the image is reproduced at the time of editing, etc., based on the subject designated at the time of shooting. By doing this,
It is possible to perform the operation of detecting the specified subject not only in the video section shot while specifying the subject, but also in the video section shot without specifying the subject in particular, which can be one of the effective processes at the time of editing.

Alternatively, in the above-mentioned method of photographing an image for editing with a subject information acquisition function, the subject information is a physical feature quantity including one or more of the position, shape, color, brightness, and movement information of the subject. This is a method for shooting an image for editing with a subject information acquisition function.

According to this method, the position, shape, color, brightness, moving direction of the object specified by the user at the time of shooting,
It is possible to measure physical quantities such as distance. This makes it possible to track the subject and detect the subject in response to relative changes in the subject within the video, such as camerawork and movement of the subject during shooting. When editing an existing video, the position, shape, color, etc. of the subject in the video to be edited
Brightness, movement information, etc. can be specified. As a result, the editor can perform the editing work while clearly grasping the subject, and can perform the simpler editing work.

Alternatively, in the above-mentioned editing video shooting method with a subject information acquisition function other than the first one, in the subject information reflecting step, the subject information is superimposed on the video read in the video reproducing step. A method for photographing an image for editing with a subject information acquisition function, characterized by having a step of clearly indicating the presence of a subject in the image.

According to this method, when editing a video in which a designated subject exists, the subject is reflected in the video to be edited, and the subject can be specified. As a result, the editor can perform the editing work while clearly grasping the subject, and can perform the simpler editing work.

Alternatively, in the above-described method of photographing an image for editing with a subject information acquisition function, in the image operation step,
A method for photographing an image for editing with a subject information acquisition function, comprising a step of editing and processing an image including the subject information.

According to this method, it is possible to handle the recorded image and the subject information while designating the subject and perform the editing work. As a result, it is possible to edit a video that includes the subject immediately after shooting, and if the video is not suitable for editing, you can shoot again, which increases the chances of acquiring better materials. It is thought that there is.

Further, another means of the present invention comprises a photographing means for photographing a subject, a subject designating means for designating the photographed subject, and an area including the designated subject.
First acquires the photographic material information, a first object information acquisition means for recording, images captured by the imaging means representing
Analyzing information to determine the physics of the specified subject
Second subject information, which is a dynamic feature amount, is acquired and recorded.
A second object information acquiring unit, and the video recording means for recording the captured image, and the object information reproducing means for reading the first or second <br/> object information before type recording, the recording
A video reproducing means for reading the read video, a subject reflecting means for reflecting the read subject information in the read video, and a video operating means for operating the video on which the subject information is reflected, An editing image capturing device with a subject information acquisition function, which is characterized by being provided.

This device is a device obtained by implementing the above-described editing video shooting method.

Alternatively, in the editing video image pickup device with a subject information acquisition function, the photographing means, the subject designation means, the subject information acquisition means, the video recording means, and the subject information recording means are combined into a single unit. An image capturing apparatus for editing with a subject information acquisition function, characterized by being housed in a housing.

This device has a function of recording video information and subject information in the same housing, and can carry and record the subject and video in various places such as outdoors and indoors. To

Alternatively, in the above-mentioned editing video photographing apparatus with a subject information acquisition function, the photographing means, the subject designation means, the subject information acquisition means, the video recording means, the subject information recording means, the video reproduction means, The subject information reproducing means, and the subject information reflecting means,
However, it is an editing image capturing device with a subject information acquisition function, which is housed in a single housing.

This device has a function of photographing, recording, and reproducing the information of the image and the subject in the same housing, and can be carried around to record the subject and the image at various places such as outdoors and indoors. It is possible to record and read out such information, reproduce it, and view it.

Alternatively, in the above-mentioned video recording apparatus for editing with a subject information acquisition function, the photographing means, the subject designating means, the subject information acquiring means, the video recording means, the subject information recording means, the video reproducing means, The subject information reproducing means, the subject information reflecting means, and
In the image capturing apparatus for editing with a subject information acquisition function, the image operating means is housed in a single housing.

This device has a function of capturing an image and a subject, acquiring information, recording and reproducing the information, and performing an operation including an editing operation on the information in the same housing. The device can be carried and used in various places outdoors and indoors.

Further, still another aspect of the present invention includes a photographing means for photographing an object, and the object specifying means for specifying an object that is the shot, the designated subject
Gets the first object information indicative of a region, the video reproduction to read the first <br/> subject information acquiring means for recording, a video recording means for pre-type recording a captured image, the recorded image Means and the video read by the video playback means
Analyzing information to determine the physics of the specified subject
Second subject information, which is a dynamic feature amount, is acquired and recorded .
A second object information acquiring unit, wherein during the read image, and the object reflecting unit for reflecting the second object information the acquired, and the image operating means for manipulating the image in which the subject information is reflected Is a video recording apparatus for editing with a subject information acquisition function.

This device is also a device obtained by implementing the above-described editing video shooting method.

Alternatively, in the above-mentioned image-taking device for photographing with a subject information acquisition function, the photographing means, the subject designating means, the image recording means, the subject recording means,
However, it is an editing image capturing device with a subject information acquisition function, which is housed in a single housing.

This device has a function of recording video information and subject information in the same housing, and can carry and record the subject and video at various places such as outdoors and indoors. To

Alternatively, in the above-described image-taking device for image-capturing with a subject information acquisition function, the image-taking means, the object-designating means, the image-recording means, the object-recording means,
The image reproducing means, the subject recording / reproducing means, the subject information acquiring means, and the subject information reflecting means,
It is an editing image capturing apparatus with a subject information acquisition function, which is housed in a single housing.

This device has a function of photographing, recording, and reproducing the information of the image and the subject in the same housing, and can be carried around to record the subject and the image at various places such as outdoors and indoors. It is possible to record and read out such information, reproduce it, and view it.

Alternatively, in the video recording apparatus for editing with a subject information acquisition function, the photographing means, the subject designating means, the video recording means, the subject recording means, the video reproducing means, the subject recording / reproducing means, the subject information. An editing video image capturing apparatus with a subject information acquisition function, characterized in that an acquisition unit, the subject information reflection unit, and the video operation unit are contained in a single housing.

This apparatus has a function of realizing shooting of a video image and a subject, acquisition of information, recording and reproduction of the information, and execution of an operation including editing work on the information in the same housing. The device can be carried and used in various places outdoors and indoors.

It should be noted that it is possible to record a program for causing a computer to execute the steps of the above-described editing video image capturing method in a computer-readable storage medium.

According to the storage medium in which the program describing the editing video image capturing method is recorded, the method of the present invention can be distributed or stored as a storage medium, and the computer of the present invention can be used by using a computer. The method can be realized.

[0048]

BEST MODE FOR CARRYING OUT THE INVENTION Specific embodiments of the method and apparatus for photographing an image for editing with a subject information acquisition function of the present invention will be described below.

First, FIG. 1 shows a flow chart of an embodiment of an editing video shooting method with a subject information acquisition function.

First, there is a scene 0 to be photographed.
Here, it is assumed that there is a subject that the photographer 2 particularly wants to photograph. By the photographing step 11, this scene is captured as the image 1 including the subject, which is the image including the subject. Next, when the photographer 2 wants to select a specific subject in the image, the subject is designated in the subject designation step 12, and the designated subject 3 is acquired. The information of the designated subject is first recorded in the subject information recording step 15. The recorded content is the specified position, time,
Physical quantities such as the shape, the area, the ratio of the area, and the distribution of the color and the brightness in the range designated as the subject are recorded.

On the other hand, the image 1 containing the subject is recorded as the image record 5 in the image recording step 14 while being requested by the photographer. The means to record the video used here is
It does not matter in particular, but it goes without saying that the recorded video must be read out and reproduced as a video in the video reproducing step 16. The video recording 5 also records various information such as ON / OFF of the camera, lens parameters, and time information.

On the other hand, based on the designated subject 3 designated by the photographer 2, one of the subject-containing image 1 immediately after photographing and the image information 7 read out from the image recording 5 by the image reproducing step 16, or , Object information 4 from both images
To extract. The subject information extracted here specifies the designated subject, and the position, movement distance, shape,
It is calculated as a physical quantity such as a change in color or brightness. The subject information 4, which is also such a physical feature amount, is recorded as the subject information record 6 in the subject information recording step 15 together with the designated subject 3. This medium and shape are not specified, but in the subject information reproducing step 17,
It must be read out and can be taken out as the subject information 8. Through this series of processing, for a specified subject at a certain point in time when the scene was photographed, the image that is being captured and recorded, or once captured, subject information is extracted based on the recorded image and saved. You are doing it.

In each case, the image information 7 and the subject information 8 which have been recorded and read once are subjected to the subject reflecting step 18,
Integrate and reflect the subject information on the video. This reflection method may announce the existence of the subject and display it as numerical information via characters, etc., or based on physical quantities such as the position, place, shape change, and movement distance of the subject, shooting The format must be understandable by the person 2 and the editor 10.

The editor 10 can operate the subject information reflected image 9 which is the image in which the subject information is reflected, through the image operation step 19. The result of the operation is output as the operated image 20. The video operation step 19 includes various operations such as rearranging the contents of the video and performing special marking on the video.

FIG. 2 shows an example of the first embodiment of the editing video image pickup device with the subject information acquisition function.

In the first embodiment, the scene 3 with a subject is included.
For 0, the photographer 50 makes full use of the photographing means 60 to acquire the image 31 including the subject. This photographing means 60 is C
The optical information is converted into electronic information using a CD or the like, and the information is converted so that it can be handled by an electric circuit. The size, the pixel, the angle of view at the time of shooting, and the focus are not particularly limited.

If there is a particular designated subject in the image 31 including the subject, the photographer 50 designates the designated subject 32 by the subject designation means 61 to determine the designated subject 32. Video 3 with these subjects
In order to store 1 and the designated subject 32, they are stored in the recording medium 52. The subject specifying means will be described in detail later.

Next, the subject-containing image 31 and the designated subject 32 are analyzed, and the subject information acquisition means 62 outputs subject information (physical feature amount) 33. The subject information 33 is also stored in the recording medium 52.

At this point, the recording medium 52 has the video recording 34 and the subject information recording 35 stored therein. I don't specify the recording media,
It is preferable that the information can be sufficiently recorded and easily retrieved. For example, it may be a magnetic disk, a medium using optical information, or a tape medium. The recording media for storing the video recording and the subject information do not have to be the same or common. For example, the image information may be stored on a magnetic tape, and the subject information may be stored on a magnetic disk. When the media are separated, information may be taken out from each media so that they can be synchronized. Regarding the information to be recorded, the recording format, size, shape, and recording medium are not particularly limited, but may be in any format that can reproduce the original information in a substantially perfect state.

Video recording 34 recorded on the recording medium
Can be read from the recording medium 52 by the video reproducing means 63 and reproduced as the video information 36.
On the other hand, the subject information record 35 includes the subject information reproducing means 64.
Thus, it is read from the recording medium 52, extracted as the subject information 37, and stored in the memory or the like.
The video recording 36 and the subject information 37 are combined by the subject reflecting means 65 to form a subject information reflecting image 38 in which the subject information is reflected. The subject information reflecting image produced by the subject reflecting means using the read image record and subject information record will be described in detail later.

Next, the created subject information reflection image 3
8, the editor 51 can perform some operations according to a predetermined procedure by the video operating means 66. The editor 51 who performs the operation is the photographer 5
The same person as 0 may be used, or different persons may be used. The number of people is not particularly limited. The operations that can be added include, for example, normal reproduction of the subject information reflecting image 38, special reproduction, rearrangement of the arrangement, addition of information not directly related to the image or the subject, and arrangement of the images. It is possible to change the order and change the frames that make up the video. This video operation means will also be described later.

Before explaining each function, an outline and an operation image thereof are shown in FIGS. The photographer operates the photographing and recording device shown in FIG. For operations after shooting and recording, the device shown in FIG. 5 is used. The devices shown in FIGS. 4 and 5 are constituted by one housing, and are one example of the devices for realizing the above-described first embodiment.

FIG. 4 shows an overview of the apparatus and its operation image when video shooting is performed. Scenery 10 including the subject
1 using the photographing lens 105, the photographing optical axis 1
The range 102 shown by 04 is photographed. An image reflecting the captured scene is displayed as a captured image 107 via the liquid crystal monitor 106. The photographer
When it is desired to focus on a particular subject in the captured image 107, the subject 103 is designated using the subject-designating pen 108. For example,
Enclose the subject in a rectangular parallelepiped or other shape, and thus specify the subject in the video being shot. On the other hand, the captured video is recorded in response to the command from the photographer. The already recorded video is displayed on the liquid crystal sub-monitor 112 as a representative image group 109 of the recorded video. It should be noted that various control buttons 110 and the shooting control button 111 perform shooting parameters such as an angle of view at the time of shooting, a focal length, and an aperture, and operations such as shooting start and end, recording start and end, update and removal of a specified subject. Do it. It goes without saying that the number of buttons may be appropriately changed depending on the function.

Next, an overview image for reproducing and displaying the photographed image will be described with reference to FIG. The captured image is the image 1 being reproduced on the LCD monitor 120.
Played as 22. At this time, it is also possible to display a representative image group of the video imaged at a time other than the video image being reproduced, as shown at 124. Then, the basic operation relating to video reproduction such as start and end of reproduction is performed using the reproduction control button 126. In addition, special playback such as fast-forward, rewind, high-speed playback, and slow playback may be performed, which may be performed via a control command on the screen or via an operation button as indicated by 127. You may realize it. It should be noted that an operation may be added to the video during reproduction. A video operation pen 121 may be provided as one of the input means for applying such an operation. Note that, in the image 122 being reproduced and displayed on the liquid crystal monitor 120, when a designated subject exists, a frame that clearly indicates the presence of the subject may be displayed. The presence or absence of this frame display may or may not be present.

Based on the fact that the photographer operates the device and the operation image thereof, the subject specifying means in the first embodiment will be described.

When shooting is performed, an image of some scene is displayed on the liquid crystal monitor. Photographer, when a specific object is located in the video specifies a region including an object using the object designation pen. The designated result is reflected as the designated subject in the video being shot. It should be noted that when the designation is made, in the present embodiment, a pen-shaped device designated by wire is used as the device, but another method of designating the region may be used. For example, it may be an application of the line-of-sight input technology, a mouse or the like may be used, or a touch panel may be provided on the liquid crystal monitor to input with a finger or the like. In this way, the subject is designated and recorded. The fact that the subject record is stored independently of the video record is as described above.

The image is time-series information, and even if the subject is designated at a certain point, the subject may change due to conditions such as camera work, the environment surrounding the subject, and movement of the subject itself. . in this case,
Two treatments are possible. The schematic diagram is shown in FIG.

In the first process (FIG. 6A), in addition to image recording, information of a subject when the subject is designated (position, shape, range, brightness of the designated subject, color, area, etc.) The specified timing is recorded and saved so that the recorded video can be synchronized when it is read out. In this process, information other than when the subject is designated is not acquired, so that it is difficult to completely deal with a change in the subject. Therefore, it may be necessary to perform a process of complementing the information by passing the information to the subject information acquisition means (or process) after recording the image.

On the other hand, in the second process ((2) in FIG. 6), for a subject once designated, the subject is tracked in the image being photographed, the subject information is sequentially updated and stored as a record. To go. In this case, it is necessary to follow up sequentially. For tracking a subject, tracking is performed by sequentially estimating a region similar to a region including a designated subject. The subject information that is sequentially calculated as the estimation result and the synchronized video are sequentially recorded.
Similarly to the first process, since the video recording and the subject information are separately recorded, it goes without saying that the sequentially calculated subject information and the video information are associated with each other.

In tracking the object, the regions are estimated by block matching as shown in FIG. In a region P (k, t) of an image f (t), a frame to be searched, (generally the next frame) f (t +
1) to search for p (k, t + 1), p (k,
The area corresponding to t) is moved variously, the difference is calculated for each corresponding pixel in the area, and the area when the difference becomes the smallest is set as the subject area. Information such as lightness and color is used as the information used when performing the difference calculation. In this embodiment, the RGB lightness difference sum is used as the information for the difference calculation.

Regarding the processing relating to the subject information,
Either the first function or the second function is selected depending on the usage form and whether or not there is a capability to acquire information.
From the point of view of information, it is desirable to be able to obtain sequentially, but it is not essential.

Next, a detailed description will be given of the subject information reflected image produced by reflecting the subject using the image record and the subject information record by using the obtained subject information. The video is recorded and read as a video by shooting. At the same time, the information recorded with the video recording,
For example, various information such as time, camera ON / OFF, and lens parameters are also read. On the other hand, information about the designated and extracted subject is also recorded and read.
In order to utilize the read information, these are combined and presented as necessary.

In the subject reflected image, when the image is reproduced, the information of the subject is also displayed. When playing the image in normal or fast-forward, the position of the subject in the image,
The location, shape, color, etc. are specified. In addition, in order to make it useful for editing or the like, there are cases where only a part of the representative images are presented without showing all the images. This is editing work,
This is a very effective method for checking whether or not you are shooting. It is also applied in conventional technology, but when checking whether the intended image is taken immediately after shooting,
If there is an image that represents the video being shot,
It is easy to understand the contents. On the other hand, in editing work,
It's also an effective way to find good materials. Up to now, among the conventional methods, an image immediately after a shot change corresponding to ON / OFF of a camera, a paranoma image reflecting camera work, and the like have been used. Specifically, it is reflected in the format as shown in (4) of FIG.

The information on the subject can also be fully utilized in a similar manner. In particular, the subject information according to the present embodiment is a subject designated by the photographer with particular attention, and it is easy to determine that the photographer is particularly important. For example, the presence / absence of a subject is determined, and only a shot section including the designated subject is handled. A method of presenting a video in which the subject is reflected, such as presenting a representative image of a section including the subject by emphasizing the subject, can be considered. This example is presented in a format like (5) in FIG. As shown in FIG. 4, such a representative image is displayed on a liquid crystal or other dedicated liquid crystal monitor during shooting or immediately after shooting to check the recorded image while shooting. I can do things. Further, when the video is reproduced, it is also displayed on the liquid crystal monitor or sub monitor. This makes it easier for the photographer and the editor to understand the content of the video including the subject. That is, in the present embodiment example, the representative image is displayed on the sub-monitor or the like in FIGS. 4 and 5, and the format is, in addition to the image immediately after the shot change as shown in FIG. 8, the image including the subject information as the representative image. Used.

Next, the operation means for adding an operation to the image will be described.

One of the operations added to the image is normal reproduction and special reproduction. Play the specified video at normal speed, change the size of the frame to play, change the speed to play, change the time or spatial resolution to play, or reverse the time direction. It is a method of playing. In addition, a reproduction method such as immediately after a shot change or only an image including the designated subject information is reproduced.

When representative images are presented on a monitor made of liquid crystal or other material, they are input and designated by making full use of a pen or other input device system, and corresponding to the representative images. The video that is playing. For example, reproduction is performed in the form as shown in (1) of FIG.

Further, the simple editing is realized by further proceeding.
For example, as shown in (3) of FIG. 9, a portion displaying a representative image and the representative image are selected, arranged on a specific area, and the video is reproduced along with the selected area. This allows
Simple rearrangement can be performed, which is effective for rough editing. It is an easy extension of the present embodiment that such operation information is stored as it is, output, and reflected in the actual editing work.

Further, as a special operation method, an application is possible in which certain subject information is designated and a video containing the same information as the subject information is searched. Images containing the same subject can be searched and images containing the same subject can be output. In this method, an area similar to the subject information is calculated by the same method as the block matching, and when there is an area smaller than a certain threshold value, an image in which the same subject exists is output.

This is one of the embodiments of the method of the present invention, but it is different not only by specifying a subject in a video and tracking the subject existing in the video at the time of shooting or reproducing. Estimate the subject in the video over time. This holds physical information of a subject designated in advance, and judges the presence / absence of the subject using the held subject information with respect to the video imaged at another time and place.
For images at different times, the image is recorded and held in advance, and this is read out as needed. Meanwhile, the specified subject information is also read. With regard to the image and subject information to be read out and the subject to be tracked and the subject to be tracked, the user who operates this may select and give an instruction, or to all recorded information. You may process automatically. It should be noted that the presence / absence of the subject and the tracking may be performed by using the comparison method based on the physical quantity described in other examples of this embodiment, and the details thereof will not be described here.

Next, FIG. 3 shows an example of a second embodiment of an editing video image pickup device with a subject information acquisition function. This embodiment example uses a circuit having the same function as that of the first embodiment example. It goes without saying that the same name has the same function.

In the second embodiment, the scene 3 with a subject is included.
While storing the subject-included video 31 from 0 in the recording medium 52, the subject-designating means 61 performs the designation work on the subject-included video 31 for the subject that the photographer 50 wants to designate, and the designated subject 32. To create. The designated subject 32 is also stored in the recording medium 52. Up to this point, there is no difference from the first embodiment.

Next, in the first embodiment, the specified subject is processed from the image containing the subject, and the subject information is extracted by the subject information acquisition means. However, in the second embodiment, the video information 36 obtained by the video reproducing means 63 read from the video recording 34 stored in the recording medium 52, and the designated video information 36 also stored in the recording medium 52. Based on the subject record 321, subject information acquisition means 62 causes subject information (physical feature amount).
33 is calculated. The subject information 33 is recorded on the recording medium 5
2 and can be read out at any time. The subject information 33 thus calculated is reflected in the image information 36 by the subject reflecting means 65, and the subject information reflected image 38
To create. This subject information reflected image 38 is also edited by the editor 5 using the image operation means 66 as in the first embodiment.
The processed image 39 can be created by the processing of step 1. It is described above that the series of processing functions performed from the video recording stored in the recording medium 52 and the designated subject recording to the construction of the operated video are the same as those in the first embodiment. On the street.

The timing of performing the processing is different between the first embodiment and the second embodiment. That is, in the first embodiment, the subject information is calculated using the captured image and the subject specified at the time of shooting, whereas in the second embodiment, the recording medium is once recorded. From the video recording recorded in and the designated subject recording,
The difference is that the subject information is calculated. In other words, the first embodiment is to extract the subject information at the same time as the process of shooting and recording the video, and the second embodiment is to separately extract the subject information after shooting and recording the video. It is a form example.

Incidentally, FIG. 10 shows an outline of the apparatus and an operation image of the second embodiment. This is a shoot
This is a case in which the device for designating a subject and fetching various information of the subject and the image and the function of processing them and operating the image in which the subject is reflected are separated not in the same housing but separately. When shooting, it is better to miniaturize the device as much as possible
Needless to say, it is convenient to use. Therefore, only the minimum devices necessary for shooting are stored in the same housing, and functions and devices that may be used even after shooting are stored in another housing. It is obvious that there may be a plurality of housings that house these post-shooting functions. The shooting function and the functions required after shooting may be connected by wire or wirelessly, and the recording medium recorded may be portable so that the information can be transferred from the shooting device to the device after shooting. , Or vice versa. The above is also applicable to the first embodiment.

This method is described as a computer program, has a function of acquiring an image, and has a recording medium (for example, a recording medium for recording various kinds of information centering on the image and the subject)
ROM, FD, CD, DVD, removable disk,
By using a computer having a memory card or the like) and having an arithmetic function of reading and executing a program on this recording medium, it is possible to sufficiently construct the same processing.

It should be noted that the above-described embodiments are some embodiments for reproducing the essence of the function, and even if some easily inferrable functions and specifications are changed, they are within the scope of the present invention. Is.

[0088]

According to the present invention, when an editor intends to edit a video centering on a subject, the video is specified by the photographer. Since it is possible to organize and construct video using the information of the subject, it is effective for simple video editing.

[Brief description of drawings]

FIG. 1 is a flowchart showing an example of an embodiment of an editing video shooting method with a subject information acquisition function of the present invention.

FIG. 2 is a block diagram showing an example of a first embodiment of an editing image capturing device with a subject information acquisition function of the present invention.

FIG. 3 is a block diagram showing a second embodiment of an editing image capturing device with a subject information acquisition function of the present invention.

FIG. 4 is a diagram showing an outline of a device and an operation image at the time of shooting an image according to the first embodiment.

FIG. 5 is a diagram showing an outline of a device and an operation image at the time of reproducing a video according to the first embodiment.

6 (1) and 6 (2) are diagrams showing an example of a subject information reflecting image in the embodiment of the present invention.

7 (a), (b) and (c) are diagrams for explaining a tracking algorithm of a subject region in the embodiment of the present invention.

FIG. 8: (1), (2), (3), (4), (5)
It is a figure which shows the example of a to-be-photographed object reflected image in the example of embodiment of this invention.

9 (1), (2), and (3) are diagrams showing an example of video operation in the embodiment of the present invention.

FIG. 10 is a diagram showing a device overview and an operation image of a second embodiment example of the present invention.

FIG. 11 is a diagram showing an example of an image centering on a specific subject.

FIG. 12 is a diagram showing an example of capturing a landscape video.

[Explanation of symbols]

0 ... Scene with subject 1 ... Video with subject 2 ... Photographer 3 ... Specified subject 4 ... Subject information (physical feature amount) 5 ... Video recording 6 ... Subject information recording 7 ... Video information 8 ... Subject information 9 ... Subject information reflected image 10 ... Editor 11 ... Shooting process 12 ... Subject designation process 13 ... Subject information acquisition process 14 ... Video recording process 15 ... Subject information recording process 16 ... Video playback process 17 ... Subject information reproduction process 18 ... Subject reflection process 19 ... Video operation process 20 ... Operated video 30 ... Scene with a subject 31 ... Video with subject 32 ... Specified subject 321 ... Record of specified subject 33 ... Subject information (physical feature amount) 34 ... Video recording 35 ... Subject information recording 36 ... Video information 37 ... Subject information 38 ... Subject information reflected image 39 ... Operated video 50 ... Photographer 51 ... Editor 52 ... Recording medium 60 ... Photography means 61 ... Subject designation means 62 ... Subject information acquisition means 63 ... Video playback means 64 ... Subject information reproducing means 65 ... Subject reflection means 66 ... Video operation means

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-9-247707 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 5/76-5/956

Claims (15)

(57) [Claims] 1. A photographing process for photographing a subject, a subject specifying process for designating the photographed subject, a video recording process for recording the photographed video, and a video reproducing process for reading the recorded video. When having a subject reflecting step of reflecting the subject information obtained for the designated object in said read image, and a video operation step of operating an image in which the subject information is reflected, the subject In the reflecting step, the subject information is read by the video reproducing step.
On the captured image, revealing the existence of the subject in the image
An editing video shooting method with a subject information acquisition function, which is characterized by having a step of showing . 2. A photographing step of photographing a subject, and a subject designation step of designating the photographed subject.
A first subject information obtaining step of obtaining and recording first subject information representing a region including the designated subject, and analyzing the video information photographed by the photographing step,
The first is a physical feature amount of the specified subject.
Second subject information to acquire and record the second subject information
An acquisition step, wherein the image recording step of recording the captured image, wherein the recorded first or second of the <br/> Utsushitai information reproduction process to read out the subject information, the recorded video And an image reproduction step of reading out the read object information in the read out image.
Subject reflecting step of reflecting the subject, and a video manipulator operating the video reflecting the subject information.
Degree and a subject information acquisition function with editing video imaging method which is characterized in that it has. 3. A photographing step of photographing a subject, and a subject designation step of designating the photographed subject
And a first subject information representing an area including the designated subject.
Gets the broadcast, the first object information obtaining step of recording, a video recording process of recording the captured image, and video reproduction step of reading the recorded video, read by the image regeneration step By analyzing the video information, the physical features of the specified subject
A second object information acquisition step of acquiring and recording certain second object information, and the acquired second object information in the read image.
A subject reflecting step of reflecting body information, and a video manipulator operating a video reflecting the subject information.
Degree and a subject information acquisition function with editing video imaging method which is characterized in that it has. 4. The editing video image capturing method with a subject information acquisition function according to claim 1, wherein the subject information is one of a subject position, shape, color, brightness, and movement information. An editing video shooting method with a subject information acquisition function, which is a physical feature quantity including one or more. 5. The editing video image capturing method with a subject information acquisition function according to claim 2 , wherein in the subject information reflecting step, the subject information is read by the image reproducing step. An editing video shooting method with a subject information acquisition function, comprising a step of superimposing the presence of a subject in the video overlaid on the projected video. 6. Any one of claims 1, 2, 3, 4, and 5.
In the video shooting method for editing with a subject information acquisition function, the video operation step includes a step of editing and processing a video including the subject information, and a video shooting for editing with a subject information acquisition function. Method. 7. A photographing means for photographing an object, and the object specifying means for specifying an object that is the photographing, the first object scene object information <br/> paper representing a region including the specified object Analyzing the first subject information acquisition means for acquiring and recording and the image information photographed by the photographing means,
The first is a physical feature amount of the specified subject.
Second subject information to acquire and record the second subject information
Acquisition means, video recording means for recording the captured video, subject information reproducing means for reading the previously recorded first or second subject information, and video reproducing means for reading the recorded video, Subject information acquisition function, comprising: subject reflecting means for reflecting the read subject information in the read video, and video operating means for operating the video in which the subject information is reflected. Video equipment for editing. 8. The editing video image capturing apparatus with a subject information acquiring function according to claim 7, wherein the image capturing unit, the subject specifying unit, the subject information acquiring unit, the image recording unit, and the subject information recording unit. , Is housed in a single housing, and is a video recording apparatus for editing with a subject information acquisition function. 9. The editing video image capturing apparatus with a subject information acquiring function according to claim 7, wherein the image capturing unit, the subject specifying unit, the subject information acquiring unit, the image recording unit, the subject information recording unit,
An image capturing apparatus for editing with a subject information acquisition function, wherein the image reproducing unit, the subject information reproducing unit, and the subject information reflecting unit are housed in a single housing. 10. The editing video image capturing apparatus with a subject information acquiring function according to claim 7, wherein the image capturing means, the subject specifying means, the subject information acquiring means, the image recording means, the subject information recording means,
An image capturing apparatus for editing with a subject information acquisition function, characterized in that the image reproducing unit, the subject information reproducing unit, the subject information reflecting unit, and the image operating unit are contained in a single housing. . 11. A photographing means for photographing a subject, a subject designating means for designating the photographed subject, and first subject information representing an area including the designated subject.
It gets the broadcast, the first object information acquisition means for recording, a video recording means for recording images that have been pre-Symbol captured, and image reproducing means for reading the recorded video, read by the image reproducing means Analyze video information
Then, with the physical feature amount of the specified subject
To obtain a certain second object information, and the second object scene <br/> body information acquiring means for recording, during said read image, the second object scene <br/> said acquired An image capturing apparatus for editing with a subject information acquisition function, comprising: a subject reflecting unit that reflects body information; and a video operating unit that operates a video on which the subject information is reflected. 12. The editing video image capturing apparatus with a subject information acquiring function according to claim 11, wherein the image capturing unit, the subject specifying unit, the image recording unit, and the subject recording unit are provided in a single housing. A video recording device for editing with a subject information acquisition function characterized by being stored. 13. The image-capturing apparatus for editing with a subject information acquisition function according to claim 11, wherein the image-capturing unit, the subject-designating unit, the image-recording unit, the subject-recording unit, the image-playing unit, and the subject-recording unit An editing image capturing apparatus with a subject information acquisition function, characterized in that a reproducing unit, the subject information acquiring unit, and the subject information reflecting unit are housed in a single housing. 14. The image-capturing device for editing with a subject information acquisition function according to claim 11, wherein the image-capturing unit, the subject-designating unit, the image-recording unit, the subject-recording unit, the image-playing unit, and the subject-recording unit An editing video image capturing apparatus with a subject information acquisition function, characterized in that a reproducing means, the subject information acquiring means, the subject information reflecting means, and the video operating means are contained in a single housing. 15. A computer-readable storage medium for storing a program for causing a computer to execute the steps of the editing image capturing method according to any one of claims 1, 2, 3, 4, 5, and 6. A storage medium having a program recorded therein, which describes a method for capturing an image for editing, which is recorded on the medium.