JP2000251450A - Picture detecting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate the generation of an IDX icon and to clearly indicate an M-icon, which has conducted a detection process, and an M-icon, which has not conducted the process, by storing detection information relative to a specific person stored in the data structure of an IDX icon into the data structure of an M-icon. SOLUTION: A detction process is conducted for a person A in an ID window and the resulting data are stored in a data structure corresponding to an attached M-icon, Moreover, an M-icon, which becomes a detection object, is changed while conducting a detection process for a person B and a person C. Having completed the detection process for the persons B and C, an M-icon is attached again and an IDX window is opened. The portion of nondetected segments 250 to 252 represents the frame information of an M-icon for which no detection process is conducted. Thus, the M-icon, which has conducted a detection process against a person who is a detection object, and the M-icon, vhich has not conducted the process, are displayed on an ID window.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image editing apparatus, and more particularly to an image detecting method used when editing by operation on a display screen.

[0002]

2. Description of the Related Art In recent years, diversification and enhancement of software have been rapidly progressing along with the increase in speed and function of computers. By using these computers and software, production facilities for producing television broadcast programs and video programs are becoming more sophisticated.

[0003] In particular, in recent program production, a video tape recorder relied on a tape counter instead of a video editing method of repeating fast-forwarding and rewinding, but an information storage device capable of random access (eg, handling and storage). Video information is edited using a moving image editing apparatus having an easy and inexpensive hard disk. By adopting more sophisticated and diversified computers and software for such moving image editing devices,
Increasingly sophisticated moving picture editing devices are being developed.
Here, the video information is all information relating to video including audio.

FIG. 3 is a block diagram showing an example of the configuration of a moving image editing apparatus. 3010 is a CPU (Central Processing Uni
t: central processing unit), 3020 is a memory, 3030 is a cut change point detector, 3040 is an image reproducing device, 3050 is a video interface, 3060 is an information storage device, 3070 is a frame buffer, 3080 is a monitor, and 3090 is an input device. , 3100 is a bus,
Reference numeral 3110 denotes an image signal output terminal. This moving image editing apparatus has a CPU 3 that controls and edits each device constituting the moving image editing apparatus under the control of software (program).
010 and a hierarchical structure (tree structure) representing control information of the CPU 3010, moving images, scene images, cut images, and editing status
Monitor 3080 that displays information about editing, such as
And an input device 3090 such as a mouse or a keyboard for selecting information displayed on the screen of the monitor 3080 and inputting a command to the CPU 3010, and a frame buffer memory for storing image information to be displayed on the monitor 3080. 3070, an image reproducing device (for example, VTR: Video Tape Recorder) 3040 for reproducing an image signal of a moving image used for editing from a recording medium and outputting the signal to a video interface 3050, and a moving image input from the image reproducing device 3040. The image signal is converted into image information in a format handled by the moving image editing device, and output to an information storage device (for example, a magnetic storage device such as a hard disk) 3060, a cut change point detecting unit 3030 and a memory 3020, and the moving image editing device A video output from a video signal output terminal 3110 which converts image information of a moving image edited in a format handled by The video interface 3050 compares the adjacent frame images of the frame images constituting the moving image, and if the image information of the difference between the frame images changes by more than a predetermined amount, the adjacent frame images are different cut images. The cut change point detection unit 3030, which determines and belongs to the group, stores control software for the CPU 3010, and stores various control information (frame numbers of a series of moving images, detected cut change The frame number of the point, the display image (M-Icon: Moving Icon) in which the image size displayed on the screen of the monitor 3080 is reduced, the M-icon file name, the cut image number sequentially assigned to the cut image in order, A memory 3020 for temporarily storing sequence numbers indicating the order of a plurality of cut images, etc.) and a video interface reproduced from the image reproducing device 3040. The image information of the moving image used for editing converted to the required format in the memory 3050 is stored, and the editing information and control information used by the moving image editing device temporarily stored in the memory 3020, such as an M-icon, are stored. An information storage device 3060 using, for example, a hard disk that is randomly accessible, and a bus 3100 used for digital information transmission between devices in the moving image editing device.
And an image signal output terminal 3110 for outputting image information of a moving image.
It has.

The bus 3100 includes a CPU 3010, a memory 3020, a cut change point detecting unit 3030, a video interface 3050, an information storage device 3060, a frame buffer memory 3070, a monitor 3080, and a CPU 3010 of each device constituting the moving image editing device. An input device 3090 is connected, and the CPU 3010 sends and receives various signals to and from these devices, and these devices are controlled by access signals from the CPU 3010. The instruction to the CPU 3010 is input from an input device 3090 such as a mouse or a keyboard.

[0006] By using the moving picture editing apparatus having the above-mentioned structure and having the required software in the memory 3020, the image information of the moving picture to be edited is stored and managed in the memory 3020 and the information storage device 3060. can do.
Still further, the moving image editing device displays on the screen of the monitor 3080,
Editing of moving images by displaying cut images and scene images of moving images stored in the memory 3020 and the information storage device 3060 as reduced M-icons and forming a tree structure in units of cut images and scene images Can work.

Here, a cut image, a scene image, an M-icon, and the like will be described. A cut image is a set of frame images shot from the start to the end of one shot by a television camera. Further, a “scene image” is a set of a plurality of cut images obtained by shooting a predetermined shooting target, and image data constituting a moving image stored in the information storage device 3060 or the like is a set of a plurality of scene images. It is. A plurality of scene images can be further compiled under a specific shooting theme to create an edited moving image (program) of the specific shooting theme.

[0008] As described above, a moving image edited under a specific shooting theme includes several cut images selected from a plurality of cut images, which are image materials, according to the specific shooting theme, and a plurality of cut images. An image having a hierarchical structure (tree structure) such as a scene image in which several selected cut images are grouped together and a moving image having a shooting theme (or title) in which the scene images are grouped under a specific shooting theme. It can be organized as data. In addition, the tree structure includes a frame number of a series of moving images, a frame number of an image of a detected cut change point, a cut image number serially assigned to the cut image, and a plurality of cut images constituting a scene image. Are arranged and arranged, and the above-described control information includes the hierarchical structure information. An M-icon is a GUI (Graphical User Interface) corresponding to a set (cut image) of frame images from the start to the end of one imaging operation.
e: User interface using graphics)
Parts. The M-icon is a display image obtained by reducing the size of the image at a required ratio, and is used when displaying the whole or a part of a moving image including a plurality of frame images or a scene image at a high speed (for high-speed display). ) Or as a representative image of the cut image.

The display screen of the monitor 3080 is a GUI screen for displaying editing information such as a cut image used for editing a moving image, a scene image, and a tree structure indicating an editing state. The editor uses an input device (eg, a mouse) 3090 to
An instruction can be given to the moving image editing apparatus by selecting the editing information on the screen of the monitor 3080 that displays the editing information.

As described above, the moving image editing apparatus described above can create a program by storing and managing moving images to be edited as a tree structure, and can perform editing work in units of cut images and scene images. It has become.

FIG. 4 shows a specific example of the moving picture editing apparatus shown in FIG. 3 in which a news program is assumed as an editing material and another news program is created for each content (for example, a digest is created). Monitor 3080 if you like)
FIG. 7 is a diagram showing an example of an editing window displayed on the screen of FIG. 1000 is an editing window, 1010 is an editing area, 10
20 is the material area, 1030, 1031, 1032, 1033, ‥‥‥, 10
3K, 103M, 103N, and ‥‥‥ are M-icons of the image material displayed in the material area 1020 (however, K, M, and N are natural numbers, K = N-2, M = N-1), 1040 Is a tree structure composed of M-icons representing the edited contents of the edited moving image, 1050 is an editing function button group for selecting various control functions used for editing work, 1051 is a character detection button, 1052 is a tree structure creation Button, 1070 is a slider that slides the display screen of the editing area 1010 in the horizontal direction, 1080 is a slider that slides the display screen of the editing area 1010 in the vertical direction, 1090 is a slider that slides the display screen of the material area 1020 in the vertical direction, Reference numeral 1210 denotes an M-icon representing a cut image registered in the editing area 1010, reference numeral 1220 denotes an M-icon representing a scene image constituted by one or a plurality of cut images, and reference numeral 1230 denotes an M-icon constituted by one or more scene images. Icon, 1240 is edited according to the theme Moving image representing the (program) M- icon 810 is a mouse pointer displayed on the GUI screen when performing GUI operations using a pointing device such as a mouse.
Although the corresponding reduced display image is displayed on the surface of the M-icon, it is complicated, and therefore is omitted in this specification unless otherwise required.

Referring to FIG. 4, a tree structure 1040 composed of M-icons in an editing area 1010 is composed of one or more M-icons 1210 representing cut images selected for editing. At the top of the hierarchy, an M-icon 1220 representing a scene image is configured, and further, an M-
One or more icons 1220 are grouped together to form an M-icon 1230 representing a set of specific scene images at the top of the hierarchy, and finally a plurality of M-icons 1230 representing a set of specific scene images. M-icons 1240 representing programs that are gathered and edited according to the theme are configured.

The number of hierarchies of the tree structure 1040 is arbitrary, and it is needless to say that hierarchies can be formed further above. Since the size of the screen of the editing area 1010 is limited, the entire tree structure 1040 is not displayed in FIG. In order to display this part that is not displayed,
By operating the slider 1070 or the slider 1080 with the input device 3090 (FIG. 3), the entire tree structure 1040 is viewed by moving the display in the required horizontal or vertical direction.

In the material area 1020, M-icons created from different cut images detected by the cut change point detection unit 3030 in FIG.
Icons 1030, 1031, 1032, 1033, ‥‥‥, 103K, 103
It is temporarily placed as M, 103N, @, and is displayed so that it can be viewed. Editing work proceeds, images are called, and M-icons 1030, 1031, 1032, 1033,.
At the stage when 3K, 103M, 103N, and 作成 are created and displayed, M-icons 1030, 1031, 1032, 1033, ‥‥‥, and 103 are displayed.
K, 103M, 103N, and ‥‥‥ are displayed in the material area 1020 in chronological order from left to right and from top to bottom.

The M-icon 103 in the material area 1020
0, 1031, 1032, 1033, ‥‥‥, 103K, 103M, 103N, ‥
As for ‥‥, similarly to the editing area 1010, when the number of M-icons is large and is not displayed on the screen at one time, the slider 1090 is operated by the input device 3090 to move the display in the vertical direction. View all M-icons.

In order to perform editing, this material area 1020
, Select the M-icon required for editing, and then press the tree structure creation button 1052 in the function button group 1050. As a result, the selected M-icon 1030 is displayed in the editing area 1010.
Moved or copied to In this way, the editing is advanced by configuring the tree structure 1040. In FIG. 4, the editing has progressed to some extent, a tree structure 1040 has already been formed in the editing area 1010, and the M-icons 1030, 1031, 1032, 1033, ‥‥‥, 103K, 103M, Ten
Select the desired M- icon from 3N and ‥‥‥
This is the state where editing is continued from. Also, material area 1020
And the M-icon displayed in the editing area 1010 are drawn as one card in the drawing, but this is an abbreviated drawing, and the M-icon is displayed on the display screen of the actual monitor 3080. Are displayed in a shape in which the number of frame images included in each of the M-icons overlaps.

The editing of the moving image proceeds as described above. The image material that is actually required for creating the program includes the image information (cut image or one moving image) stored in the information storage device 3060 in FIG. It is unlikely that all of a series of moving images read as a file). That is, material area 10
Multiple M-icons 1030, 1031, 103 displayed on 20
All of 2,1033, ‥‥‥, 103K, 103M, 103N, ‥‥‥ are not required, only the image material associated with some M-icons selected for editing purpose or imaging theme. Used for programs. Further, since the image material usually includes a plurality of scenes, the information storage device 3060
Many M-icons 1030, 1031, 1032, 1033, ‥‥‥, 103K, included in the image material or image information read from
The M-icon required for editing is searched from 103M, 103N, and ‥‥‥.

The operation of the moving picture editing apparatus according to the above-described conventional moving picture editing method and the method of editing a moving picture while creating a tree structure using M-icons are described in, for example, JP-A-10-51733. It is disclosed in the gazette. Also,
As means for detecting a change point of a cut image, for example,
Japanese Patent Application Laid-Open No. 2-184181 discloses a technique of analyzing illuminance, volume, color tone, and the like of a moving image and detecting a portion where these changes are remarkable as a change point of the cut image.

By the way, in order to edit a moving image as described above, a plurality of M-icons 1030, 1031, 1032, 1033,..., 103K, 103M,
It is necessary to find the M-icon required for editing from 103N, ‥‥‥. The function to support searching for the necessary M-icon is the character detection function. This character detection function has a function of detecting a specific person from image materials or a plurality of cuts or scenes. This particular person is temporarily assumed to be “X”.

This character detection function is selected by the operator by combining the features of a material or a plurality of cuts or scenes, that is, the characteristics of the person “X” in the detected section.
A frame and a pixel region corresponding to the combination of the colors are found using an image recognition technique.

The operator selects a feature (for example, a person “X”) of a person (for example, a person “X”) to be detected in a series of selected M-icons (that is, a detected section in which a character appears to be present). Specify the combination of hair color, skin color, and clothing color). As a result, the moving image editing apparatus detects a video area that matches the specified color combination by image processing, and displays the information of the matched video area and its frame using the GUI.
Display on the screen.

The operation procedure of the character detection function will be described with reference to FIGS. 4, 2 and 13.

FIG. 4 shows an example of the editing window displayed on the monitor 3080 (FIG. 3) in the conventional nonlinear editing apparatus, which has already been described.

In FIG. 4, the operator sets an M-icon corresponding to the cut or scene to be detected in the editing area 10 in order to detect a person (for example, a person "X").
Select with the mouse pointer 810 from 10 or the material area 1020 and click the mouse. At this time, the M-icon is selected, and setting the M-icon to the selected state in this way is called attachment. This display of the selected state is performed by, for example, enclosing the selected M-icon with a thick frame,
Change the display color of the icon, blink the M-icon,
And so on.

For example, the M-icons 1031 to 1033 in the material area 1020 are attached, and then the editing function button group 10
The character detection button 1051 in 50 is clicked. As a result, the window shown in FIG. 2 is displayed on the monitor 3080. On this window, the process of detecting the person “X” is performed. FIG. 2 is a diagram for explaining how to determine a feature of a person to be detected when performing person detection and perform person detection. 300
Is the ID window, 301 is the OK button, 302 is the Cancel button, 303 is the slider knob, 304 is the slide cursor,
305 is an enlarged display image, 306 is an enlarged display area, 307 is an object button, 308 is a color designation frame, 309 is a search button,
311 is a still image view slider, 312 is an object name entry field, 313 is an object frame adjustment button, 314 is a detection section addition button, 315 is a detection section deletion button, 321 to 330 are frame images, 809 is a film image, and 810 is a mouse It is a pointer.

In FIG. 2, frame images 321-330 included in the attached M-icons 1031-1033 are displayed on the film image 809. Frame images 321-2
30 are arranged in order along the time axis, the left and right parts of the screen that are not displayed are the still image view slider 311
By scrolling the slider knob 303 to the left or right, the displayed frame image can be slid and viewed. This scroll processing can be operated by, for example, moving the mouse pointer 810 to the slider knob 303 and dragging with the mouse, or attaching the arrow portion of the slide cursor 304. An enlarged frame image 305 is displayed in an enlarged display area 306 at the lower left of the ID window 300. The enlarged frame image 305 is an enlarged display of the leftmost frame image 321 among the frame images 321 to 330 in the film image 809. In order to detect the person "X", the feature to be detected as the person "X" is determined using the enlarged display image 305. The frame image 321 displayed on the far left
Is linked with the enlarged display image 305, the operator scrolls the frame image in the film image 809 to display the enlarged image 305 in the enlarged display area 306 as an enlarged display image 305 for use in the detection processing of the person "X". Can be done. The operator scrolls the frame image in the film image 809 and looks at the enlarged display image 305 to select a frame image that is considered to be optimal for the process of detecting the person “X”. Next, a characteristic color combination possessed by the person “X” is designated from the enlarged display image 305. Of the frame images 321 to 330, the frame image 321 displays a simple image, but the other frame images 322
The display of images is omitted for items # 330.

To specify a characteristic color combination, click the object button 307, and then specify a color specification frame 308 that defines a range for specifying a characteristic color from the enlarged display image 305. Do it. The specification of the color specification frame 308 is
This is the same as the method of creating a rectangular frame in drawing on a personal computer or the like. Needless to say, after designation, the color designation frame 308 can be moved and its size can be changed. The color of the portion surrounded by the color designation frame 308 is the characteristic color of the person “X” (the pixel area occupied by the person “X” in the frame image). In the example of FIG. 2, the color of the clothes worn by the person “X” is specified, but in addition, for example, a hair color, a skin color, a clothes color, and the like are specified. The data of this color combination is used for the detection processing. The above-described character detection function is disclosed in, for example, Japanese Patent Application Laid-Open No. 6-223179.

After the color designation is completed, a search button 309
When the button is clicked, detection processing is performed on the M-icons 1031 to 1033. After performing the detection process, the detection result for the person “X” is displayed on the ID window 300 as shown in FIG. 13 (the description of FIG. 13 will be described later). Click the OK button 301 to close the ID window to keep the detection results, or click the Cancel button 302 to not keep the detection results.
Click to close the ID window.

Two results are obtained by this character detection function. One is information on a detection section indicating where the person "X" exists in the image material or the cut, and the other is at what position (for example, left, center, etc.) of the frame image in the detection section. Position information indicating whether a person “X” exists at the right, upper, lower left, etc.).

However, in the current character detection function, erroneous detection or oversight is likely to occur due to the problem of recognition accuracy. Therefore, it is necessary for the operator to check and correct the ID window manually using a mouse while looking at the ID window.

However, in the current character detection function, erroneous detection or oversight is likely to occur due to the problem of recognition accuracy. Therefore, it is necessary for the operator to check and correct the ID window manually using a mouse while looking at the ID window.

FIG. 8 shows the M-icons 1031 and 10 shown in FIG.
This is an example of an edit window of the result of selecting characters 32 and 1033 and performing character detection as described above. Here, the same components as those in FIG. 4 are denoted by the same reference numerals.
3 and 14 are IDX icons.

After performing the character detection, the ID shown in FIG.
When the OK button 301 of the window 300 is attached, the ID window 300 is closed while retaining the detection result, and the IDX icon 12 is displayed below the material area 1020 of the editing window 1000. Further, each time a character is detected by selecting another M-icon, the display of the IDX icons 13, 14, and さ ら に further increases.

Next, an example of a conventional GUI operation related to confirmation and correction of a person detection result will be described with reference to FIG. FIG.
9 is an example of a display of an ID window when a person detection result is confirmed or corrected. The same components as those in FIG. 2 are denoted by the same reference numerals. In addition, 80 is an ID window, 800, 801, 80
2, 802 ', 803 are detection arrows, 804, 805, 806, 807, 808 are object frames, 809' is a film image, 811, 812
Is the detection section, 813, 814, 823, 824 is the film image 80
Frame images arranged in order along the time axis from the left of the screen in 9 ', and 817, 818, 820, 821, and 822 are pixel regions occupied by the person "X" in each frame image. In the ID window 80 of FIG. 13, the detection arrows 800 to 803 displayed above the film image 809 '.
Are arrows indicating the start and end of the detection section. Detection arrow 80
0 and the detection arrow 802 indicate the detected head, which is called a begin point. Further, the detection arrows 801 and 803 are the detected end points, and are called end points. The section between the detection arrow 800 of the begin point and the detection arrow 801 of the end point is a detection section 812.
The section between the begin point detection arrow 802 and the end point detection detection arrow 803 is a detection section 811. Detection interval
Object frames 804 to 808 displayed on the frame images belonging to 811 and the detection section 812 indicate positions indicating the position of the person “X” in the frame image, and the character detection has been performed. Displayed as a result.

The correction of the detection section and the object frame is performed as follows. First, the correction of the detection section will be described. In FIG. 13, a case will be described in which the begin point of the detection arrow 802 is moved to the frame 813 one frame to the left to extend the detection section 811. The mouse pointer 810 is a GUI component that can be moved on the display screen by a mouse (not shown), and can be operated by a general pointing device in addition to the mouse. First, the user moves the mouse pointer 810 to the begin button 815 and clicks the mouse. Next, the mouse pointer 810 is moved to the frame 813 and the mouse is clicked. Then, a detection arrow 802 'indicating a new begin point is displayed. Then, by clicking the detection arrow 802 which is the original begin point, the detection arrow 802 representing the original begin point disappears. As a result, the detection section 811 is extended one frame to the left. Similarly, to modify the end point, click the end button 816,
After clicking the frame to be the end point, click the end point detection arrow to change the end point.

Next, the modification of the object frame will be described. This correction is also performed using the mouse pointer 810. In FIG. 13, a case where the object frame 806 of the frame image 814 is modified is taken as an example. In FIG. 13, an object frame 805 of a frame image 824 is a pixel region 8 indicating a person “X”.
The object frame 806 of the frame image 814 has a pixel area 8
It is slightly larger than 20, and is displayed at a position from the center. In this case, when the operator looks at the frame image 814, the position is shifted to the position of the object frame 806, and the pixel region of the person “X” is erroneously recognized as being at the center of the frame. X "pixel area
It is necessary to correct the size and position of the object frame 806 in order to match with 820. Object frame 804, 805, 806, 80
Modification of the size and position of 7,808, other than the method of deleting the current object frame and newly setting, for example, Microsof
It can be executed by a well-known method such as screen operation and object operation in Windows 95, which is one of the operating systems (OS) of the company t.

When the operation of correcting the object frame 806 to the pixel area 820 of the person “X” is performed, the pixel area of the person “X” for all the object frames after the object frame 806 ( For example, the pixel areas 821 and 822 of the person “X” in the object frames 807 and 808)
Is also corrected to the same size and position as the object frame 806.

When the cancel button 302 shown in FIG. 2 is attached during or after the detection of the characters, the detection result is canceled and the ID window 300 is closed. Also, I
When the OK button 301 of the D window 300 is attached, the ID window 300 is closed while retaining the detection result, and the IDX icon 12 is displayed below the material area 1020 of the editing window 1000 as shown in FIG. In the data structure of the IDX icon 12, information on the result of the detection process for the person "X" is stored. Also like this
The IDX icon appears in the edit window 100 to detect people.
Within 0, one is created for all M-icons attached at the same time. That is, every time a different M-icon is attached to perform the person detection process, a new IDX icon 13,1
4 and ‥‥‥ are created one after another. When the IDX information button 1053 is attached by double-clicking or selecting the IDX icon 12 with a mouse, an IDX window showing IDX information as shown in FIG.

FIG. 6 is a diagram showing a display example of the IDX window, in which the detection results for each of the persons "A", "B", and "C" are displayed. This is a result of performing detection processing on the same M-icon groups 1031 to 1033 described above for different persons. 600 is an IDX window, 603 is a detection range display area, 604, 605, and 606 are areas, 611 is a target display field, 614 is a detection section, 616 is a non-detection section, 631, 632,
633 is the target image, 634, 635, and 636 are personal names, 63
7, 638, 639 are detection section graphs, 650 is a detection result display column,
Reference numeral 640 denotes a slider for sliding the display of the detection result display column 650 in the horizontal direction, reference numeral 641 denotes a slider for sliding the display of the detection result display column 650 in the vertical direction, reference numeral 810 denotes a mouse pointer, and reference numeral 241 denotes a close button.

In FIG. 6, the detection range display area 603 on the IDX window 600 is displayed in chronological order in the horizontal direction. Furthermore, the detection range display area 603 is now divided into three areas 604 to 606. The areas 604 to 606 are divided according to the cut length (= the number of frames) of the M-icons 1031 to 1033 attached by the operator. That is,
The number of frames of the M-icon 1031 and the number of frames of the area 604 correspond, and the number of frames of the M-icon 1032 and the area 605 and the number of frames of the M-icon 1033 and the area 606 also correspond. In the target display column 611, target images 631, 632, 633 of the detected person and their names 631 are displayed.
4,635,636 are displayed vertically. For example, if the person “A” is detected, the target image 631 and the person name
634 is displayed in the target display column 611, and the detection section graph 637 is displayed in the detection result display column 650 on the right side thereof. Further, when the person “B” is detected, a target image 632 and a person name 635 are displayed in a target display column 611 below the person “A”, and a detection result display column 650 on the right side thereof.
, A detection section graph 638 is displayed. Similarly, the person "C"
Is detected, a target image 633 and a person's name 636 are displayed below the target display column 611, and a detection section graph 639 is displayed in a detection result display column 650 on the right side thereof. These target images 631, 632, and 633 are frame images of an enlarged display image used to acquire a characteristic color for detecting the persons “A”, “B”, and “C” (for example, FIG. 2 frame image 304). Each detection target name is displayed in the person names 634 to 636 below the target images 631 to 633. Detection interval graph 637, 6
Since the display fields of 38 and 639 are finite, the detection area may be large and not all the detection areas may be displayed at once. In that case, the display is scrolled in the horizontal direction by operating the slider 640. Further, even when the number of detected persons is large, all of the detected persons cannot be displayed at one time (in the example of FIG. 6, the number of persons that can be displayed at one time is three). In this case, the slider
Use 641 to scroll the display vertically. As described above, in the areas of the detection section graphs 637 to 639, the detection results of the persons “A”, “B”, and “C” executed for the cut, that is, the M-icons 1031 to 1033 are shown. I have.

Next, the detection result will be described using the detection section graph 637 as an example. The detection section 614 includes the M-icons 1031-1.
Regarding 033, a section where the person “A” is detected is shown between the areas 604, 605 and 606, and a non-detection section 616 is a section where no person is detected. For example, the area 604 is a time-series arrangement of a set of frame images associated with the M-icon 1031, and the area 605 is an M-icon 1032
Are arranged in time with respect to the set of frame images associated with the M-icon 1033. The area 606 is arranged in time with respect to the set of frame images associated with the M-icon 1033. In addition, one frame image is displayed with a predetermined width, and in FIG.
4 is a diagonal pattern, and the non-detection section 616 is displayed as being distinguished as white. This display of detection / non-detection may be any display as long as it can be distinguished.For example, the detection section 614 may be displayed in orange, and the non-detection section 616 that has not been detected may be displayed in gray. You may make it blink.

The detected section 614 and the non-detected section 616 are also displayed on the side of the M-icons 1031 to 1033 in FIG. 8 (not shown). FIGS. 12A, 12B and 12C show the M-icon 1 in FIG.
It is a figure showing that the result of person detection is displayed on the side of 031-1033. As in FIG. 6, the area 604 indicates the area of the M-icon 1031, the area 605 indicates the area of the M-icon 1032, and the area 606 indicates the area of the M-icon 1033. The width of the side of the M-icon is the number of frames (1
(Displayed with a certain thickness for each of the three frame images), so that the side of the M-icons 1031 to 1033 is displayed as oblique lines as much as the number of frames as the detection section 614. Also, the non-detection section 616 is displayed in white on the side surfaces of the M-icons 1031 to 1033 by an amount corresponding to the number of the frames. This display may be set so that it is always displayed as a detection result.
It may be performed when the operator attaches 631. Alternatively, either one may be displayed. FIG.
In the M-icons 1031 to 1033 shown in 2 (a), (b), and (c), display images obtained by reducing the images associated with the respective M-icons are actually displayed, but are omitted in FIG. ing.

When the detection section 614 in the area 605 of the detection sections 614 is attached, for example, the detection section graph 637 is obtained.
The pattern or color of the display of the detection section to which the area 605 belongs changes from, for example, purple,
-A display is made such that a purple spotlight is irradiated toward the icon 1032 (FIG. 8). With this function, it is easy to find which detection section belongs to which M-icon. To close the IDX window 600 in FIG. 6, the user clicks the close button 241.

[0044]

The above-mentioned prior art includes the following:
There are the following disadvantages.

In the conventional character detection function, one IDX icon is created in the material area for the attached M-icon. In other words, every time a different M-icon is attached and detected, new IDX icons are created one after another in the material area, making it difficult to see the material area.

Since the IDX icon does not display information on which M-icon has performed what kind of person detection processing, the operator needs to open the IDX window corresponding to the IDX icon to check, and it takes time to check. Was.

Since there is no function of deleting a row displayed for each person in the IDX window detection result, unnecessary detection results and the like cannot be deleted.

Since the data of the color combination used in the detection processing of the persons “A”, “B”, and “C” is not stored in the data structure of the IDX icon, the color combination already used in the detection processing is different. Could not be used to detect M-icons.

An object of the present invention is to provide a more sophisticated image detecting method which eliminates the above-mentioned drawbacks, enables simple screen operations, and has a higher function.

[0050]

In order to achieve the above object, an image detection method according to the present invention uses detection information on a specific person stored in a data structure of an IDX icon.
Store in the data structure of M-icon. This eliminates the need to create IDX icons.

The IDX window is displayed based on the detection information stored in the data structure of the M-icon. However, since some of the attached M-icons have not been subjected to the detection processing with respect to the specific image, a display for distinguishing whether or not the M-icon has been subjected to the detection processing is performed. In addition, an indication of whether or not this detection processing has been performed has been added to the side of the M-icon that has only displayed the detection section.

Further, a function for deleting a detection result displayed for each specific target image is newly added.

Further, a combine function for combining detection results of different color information (for example, when the color of clothes is different) for the same object during detection processing for various objects is provided. Provided. On the contrary, an uncombining (unbinding) function for restoring the combined detection result is also provided.

Further, a preview function for reproducing only a detection section obtained by the detection processing as a moving image, and an information function for adding information of an image obtained by performing a detection processing on a detection result and storing the information in a data structure are provided. Was provided.

Further, a function for freely setting the time-series width of the detection range display area is also provided. Thereby, the width of the display such as the detection section can be freely adjusted.

Further, for each image to be detected, the data of the combination of colors used for each detection process is stored in the data structure of the M-icon so that it can be reused.

[0057]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below. The operation procedure will be described with reference to FIGS. 4 to 10 in order from the state in which the detection processing has never been performed. FIG. 4 is a diagram showing an example of an editing window displayed on the screen of the monitor 3080 in the moving picture editing apparatus of FIG. 3, which has been described in the prior art. First, the operator detects the person "A"
Attach M-icons 1031 to 1033 corresponding to cuts or scenes to be detected. Then, the character detection button 1051 of the editing function button group 1050 is attached. As a result, the IDX window shown in FIG. 5 is displayed on the monitor 3080. FIG. 5 shows an example of the IDX window in a state where the person detection processing has not been executed yet (initial state). 6000
Is the IDX window, 237, 238, 239 are the detection result display columns, 24
0 is a new button, 241 is a close button, 242 is a search button, 243 is a join button, 244 is a unlink button, 245 is a preview button, 246 is an information button, 2
47 is a visible range adjustment slider, and 248 is a correction button. Other symbols are the same as those in FIG.

At this stage, since no detection processing has been performed on the M-icons 1031 to 1033, the IDX window 6000 also displays the detection result display columns 237, 238, and 237 in the target display column 611 as shown in FIG. 239 is displayed without any data. This is the initial state.

First, detection processing is started for the person "A". When the operator clicks the new button 240, an ID window 300 shown in FIG. 2 is displayed on the monitor 3080. In the ID window 300, the operator performs a process of detecting the person "A". The details of the operation of this detection processing are the same as those of the prior art, and thus the description thereof is omitted. After performing the detection process, the detection result for the person “A” is displayed on the ID window 300 as shown in FIG. If necessary, the operator corrects the detection result. After correcting the detection result, click the OK button 30
Clicking 1 closes the ID window 300. OK button
When 301 is clicked, the data of the detection result for the person “A” is stored in the data structure corresponding to the attached M-icons 1031 to 1033. However, when the upper layer of the tree structure, for example, the M-icon 1220 “i” is attached, the M-icon 1210 “b”, “c”,
The data of the detection result is stored in the data structure corresponding to “d”. In this way, the detection result data is always
The reason why the data is stored in the M-icon is to prevent the data of the detection result from being influenced by the editing operation in the editing area 1010. That is, the M-icon 1220 in the upper layer of the tree structure
"I" is the lowermost M-icon 1210 "B", "C",
If the data of the detection result regarding “d” is stored, it is necessary to change the data of the detection result when the M-icon 1210 “c” is removed from the tree structure.

When the ID window 300 is closed, the display returns to the IDX window display again. In this case, only the detection result regarding the person “A” is displayed in the IDX window as shown in FIG. FIG. 7 shows only the detection result regarding the person “A”.
It is a DX window. Reference numeral 6001 denotes an IDX window, 631 denotes a target image of the person “A”, 634 denotes a person name, 637 denotes a detection section graph, and other symbols are the same as those in FIG. Clicking the close button 241 closes the IDX window 6001 and returns to the state of the editing window 1000 in FIG.

Further, detection processing is performed on the persons "B" and "C". However, the M-icon to be detected is changed.
For example, when detecting the person "B", the M-icon 1
If you attach 031 and 1033 and detect the person "C", use M-
Consider attaching only icon 1032. The operation of this detection processing is also the same as that of the conventional technique, and thus the description is omitted. After the detection process for the persons “B” and “C” is completed, the M-icons 1031 to 1033 are attached again, and the IDX window is opened. FIG. 1 shows an example of the result. FIG. 1 shows an embodiment of the IDX window.
The same numbers are assigned to the same functions. Others
Reference numeral 600 'denotes an IDX window, and reference numerals 250, 251, and 252 denote undetected sections indicating regions in which no person detection processing has been performed. Undetected sections 250 to 252 represent frame information of M-icons for which no detection processing has been performed. That is, undetected section 25
1 indicates that the detection process is not performed on the person “C” for the M-icon 1031 (detection section 604). Hereinafter, this state is referred to as detection unprocessed. On the other hand, the fact that the detection processing has already been performed is referred to as detection processing completed. Similarly, the undetected section 250 has not been detected for the person “B” in the M-icon 1032 (detected section 605), and the undetected section 252 has
This indicates that detection has not been performed on the person “C” in the M-icon 1033 (detection section 606). As a result, it is possible to display, on the IDX window, the M-icon that has been subjected to the detection processing and the M-icon that has not been subjected to the detection process for the person to be detected.

The display of the unprocessed state of detection is also reflected on the side of the attached M-icon. For example, consider the detection result for person "B". The operation is performed by attaching the target image 632. When the target image 632 is attached, the M-icon 1032 has not been detected for the person “B”, and the width of the side of the M-icon indicates the number of frames of the M-icon itself. A color indicating the unprocessed undetected period 250 is displayed on the entire side surface of the M-icon 1032 as shown in FIG. FIGS. 9A, 9B and 9C show the M-icons 1031 to 1031 in FIG.
FIG. 35 is a diagram showing that a detection result of a person “B” is displayed on the side of 33. As in FIG. 6, the area 604 is M
The area of the icon 1031, the area 605 indicates the area of the M-icon 1032, and the area 606 indicates the area of the M-icon 1033.

As a result, the undetected M-icon can be specified even on the editing window 1000.

Further, since the data of the color combination used in the detection processing of the persons "A", "B", and "C" is stored in the data structure of the M-icon, the unprocessed M-icon portion is not detected. This can be applied to For example, the person "C"
Undetected sections indicated by undetected sections 251 and 252,
That is, the case where the detection processing is performed on the M-icons 1031 and 1033 will be described below. First, the target image 633 is attached, and then the detection button 242 is clicked. As a result, the M-icons 1031 and 1033 are detected using the color combination data used in the detection process of the person “C”.

FIG. 10 shows an example of the detection result described above.
FIG. 10 is a diagram showing one embodiment of the IDX window of the present invention. The same components as those in FIG. 1 are denoted by the same reference numerals. In addition, 6003 is an IDX window, 280 is a detection section indicating a newly detected portion, and 282 is a new non-detection section. As described above, a detection section 280 and a non-detection section 282 that is not detected are newly added to FIG.

This function has the following advantages. First, when the process of detecting the person “C” is performed on the M-icons 1031 and 1033, there is no need to specify a color combination again. Further, since the stored color combination data is used as it is, detection processing can be performed under the same conditions as for the M-icon 1032.
Furthermore, since the detection result data is stored for each M-icon, it is also possible to find out only the M-icons 1031 and 1033 that do not have the detection result regarding the person “C”, and perform the detection processing only on the M-icon. Becomes As a result, the time for the detection process can be reduced.

Next, the function improvement of the IDX window will be described. The first is the function of combining detection results. This function is used, for example, when it is assumed that the person "A" and the person "B" are friends, and the detection results of the two persons are to be handled together in a framework of friends. As a collective method, a frame in which either the person “A” or the person “B” is present is set as a new detection section, or a frame in which the person “A” and the person “B” are together is set as a detection section. There are two methods. The operation is performed by attaching the target image 631 of the person “A” and the target image 632 of the person “B” and then clicking the join button 243. FIG. 11 is a view for explaining an embodiment of the combine. The same components as those in FIG. 10 are denoted by the same reference numerals. In addition, 6002 is an IDX window, 380 is a detection section, 383 is a non-detection section, 387 is a combine mark, and 661 is a detection section showing the detection result of a frame in which either the person “A” or the person “B” exists. The graph, 663, is the target image.

In FIG. 11, a detection section graph 661 displays a detection section 380 when a frame having one of the person “A” and the person “B” is set as a new detection section. In the target image 663, the target image 631 of the person “A” located at the top among the attached images is displayed, and a combine mark 387 (for example, “C”) indicating the combination is added to the upper left portion of the target image. Is done. Note that the detection result of the person “C” is displayed in the upper row.

Further, an uncombining (unbinding) function reverse to this is also provided. This is a function for restoring the combined detection result. The operation is performed by attaching the combined target image 663 and then clicking the unbinding button 244 with a mouse. As a result, the state returns to the state of FIG.

By the combining / unbinding function, it is possible to arrange and integrate the detection results divided by each person into abstract contents such as “friends”.

Next, the function of deleting the detection result will be described.
This is a function to delete the detection result that is no longer needed from the data structure of the M-icon. The operation is performed, for example, when deleting the detection result of the person “A”, the target image 6
After attaching 31, press the delete key on the keyboard 3090. As a result, the detection result regarding the person “A” is deleted from the data structure of the M-icon, and is deleted from the IDX window. Then, the detection results for the remaining persons "B" and "C" are packed at the top and displayed on the IDX window.

Next, the preview function will be described. The preview function is a function of playing back a movie only in the detection section obtained by the detection processing. The operation is performed by, for example, clicking the preview button 245 after attaching the target image 631 of the person “A” when previewing the detection section of the person “A”. As a result, a moving image playback window is displayed in the editing window 1000, and when playback is performed, only the frames in the detection sections 214 to 215 are previewed. Thus, the detection section can be confirmed by the moving image.

Next, the information function will be described. The information function is a function of storing information of a person who has performed a detection process on a detection result in the data structure of the M-icon. The operation is performed by attaching the target image 631 of the person "A" and then clicking the information button 246, for example, when the user wants to view or record information on the person "A".
As a result, the detection result for the person “A” is added to the person “A”.
(For example, name, date of birth, etc.) can be added.

Next, the correction operation of the detection result is performed, for example, by correcting the detection result of the person “A” by attaching the target image 631 of the person “A” and then clicking the correction button 248.

Finally, a function of adjusting the visible range of the detection result displayed in the detection section graph will be described. To adjust the visible range of this detection result, adjust the visible range slider 24.
Do this by dragging 7. View range adjustment slider 2
When the number displayed in 47 is 1, the visible range of the detection section graph is 1 second, that is, 30 frames. By moving the visible range adjustment slider 247, the display width of the detection section and the number of frames of the M-icon can be freely changed. By using this function, the display width can be reduced and displayed as a list even when displaying M-icons having any long number of frames, and the display width can be enlarged and displayed even in a detection section of only one frame. The unit of the number displayed on the visible range adjustment slider 247 can be set arbitrarily, and may be, for example, the number of frames.

In the above-described embodiment, an example of detecting an appearing person has been described as an example for explaining the image detecting method. However, it is obvious that an image targeting a person other than a person can be detected by specifying a characteristic color combination, and even if a detection method other than that described above is used, the present invention It goes without saying that the application is possible.

[0077]

As described above, according to the present invention, by storing the result of the detection processing in the data structure of the M-icon, the M-icon that has been subjected to the detection processing and the M-icon that has not been subjected to the detection processing are stored.
Icons can be specified on the IDX window and on the side of the M-icon.

As a second effect of the present invention, by storing the data of the color combination used in the detection processing in the data structure of the M-icon, only the M-icon of another undetected processing can be detected. It can be processed.

As a third effect of the present invention, by adding a combine / uncombine function, a delete function, a preview function, an information function, and a visible range adjustment function for the detection result, the function on the IDX window can be enhanced. It was planned.

[Brief description of the drawings]

FIG. 1 is a diagram showing one embodiment of an IDX window of the present invention.

FIG. 2 is a diagram showing an ID window for explaining conventional person detection.

FIG. 3 is a block diagram illustrating a configuration example of a moving image editing device.

FIG. 4 is a diagram showing a display example of an editing window.

FIG. 5 is a diagram showing one embodiment of an IDX window of the present invention.

FIG. 6 is a diagram showing a display example of a conventional IDX window.

FIG. 7 is a diagram showing one embodiment of an IDX window of the present invention.

FIG. 8 is a diagram showing an example of a conventional editing window.

FIG. 9 is a view showing that a result of person detection is displayed on the side of the M-icon.

FIG. 10 is a diagram showing an embodiment of an IDX window of the present invention.

FIG. 11 is a diagram showing an embodiment of an IDX window of the present invention.

FIG. 12 is a view showing that a result of person detection is displayed on the side of the M-icon.

FIG. 13 shows a conventional ID for confirming and correcting the result of human detection.
The figure which shows the example of a display of a window.

[Explanation of symbols]

12, 13, 14: IDX icon, 80: ID window, 23
7, 238, 239: Detection result display column, 240: New button,
241: Close button, 242: Detect button, 243: Merge button, 244: Unmerge button, 245: Preview button, 246: Information button, 247: Visibility adjustment slider, 248: Modify button, 250, 25
1,252: undetected section, 280: detected section, 282: non-detected section, 300: ID window, 301: OK button, 302:
Cancel button, 303: slider knob, 304:
Slide cursor, 305: enlarged display image, 306: enlarged display area, 307: object button, 308: color designation frame, 309: search button, 311: still image view slider, 312: object name entry field, 313: object frame adjustment Button, 314: Detection section addition button, 3
15: Detection section delete button, 321-330: Frame image,
380: detection section, 383: non-detection section, 809: film image, 600, 600 ': IDX window, 603: detection range display area, 604, 605, 606: area, 611: target display section, 614: detection section, 616: Non-detection section, 631, 632, 633: Target image, 634, 63
5,636: person name, 637, 638, 639: detection section graph, 6
40, 641: slider, 650: detection result display field, 800,
801, 802, 802 ', 803: detection arrows, 804, 805, 806, 8
07, 808: Object frame, 809, 809 ': Film image, 810: Mouse pointer, 811, 812: Detection section, 813, 814, 823, 824: Frame image, 815: Begin button, 816: End button, 817, 818,820,82
1,822: Pixel area, 1000: Editing window, 101
0: Editing area, 1020: Material area, 1030, 1031, 1
032, 1033, ‥‥‥, 103K, 103M, 103N, ‥‥‥: M-icon, 1040: Tree structure, 1050: Edit function button group, 1051: Character detection button, 1052: Tree structure creation button, 1070: , 1080, 1090: slider, 121
0: M-icon, 1220: M-icon representing a scene image, 1230: M-icon composed of one or more scene images, 1240: M-icon representing a moving image (program), 3010:
CPU, 3020: Memory, 3030: Cut change point detector,
3040: Image playback device, 3050: Video interface, 3060: Information storage device, 3070: Frame buffer, 3080: Monitor, 3090: Input device, 3100: Bus, 3110: Image signal output terminal, 6000, 6001, 6002,
6003: IDX window,

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5D110 AA04 AA13 BB20 CA42 CA43 CA46 CC01 CC02 CC04 FA02 FA09

Claims (12)

[Claims] 1. A GUI (Graphical User Interface) of a representative image representing a plurality of frame images constituting a moving image.
In a moving image editing apparatus that configures and edits a hierarchical structure on an editing screen by screen operation, from among a plurality of frame images included in the representative image,
Detecting a frame image in which a desired image is present; displaying information of the detected frame image in which the desired image is present as a detection result display screen; and displaying the specific information on the displayed detection result display screen. An image detection method characterized by displaying information of a frame image for which detection processing has not been performed on an image. 2. A GUI (Graphical User Interface) of a representative image representing a plurality of frame images constituting a moving image.
In a moving image editing apparatus configured and edited in a hierarchical structure on an editing screen by screen operation, at least one of the representative images is selected, and a plurality of frame images included in the selected representative image are selected. A reduced display in which a frame image in which a desired image exists is detected, and information of the detected frame image in which the desired image exists and a frame image in which the desired image does not exist, and a frame image selected as the desired image are reduced. An image detection method comprising: displaying a detection result display screen alongside an image; and displaying information on a frame image on which the detection processing has not been performed on the desired image on the displayed detection result display screen. 3. The image detecting method according to claim 1, wherein the feature of the desired image used for the detection is retained, and the detection process is still performed by the retained feature of the desired image. An image detection method, wherein image detection is performed on a frame image that has not been performed. 4. The image detection method according to claim 1, wherein the desired image is a person appearing in the moving image. 5. The image detection method according to claim 1, wherein the detection result is deleted on the detection result display screen. 6. The image detection method according to claim 1, wherein a plurality of detection results are combined on the detection result display screen. 7. The image detection method according to claim 6, wherein the plurality of combined detection results are returned to an original state. 8. The image detection method according to claim 1, wherein only a frame image in which the image is detected is previewed on the detection result display screen. 9. The image detection method according to claim 1, wherein information relating to a result of the detection processing is displayed on the detection result display screen. 10. The image detection method according to claim 1, wherein a display width of the frame image information to be displayed is changed on the detection result display screen. 11. The image detection method according to claim 1, wherein the representative image represents a cut composed of the plurality of images, and the cut is configured in a hierarchical structure on the editing screen by GUI operation. Select the cut on the edit screen, detect a frame image where a desired image is present from among the frame images constituting the selected cut, and place the detected information in the top of the hierarchical structure. An image detection method, wherein the image data is stored in a data structure of the lower cut. 12. The image detection method according to claim 1, wherein a result of the detection processing is displayed on a side face of the representative image.