JPH06309381A - Moving image processor - Google Patents

Abstract

PURPOSE:To provide a moving image processor in which each scene of a moving image can be easily indexed, and a retrieving condition can be easily generated in order to easily retrieve the desired scene in a short time. CONSTITUTION:One or plural central frames representing each scene are arbitrarily decided by a scene editor 24 for each scene classified in a shorter time than an original moving image C by a scene change detecting device 20, and a key word indicating the meaning content of the scene is applied to each scene. Then, a key frame which easily reminds a user of the key word is selected from the central frames of the scene by a key frame generating device 36. At the time of retrieval, the user selects the plural key frames, and applies a logical condition. A retrieving device 42 replaces the selected key frames with the key words, generates a key word retrieval expression described under the logical condition of the key word, and retrieves the desired scene from the moving image based on the generated key word retrieval expression.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture processing apparatus, and more particularly to a moving picture processing apparatus for retrieving a predetermined scene from a moving picture composed of a plurality of temporally continuous still pictures.

[0002]

2. Description of the Related Art With the spread of video equipment in recent years, a huge amount of video information such as still images is accumulated in museums and broadcasting stations. In order to obtain a desired image from the stored video information, it is necessary to efficiently add an index and perform a search. When moving image data composed of a plurality of temporally continuous still images is used as this image, the efficiency of indexing and searching is remarkably exhibited in order to obtain a desired image.

Conventionally, a moving image database is constructed by storing a plurality of still images forming a moving image by a computer as image data and accumulating position information of each image data in the moving image in a tree structure. A moving image processing apparatus has been proposed which searches for desired image data from a moving image database and displays the searched moving image on a display device such as a monitor (Japanese Patent Laid-Open Nos. 61-368968 and 61-29939). No.

In this moving image processing apparatus, the display is moved from the upper hierarchy to the lower hierarchy of the tree structure according to a user's instruction, and finally the image data desired by the user is read out and displayed on the display device. I am letting you.

[0005]

However, in general, a moving image is divided into scenes by cutting or switching camera angles, and this scene search is often performed.
In the moving image database based on the tree structure as described above, the still images corresponding to the lower layers included in the still images corresponding to the selected upper layer are sequentially displayed to finally search for a desired image. Because of overlapping, if the selection instruction is mistaken during the search, the image finally obtained may not reach the image of the scene desired by the user. Therefore, when constructing a moving image database, it is necessary to carefully set a tree structure with an appropriate number of branches and a depth of hierarchy. Further, when there are a plurality of desired scenes and it is desired to compare them, even if a specific one scene is reached, it is necessary to follow the tree structure for each of the other scenes in order from the upper hierarchy, and the operation becomes redundant. At the same time, it lacks a list for comparison.

In order to display this still image, it is possible to search a scene by inputting a keyword.
It is difficult to include all the information in the keyword,
The setting of keywords is complicated to include all the necessary information.

Further, even if the keywords have the same meaning, the keywords specified by the user at the time of searching may not match the keywords actually registered in the database, and as a result, a scene that satisfies the user may not be searched. In some cases, it may not be possible to think of what the user may input as a keyword. In such a case, the user has to make trial and error to find an appropriate keyword for search, which deteriorates the search efficiency.

Further, because of the versatility of the meaning and content of moving images, it is difficult to prescribe a standard term expression that allows all users to easily specify a keyword for facilitating search and the like. is there.

In consideration of the above facts, the present invention can easily and easily index each scene included in a moving image in order to easily and quickly retrieve an image of a desired scene. An object is to obtain a moving image processing device that can generate search conditions.

[0010]

In order to achieve the above object, the moving image processing apparatus according to the invention described in claim 1 is an image composed of a plurality of still images which are temporally continuous and have the same or similar meaning content. A storage unit that stores a moving image configured by including a plurality of scenes with one group as one scene and an image selected from at least one representative image representing each scene as a key image; and a plurality of the key images. Display means for displaying, selection information for selecting the key image displayed on the display means, and input means for inputting search conditions for the selected key image or search conditions for a plurality of selected key images Search means for searching a scene based on the selection information and the search condition.

According to a second aspect of the present invention, in a moving image processing apparatus, a moving image composed of a plurality of temporally continuous still images, and an image group composed of a plurality of still images having the same or similar semantic content as one scene. , A first selecting means for selecting at least one still image representative of the meaning and content of each scene from each classified scene, and a key image from the selected still images. Second selection means for selection and storage means for storing the key image are provided.

The first selecting means may select at least one still image related to the meaning and content of each scene from each classified scene.

According to a third aspect of the present invention, in the moving image processing apparatus according to the second aspect, the second selecting means is
It is characterized in that a still image having a semantic content in which the union of the semantic content of the selected still image matches the union of the semantic content of all the scenes of the moving image is selected as a key image.

[0014]

A moving image used in the moving image processing apparatus according to the first aspect of the present invention includes a plurality of still images that are temporally continuous and have the same or similar meaning content as one scene. It is configured to include. The moving image is stored in the storage unit, and the storage unit stores at least one representative image representing each scene of the moving image, that is, an image selected from the representative still images as a key image. The display means displays the stored key image. Therefore, a key image related to the meaning and content of each scene is displayed on the display means. The input means inputs selection information for selecting a key image displayed on the display means, a search condition for the selected key image or a search condition for a plurality of selected key images. With this, a key image can be selected by an intuitive image of the user, and a search condition corresponding to this key image or a search condition that can have relevance between a plurality of key images can be input. Therefore, if a search is performed by the search means based on the input selection information and search conditions, the scene desired by the user can be obtained as a search result.

According to a second aspect of the present invention, in a moving image processing apparatus, a moving image composed of a plurality of temporally continuous still images is grouped into an image group composed of a plurality of still images whose meanings are the same or similar by classification means. Are classified into a plurality of scenes. For each of the classified scenes, at least one still image representative of the semantic content of the scene or a still image related to the semantic content is selected from the scenes classified by the first selecting means. The second selecting unit selects a key image from the still images selected by the first selecting unit. This key image is stored by the storage means. In this way, by storing the key image in the storage means, information related to the meaning and content of each classified scene is stored. Therefore, the still image itself can be used as information (key image) corresponding to the keyword without being limited to the unified expression of the keyword or the like designated by the user at the time of search. Therefore, as the information used when recording the key image, a free keyword can be used for convenience.

Further, in the second selecting means, as described in claim 3, the set of meanings of the selected still images matches the set of meanings of all the scenes of the moving image. You may make it select the still image with such meaning content as a key image. By doing so, when the meaning / content of one scene exists over a plurality of scenes, the number of still images having the meaning / content may be one. Therefore, the number of selected key images can be reduced, and the minimum required still image can be selected.

[0017]

Embodiments of the present invention will now be described in detail with reference to the drawings. In the present embodiment, the present invention is applied to a moving image processing system 10 that performs an indexing process for assigning an index to each scene of a moving image and a search process for searching for each scene.

As shown in FIG. 1, the moving image processing system 10 includes a moving image storage device 12, a scene change detecting device 20, a scene editor device 24, a key frame generating device 36, and a searching device 42 each including a microcomputer. It consists of

Regarding each of these components, the present applicant has already filed an application (Japanese Patent Application No. 4-160034, Japanese Patent Application No. 4-160034).
-65620) and Open Technical Report (IBM Technical Di
Sclosure Bulletin. Vol.34 No.10A 1992.3).

The moving picture storage device 12 comprises a laser disk player (hereinafter referred to as an LD player) 14 and a video signal converter 16, and is connected to a TV monitor 18 for displaying a moving picture. The LD player 14 of the moving image storage device 12 can be loaded with a laser disc (LD) recording a plurality of temporally continuous still images (one frame image, hereinafter referred to as a frame).
A moving image is obtained by reproducing this LD. Also, L
A frame of a video signal (analog signal) such as NTSC is recorded in D, and this video signal is output as image data converted into a digital signal by the video signal converter 16.

The moving image storage device 12 receives the image data output from the video signal converter 16,
It is connected to the scene change detection device 20 having the first storage device 22 and also connected to the scene editor device 24. The scene change detection device 20 is also connected to the scene editor device 24.

The scene editor device 24 has second, third and fourth storage devices 26, 28, 30 and has a display device 32 and an input device 34. The scene editor device 24 is connected to a key frame generation device 36 and a search device 42.

The key frame generation device 36 has fifth and sixth storage devices 38 and 40. The search device 42 includes a search expression generation device 44 and a search processing device 46, and includes a display device 48 and an input device 50.

The operation of this embodiment will be described below. In the present embodiment, a predetermined frame included in the moving image C is displayed as a keyword image (hereinafter referred to as a key frame) as a key image including keyword information that is an element at the time of retrieval and image information of the frame, A query expression using keywords to search for a desired scene by giving a keyframe selection instruction while giving a logical condition (hereinafter referred to as a keyword search expression)
Q is generated, and a desired scene is searched from the moving image C based on the generated keyword search formula Q. First, the details of the key frame generation process for generating this key frame are shown in FIG.
Also, description will be made with reference to FIG.

The scene change detection device 20 is a physical change of frames such as cutting and switching of camera angles and meaning contents according to the temporal flow of the image data of the moving image C supplied from the LD player 14. The frame in which the change (hereinafter referred to as scene change) has occurred,
The frame number fno of the detected frame is stored in the first storage device 22 as the scene change information I1. As a result, all the frame numbers fno in which the scene change has occurred in the moving image C are stored. The frame group between the adjacent frame numbers fno is scene Si (i: 1, 2, ...
・) In this embodiment, five scenes S
A moving image C composed of 1, S2, S3, S4 and S5 will be described below as an example.

The scene change information I1 stored in the scene change detecting device 20 is output to the scene editor device 24. The scene editor device 24 selects, based on the input scene change information I1, a representative frame (e.g., a head frame) 80 that represents the semantic content of each scene Si divided by the scene-changed frames. The scene editor device 24 selects the representative frame 80 and the frame number fno of the selected representative frame 80.
Is output to the moving image storage device 12. As a result, the image data of the representative frame 80 (hereinafter referred to as representative image data) output from the video signal converter 16 of the moving image storage device 12 is input to the scene editor device 24. The scene editor device 24 stores and stores the input representative image data in the second storage device 26. The scene editor device 24 also displays the start frame number, the end frame number, the representative frame number (for example, the start frame number) of the scene Si corresponding to the representative frame 80 of the stored representative image data, and the image data of the representative frame. The stored storage address of the second storage device 26 is stored in the third storage device 28 as scene information I2 regarding the scene Si.
In Table 1 below, scene information I regarding a moving image C having five scenes S1 to S5 and having frame numbers 0 to 499 is shown.
The example of 2 was shown.

[0027]

[Table 1]

Here, the user operates at least one keyword W for each scene Si of the moving image C by operating the input device 34 and the display device 32 of the scene editor device 24.
Generate and allocate j (j: 1,2 ...). In the scene editor device 24, the keyword index information I in which the correspondence between the keyword Wj generated by the user and the frame group included between the start frame number and the end frame number corresponding to the keyword Wj is expressed as a table
3 is stored in the fourth storage device 30.

That is, when the user designates the representative frame 80 from the input device 34, the scene editor device 24 is stored in the scene information I2 stored in the third storage device 28 and the second storage device 26. The representative image data is read and the representative frame 80 is displayed on the display device 32. At this time, the user can give an instruction to reproduce the scene Si corresponding to the representative frame 80 when the representative frame 80 is displayed.
When 4 receives a scene reproduction instruction from the user, the LD player 12 is instructed to reproduce only the instructed scene Si on the TV monitor 18. This allows
Only the designated scene Si of the moving image C is displayed. The user can input the keyword Wj with the input device 34 while reproducing the representative frame 80 or the scene Si corresponding to the representative frame 80. Scene editor device 2
4, when the keyword Wj is input, the start frame number and the end frame number to which the keyword Wj is assigned are stored in the fourth storage device 30 as the keyword index information I3. The start frame number and the end frame number are input by a user's instruction during reproduction of the scene Si or an instruction from the input device 34. Table 1 above
1 of the keyword index information I3 when the moving image C of
Examples are shown in Table 2.

[0030]

[Table 2]

Therefore, keywords correspond to each scene Si as follows. First, the scene S1 has frames with frame numbers fno of 0 to 99, the keyword {car} corresponding to the frame group of frame numbers fno of 0 to 199, and the frame group of frame numbers fno to 0 to 99. Will include the keyword {mountain} corresponding to. As a result, the keyword of the scene S1 is {car, mountain}
Corresponding to. Similarly, scene S2 is {car, woman},
Scene S3 is {female, male}, scene S4 is {female,
Mountain} and scene S5 correspond to the keyword group of {mountain}. A plurality of keywords described in {} represents a union of these.

Next, in the key frame generation device 36, when the user turns on a key frame generation processing execution switch (not shown), the key frame generation routine of FIG. 3 is executed and the routine proceeds to step 102. In step 102, the scene information I2 and the fourth scene information I2 stored in the third storage device 28 are output by outputting a signal to the scene editor device 24.
The keyword index information I3 stored in the storage device 30 is input. In the next step 104, some of the representative frames 80 are selected as the key frames 82 according to the selection criteria described later. Keyword / keyframe correspondence information (hereinafter referred to as WF correspondence information) I4 representing a relationship between the selected keyframe 82 and keywords corresponding to this keyframe 82 is generated and stored in the fifth storage device 38. The image data of the representative frame 80 selected as the key frame 82 is stored as the image data of the key frame 82 (hereinafter referred to as key image data) in the sixth storage device 40 (step 108), This routine ends.

In this case, the sixth storage device 40 may store the address information of the key frame 82 instead of the image data.

The selection criterion is such that the set of keywords Wj assigned to the scene Si to which the representative frame 80 selected as the key frame 82 belongs covers the set of keywords Wj assigned to all the scenes Si. That is.

For example, in Table 1 above, three representative frames 8 whose frame numbers fno are f2, f3 and f5
0 is selected as the key frame 82. Frame number f
The keywords assigned to the scenes S2, S3, and S4 to which the representative frames 80 of nos are f2, f3, and f5 are, for example, scene S2 is {car, woman}, scene S3 is {woman, man}, and scene S5 is {. Mountain}. The union of the keywords of the scenes S2, S3, S5 matches the set of all keywords assigned to all the scenes Si (scenes S1 to S5) {car, woman, man, mountain}. Meet

Although a plurality of sets of key frames 82 satisfying the above selection criteria can be determined, it is preferable that the number of selected key frames 82 is small so that the user can easily see the outline at the time of a search process described later. So, for example,
When the set of keywords Wj corresponding to a certain key frame 82 is a subset of the set of keywords Wj corresponding to another key frame 82, the keyword corresponding to the former also corresponds to the latter, The former may not be held redundantly as the key frame 82.

Table 3 shows an example of the WF index information I4 in which the identifier of the key frame 82 and the keyword are associated with each other.

[0038]

[Table 3]

Next, details of the search processing using the key frame 82 described above will be described with reference to FIGS. 4 to 6.

In the search device 42, the keyword search formula Q is generated in the search formula generation device 44 by an instruction of the key frame 82 according to the operation of the input device 50 and the display device 48 by the user, and the generated keyword search formula Q. The search processing device 46 performs search processing based on the above. The representative frame 80 of the scene Si as the search result is displayed on the display device 48. Further, when the user gives an instruction to reproduce the scene Si, the scene S is displayed to the moving image storage device 12.
The reproduction instruction signal of i is transmitted.

That is, when the user turns on a search process execution switch (not shown) in the search formula generation device 44, the keyword search formula generation routine of FIG. 5 is executed, and the process proceeds to step 112 to send a signal to the key frame generation device 36. By outputting, the key frame generator 36 outputs W
The F correspondence information I4 and the key image data are read and stored in a memory (not shown). In the next step 114, the WF correspondence information I4 and the key image data are read from the memory (not shown), and the key frame 8 is read as shown in FIG.
The list of No. 2 is displayed on the display device 48.

The display screen 60 of the display device 48 is provided with a key frame display area 62 for displaying a plurality of key frames 82 (four in this embodiment), and the key read in the key frame display area 62 is displayed. A key frame 82 corresponding to the image data is displayed.

At the upper part of each key frame display area 62, an identifier display area 63 for displaying a key frame identifier for identifying the displayed key frame 82 by a serial number or the like is provided, and the key frame 82 is also displayed. A selection check button 64 for instructing selection is provided. The selection check button 64 indicates the selection of a key frame by turning on / off the mouse or the like instructed by the user from the input device 50, and displays the presence / absence of selection.

A search button 68 for instructing search execution and a scroll button 70 are provided on the right side of the display screen 60 (on the right side of the paper surface of FIG. 4). scroll·
The button 70 is an instruction area for displaying the front and rear key frames 82 when there are many selectable key frames 82 and all cannot be displayed on the same screen. The scroll button 70 is composed of a roll-up button 70A for sequentially displaying the key frames of the previous screen and a roll-down button 70B for sequentially displaying the key frames of the next screen.

The upper portion of the display screen 60 is a logical operator display area 66A, and a plurality of logical operator selection buttons 66 are provided in the logical operator display area 66A.
The logical operator selection button 66 is a button for designating a logical operator for connecting the instructed key frame 82. Further, the lower part of the display screen 60 is an optional condition input field 72. This optional condition input field 7
In 2, a query expression (hereinafter, referred to as a search expression) for a desired scene search process directly input by the user through the input device 50 is displayed.

At the next step 116, the user's selection instruction input, which is instructed to be selected, is read while visually observing the display screen 60, and the process proceeds to step 118. In step 118, it is determined whether or not the scroll button 70 is turned on. If it is turned on, in step 120, the roll-up button 70A and the roll-down button 70 are turned on.
The key image data of the screen corresponding to one instruction button of B is fetched, the process returns to step 114, the corresponding key frame is displayed, and the selection instruction is continued. On the other hand, if the determination is negative, the process proceeds to step 122.

At step 122, it is judged whether or not the search button 68 is instructed, and the processes of steps 116 to 118 are repeatedly executed until the search button 68 is instructed. When the search button 68 is instructed, the process proceeds to step 124. In step 124, the key frame 82
Is selected, or whether a character string is input in the arbitrary condition input field 72, it is determined whether or not the search condition is input. If the determination is negative, the process proceeds to step 116. Return.

A keyword search expression Q is generated as will be described later in response to direct input of a search condition into the arbitrary condition input field 72 or selection of a keyframe and an instruction to connect the selected keyframes with a logical operator. .

In this embodiment, the logical operator for connecting the key frames 82 is "AND",
"OR", "&", "|" and "¬" are used. The logical operators "AND" and "OR" are keywords Wj corresponding to two key frames 82 that are operands.
Is an operator for performing a set operation on a set of. The logical operator "AND" is an operator for finding the intersection,
"OR" is an operator for obtaining a union.

For example, an instruction to connect the key frame 82 of the identifier 1 and the key frame 82 of the identifier 2 shown in FIG. 4 with the logical operator "AND" can be expressed by the following retrieval formula (1).

1 AND 2 (1) In this case, as shown in Table 3 above, the search condition of scene Si having the keyword {female} common to the key frames 82 of identifiers 1 and 2 should be specified. Equivalent to.

Similarly, an instruction to connect the key frame 82 of identifier 1 and the key frame 82 of identifier 2 with the logical operator "OR" can be expressed by the following retrieval formula (2).

1 OR 2 (2) Therefore, it corresponds to specifying the search condition of scene Si having any one of the keywords {car}, {female}, and {male}.

On the other hand, the logical operators "&", "|", and "¬" are used to make clear that they are the logical operators themselves of the retrieval expression.

For example, an instruction to connect the key frame 82 of the identifier 1 and the key frame 82 of the identifier 2 with the logical operator "&" can be expressed by the following retrieval formula (3).

1 & 2 (3) In this case, {car or woman} and {woman or man}
Is equivalent to specifying the search condition of scene Si, that is, scene Si having any one of the keywords {car and woman}, {female}, {car and man}, and {female and man}.

Similarly, an instruction to connect the key frame 82 of the identifier 1 and the key frame 82 of the identifier 2 with the logical operator "|" can be expressed by the following retrieval formula (4), and the logical operator "¬" The instruction using “” can be expressed by the search formula (5).

1 | 2 (4) ¬ 3 (5) Therefore, in the search expression (4), the search condition that the keyword is {car or woman} or {female or man}, that is, {car }, {Female}, or {male}, which corresponds to specifying a search condition of scene Si having any of the keywords. The search formula (5) corresponds to specifying a search condition that the keyword does not include {mountain}, that is, specifying that scene Si that does not include {mountain} is searched.

The search formula described above can be directly input by the input device 50 using the identifier of the key frame 82 and the logical operators "AND", "OR", "&", "|" and "¬". The entered search formula is displayed in the optional condition input field 72. For example, as shown in the following search expression (6), search conditions can be specified by a combination of the search expressions (1) and (3).

(1 AND 2) & 3 (6) In this case, () indicates {female} of the keyword Wj common to the key frames, and "&3" indicates that the keyword is {female} and This is equivalent to designating the search condition of scene Si corresponding to {mountain}.

In the next step 126, it is determined whether or not a character string is input in the arbitrary condition input field 72. If the determination is affirmative, the user has directly input the search expression, so the input character string Is used as it is as a search expression and the process proceeds to the next step 130. On the other hand, in the case of negative judgment,
After proceeding to step 128, a character string in which the identifier of the instructed key frame 82 and the logical operator are connected is generated as a search expression, and then proceeds to step 130.

Step 130 is a process for generating the keyword search formula Q based on the above search formula. That is, referring to the WF correspondence information I4, the identifier of each key frame 82 in the search formula is replaced with a set of corresponding keywords Wj. At this time, the logical operator AND is interpreted as a common set operation and the logical operator OR is interpreted as a union operation to evaluate the search expression. For example, in the above search formula (1), the identifiers 1 and 2 of the key frame 82 are respectively set to the keyword Wj
Substituting {car, female}, {female, male} for the common set, the keyword search expression Q becomes Q = {female}.

For the above retrieval formula (6), the identifiers 1, 2 and 3 of the key frame 82 are replaced with keywords {car, female}, {female, male} and {mountain}, respectively, and A
When a set operation is applied to ND, a keyword search expression Q
Is Q = (1 AND 2) & 3 = ({car, woman} AND {female, man}) & {mountain} = {woman} & {mountain}.

In this way, the logical operators "AND", "O"
After the evaluation of "R" is completed, the generated keyword search expression Q is output to the search processing device 46, and this routine is completed.

Next, in the search processing device 46, when the keyword search formula Q is input from the search formula generating device 44, the process shown in FIG.
A search process execution routine based on the keyword search formula Q shown in is executed. First, at step 132, the keyword index information I3 stored in the fourth storage device 30 is input by outputting a signal to the scene editor device 24. In the next step 134, the keyword index information I3 is referred to obtain the start frame number and the end frame number corresponding to the keyword of the input keyword search expression Q, and the process proceeds to step 136. In step 136, the scene information I2 stored in the third storage device 28 is input by outputting a signal to the scene editor device 24. At the next step 138, it is determined whether or not the keyword search expression Q includes a logical operator. If the determination is affirmative, at step 140, the start frame number at which the keyword search expression Q becomes true based on the logical operator. And the end frame number are obtained and the process proceeds to step 142. On the other hand, in the case of negative determination, in step 141, the start frame number and the end frame number are directly obtained from the scene information I2, and the process proceeds to step 142.

In step 142, the representative frame number and the storage address of the representative frame 80 are obtained by referring to the scene information I2 of the scene Si included in the frame section determined by the obtained start frame number and end frame number. This representative frame number and representative frame 8
In step 144, the representative frame 80 is displayed on the display device 48 according to the storage address of 0. In the next step 146, it is determined whether or not the user has instructed the representative frame 80 and the scene reproduction by the input device 50. If the determination is affirmative, the scene Si instructed in step 148 is displayed on the TV monitor 18 Then, the LD player 12 is instructed to play back. On the other hand, in the case of a negative determination, this routine is finished as it is.

That is, in the case of the retrieval formula (1), referring to the keyword index information I3 in Table 2 above, the frame section [100, 399] corresponding to the keyword {female}
To get Next, search the scene information I2 in Table 3 above,
Scenes S2, S3 and S4 are obtained as scenes of the search result. The search processing device 46 reads the representative image data of the representative frame of the resulting scene Si and displays it on the display device 48.

Similarly, in the case of the above retrieval formula (6), the frame section [100, 3 corresponding to the keyword {female}
99] together with the frame section [300, 499] or [0, 99] corresponding to the {mountain} keyword. As a result, [300,399] is obtained as a common section between {female} and {mountain}. Therefore, the scene S4 is obtained as the search result.

As described above, in this embodiment, for each scene obtained by dividing the entire moving image in a short time, one or a plurality of representative frames representative of the scene are arbitrarily set, and the meaning of the scene is defined for each scene. Keywords that represent the content are attached. When viewing the representative frame from the representative frames of the scene, the user selects a key frame that is easy to associate as a keyword or an image that is an element for scene search. Thus, the user can easily specify the search condition by selecting a plurality of key frames and specifying the logical condition at the time of search. By substituting a keyword corresponding to the selected key frame, a keyword search formula described by the logical condition of the keyword can be generated, and a desired scene can be searched from the moving image based on the generated keyword search formula.

Therefore, as a keyword used as an index for retrieving a predetermined scene included in a moving image having a wide variety of meanings, a standard term based on a complicated term control using a concept (thesaurus) described by a hierarchical relationship is used. The keyword can be selected at the discretion of the index creator without the need to limit it to specific term expressions. Therefore, flexibility is increased in selecting a keyword corresponding to each scene, and an index can be easily generated.

Since the key frame for determining the search condition is a predetermined representative frame in the moving image and also includes image information for visually presenting the meaning and content of the scene,
By directly selecting the key frame, the user does not need to express the meaning content of the moving image using a specific keyword, and can intuitively specify the search condition. Even if the terms used by the indexer and the searcher are different, the key frame that determines the search condition is a predetermined representative frame in the moving image, so the search condition for the scene intended by the searcher is indexed. This can be set regardless of the term expression expected by a person, and the omission of search and the search error due to the difference in term expression can be reduced.

In this embodiment, the moving image processing system has been described as an example including a plurality of devices, but the present invention is not limited to this, and the indexing process and the search are performed by one moving image process. You may make it process.

Further, in the above embodiment, the keywords are assigned to the representative frame and the key frame corresponding to each scene, but the present invention does not require this keyword assignment. That is, at the time of indexing, if a keyframe and a scene are associated with each other only through the identification number of a keyword, it is easy to select a keyframe and give a logical condition at the time of searching even if a keyword is not assigned at the time of indexing. A desired scene can be obtained as a search result.

[0074]

As described above, according to the moving image processing apparatus of the present invention, the user can instruct the search condition by selecting a still image included in the moving image at the time of searching, so that the meaning of the moving image is described. There is an effect that the search condition can be intuitively specified without considering the term expression, and the omission of the search due to the difference in the term expression between the user when adding the index and the user when performing the search is reduced.

Further, according to the present invention, for moving images having various meanings, it is not necessary to limit to standard term expressions by term control as keywords for constructing a moving image database, and index creation is possible. Only by the person
Since the keyword can be flexibly selected, the index of the moving image database can be easily generated.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a moving image processing system of this embodiment.

FIG. 2 is an image diagram illustrating a process of generating a key frame.

FIG. 3 is a flowchart showing the flow of a key frame generation process of this embodiment.

FIG. 4 is an image diagram showing a display screen of a display device during a search process by the search device of the present embodiment.

FIG. 5 is a flowchart showing a flow of a keyword search expression generation routine.

FIG. 6 is a flowchart showing the flow of a search processing routine.

[Explanation of symbols]

 10 Moving Image Processing System 36 Key Frame Generator 42 Search Device 44 Search Formula Generator 46 Search Processor 80 Representative Frame (Representative Image) 82 Key Frame (Key Image)

Claims (3)

[Claims] 1. A moving image composed of a plurality of still images that are temporally continuous and have the same or similar meanings as one scene is stored, and each scene is represented. Storage means for storing an image selected from at least one representative image as a key image, display means for displaying a plurality of the key images, selection information for selecting the key image displayed on the display means, and selection A moving image process including: an input unit for inputting a search condition for the selected key image or a search condition for a plurality of selected key images; and a search unit for searching a scene based on the selection information and the search condition. apparatus. 2. A classifying unit for classifying a moving image composed of a large number of still images temporally continuous into a plurality of scenes, each of which is made up of an image group composed of a plurality of still images having the same or similar meanings. A first selecting unit that selects at least one still image related to the meaning and content of each scene from each classified scene, a second selecting unit that selects a key image from the selected still images, and a key image A moving image processing apparatus comprising: a storage unit that stores the image. 3. The second selection means selects a still image having a semantic content in which the union of the semantic content of the selected still images matches the union of the semantic content of all the scenes of the moving image. The moving image processing apparatus according to claim 2, wherein the moving image processing apparatus is selected as a key image.