JP2004297305A - System and program for configuring data base, system and program for retrieving image, and image recorder/reproducer - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recorder/reproducer in which scene information can be acquired automatically for dynamic image data of a personal image subject, data base of scene information can be made easily, a local variation in scene can be determined readily, and image retrieval (scene retrieval) can be carried out along the movement of a person. <P>SOLUTION: Under the control of a CPU 6, still image data are acquired from dynamic image data of a first recording medium 1 reproduced by a first drive 11 and stored in a buffer memory 3. The CPU 6 searches a person in the still image data stored in the buffer memory 3 by utilizing an image processing program or the like in the memory 4 and extracts a region including the person if any. Under the control of a CPU 6, partial image data thus extracted are recorded on a second recording medium 2 by using a second drive 12 while being associated with address information on the first recording medium 1 of the still image data from which the region including the person is extracted. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for easily searching for a specific scene from moving image data recorded on a recording medium, and relates to a database construction device, a database construction program, an image retrieval device, an image retrieval program, and an image recording / reproduction. Equipment related.
[0002]
[Prior art]
First, terms used in this specification will be defined. A moving image is composed of a sequence of still images, and an arbitrary sequence of still images constituting a part of the entire moving image is defined as a scene. The scene information is information that can be distinguished from the image data itself of the still images that constitute the scene, is given to each scene, and can specify the scene. Further, the image data of a moving image is referred to as moving image data, and the image data of a still image is referred to as still image data. When there is no need to distinguish between the moving image data and the still image data, they are simply referred to as image data.
[0003]
Today, recording of a moving image on a recording medium becomes easy, and there is a demand for the development of a technology capable of efficiently editing the recorded moving image. However, it is redundant to search for a scene including a specific object or an event included in the moving image while reproducing the moving image while the user himself looks at the screen, and the user can find the target scene in a short time. It is difficult.
[0004]
Therefore, conventionally, scene-related information (hereinafter, scene information) is given to each scene constituting a moving image, and this is stored in a database, and moving image data is searched using the scene information. ing.
[0005]
As scene information, for example, Patent Document 1 discloses a position in a moving image (eg, start / end frame number, time code), a meaning content of a scene (eg, keyword, attribute, representative frame), a relationship between scenes (eg, Information such as an identifier of a parent or child scene) and information of a scene change (for example, a position of a change point in a moving image, a type of change, and certainty) are listed.
[0006]
FIG. 8 shows a configuration diagram of a scene information input system described in the document. In this case, the scene information editor 112 extracts still image data representing the plurality of scenes from the representative frame file 122, and extracts the still images along with the time axis and in chronological order along the time axis. To be displayed. Then, when an instruction from the user is received through the input device 115, moving image data is extracted from the LD 116 and displayed on the TV monitor 118 for a period corresponding to a part of the instructed time axis.
[0007]
Further, the scene information editor 112 extracts scene information, which is information given to those scenes, from the scene information file 121 and simultaneously graphically displays the scene information on the screen of the display 114. When the user inputs an editing command through the input device 115, the editing command is executed and the scene information file 121 is edited.
[0008]
FIG. 9 shows a procedure for constructing scene information described in the document. In this method, a moving image composed of a temporally continuous frame sequence is divided into scenes, which are partial frame sequences, the division into scenes is confirmed and corrected, scene information is input, and the input information is processed. 5 shows a flow of processing up to registration in a moving image database.
[0009]
A) detecting a scene change point, generating scene information based on the detected scene change point, storing the scene information in a scene information file, b) displaying the scene information graphically on a display, and c) inputting. Step d) of editing the scene information file by executing an editing command instructed by the user using the means, and step e) of registering scene information of the scene information file in the database means.
[0010]
[Patent Document 1]
JP-A-5-334374 (December 17, 1993)
[0011]
[Problems to be solved by the invention]
In retrieving moving images, database creation of scene information is a significant core technology.However, the technology described in the above-mentioned literature is such that a user himself inputs scene information and constructs a database. After all, it is a difficult task for the user.
[0012]
Therefore, a system that can automatically input scene information is required, but it is very difficult to automatically input scene information as complete information. This is because the content of the scene information is very diverse, such as the person, the names of all the objects existing there, the location of each component, and the brightness of each component, all of which are to be input.
[0013]
Although the technique of the above-mentioned document is an invention that helps to construct a database of scene information, it is the same as the prior art of the document in that it is necessary to handle all information relating to the content of image data as scene information. And automation is difficult.
[0014]
Further, according to the technology of the above-mentioned literature, scenes are divided by capturing changes in the scenes. However, it is difficult to judge a change in an image with a small change, and when a change occurs locally (a part of the screen), including a search for a place where the change occurs, a very difficult change judgment function is provided. Required.
[0015]
For such a reason, the database (moving image database) constructed by the above-mentioned document system cannot be an effective database for any kind of image while the content of the scene information is precise. .
[0016]
For example, a plurality of scenes in which a scene change is local or small, such as when there is a request to cut out a still image from a moving image in a snap shot in a moving image mainly composed of a person, is used for scene information search. Depending on the selection of the keyword to be used, the search may be performed at the same time.
[0017]
SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and further obtains scene information as a person included in a still image or a combination of a person and additional information thereof, by further dividing still image data constituting moving image data. For moving image data mainly composed of a person image, scene information can be automatically obtained as image information including a person without bothering the user, thereby facilitating the creation of a database of the scene information. To provide a database construction device, a database construction program, an image retrieval device, an image retrieval program, and an image recording / reproducing device that can easily determine a typical scene change and the like and enable an image retrieval or the like in accordance with the movement of a person. It is an object.
[0018]
[Means for Solving the Problems]
In order to solve the above-mentioned problems, a database construction apparatus of the present invention includes: a still image data acquisition unit configured to acquire still image data from moving image data of a first recording medium reproduced by a first drive; A storage unit for storing the still image data obtained by the image data obtaining unit; and a person included in the still image data stored by the storage unit, and a predetermined person including the person when the person is included. A search and extraction unit for extracting an area as partial image data; and, when the partial image data is extracted by the search and extraction unit, the extracted partial image data is extracted from the still image data as the extraction source on the first recording medium. Recording means for recording the information on the second recording medium using the second drive in association with the address information.
[0019]
According to this, the still image data acquiring means acquires the still image data from the moving image data of the first recording medium reproduced by the first drive. The obtained still image data is accumulated in the accumulation unit, and the search and extraction unit searches for the person included in the still image data accumulated by the accumulation unit, and includes the person when the person is included. A predetermined area is extracted as partial image data. The partial image data extracted by the search and extraction unit is stored in the recording unit by associating the extracted partial image data with the address information on the first recording medium of the still image data from which the extraction is performed. The data is recorded on the second recording medium using the drive.
[0020]
The partial image data recorded on the second recording medium characterizes a scene with a moving image including the still image data already extracted therefrom, such as the act of the person, that is, the still image data. Since the information is included, the partial image data itself becomes scene information by displaying the partial image data. Also, when a plurality of pieces of person information can be obtained from the partial image data, the configuration of the plurality of persons is important scene information. In other words, these pieces of partial image data become scene information corresponding to information obtained for detecting a scene change for the user as it is.
[0021]
Since such scene information is partial image data obtained by cutting out a region including a person, the database can be constructed by automatically acquiring the scene information without using a user's hand using existing technology. .
[0022]
Accordingly, it is possible to provide a scene information database construction apparatus suitable for moving image data mainly composed of a person image and capable of automatically acquiring scene information without bothering the user.
[0023]
By mounting such a database construction apparatus in an image recording / reproducing apparatus, it is possible to easily determine a local scene change or the like, and to perform an image search or the like according to the movement of a person. It is possible to provide an image recording / reproducing apparatus that can easily edit the contents of image data and the like.
[0024]
Further, in the database construction device of the present invention, the search and extraction means may further include, when extracting the partial image data, additional information including numerical information indicating the number of persons included in the partial image data. The acquired recording means may be characterized in that the recording means records the additional information corresponding to the partial image data together with the partial image data in the second recording medium so that the additional information can be associated with the partial image data in a tree structure.
[0025]
According to this, additional information including number information indicating the number of persons included in the partial image data is obtained, and the additional information corresponding to the partial image data can be associated with the partial image data in a tree structure. Since the data is recorded on the second recording medium, even if the partial image data including the person is obtained as an image in which two persons overlap, the data recorded on the second recording medium has It can be managed separately for two people.
[0026]
Further, the additional information may further include color information indicating the color characteristics of each person and / or shape information indicating the shape characteristics of each person.
[0027]
If color information is used, it is possible to search for a specific scene using, for example, a costume worn by a specific person in the entire moving image as a key. In another example, if shape information is used, a change in the shape can be captured, and a specific scene can be searched using the behavior of a specific person as a key.
[0028]
Therefore, when the moving image data is searched based on the partial image data extracted from the region including the person and extracted from the area, the key items of the search are increased, and a database that can perform a more effective search is constructed. can do.
[0029]
Further, in the database construction apparatus of the present invention, it is preferable that the search and extraction unit extracts the partial image data so that all the persons included in the still image data are included in any of the partial image data.
[0030]
According to this, all the persons included in the still image data are acquired as partial image data independently or together with other persons, so that the content of the scene information becomes more accurate information.
[0031]
Further, in the database construction device of the present invention, the search and extraction means may be characterized in that the search and extraction are performed while appropriately changing a search area when searching for a person.
[0032]
According to this, since the search area when searching for a person is appropriately changed, the search area can be set even for moving image data in which the ratio of personal information to still image data changes arbitrarily at the time of search. The search area can always be appropriately determined, for example, by gradually reducing the size from the screen maximum range, and the search can be performed more efficiently. In addition, even when a search is performed as an area including a plurality of persons in the first search, it is possible to easily perform another search by changing the search area in order to further increase the information accuracy.
[0033]
In the database construction device of the present invention, the search and extraction means may be characterized in that the still image data is divided into a plurality of regions in advance, and the divided regions are searched as search regions.
[0034]
According to this, the still image data is divided into a plurality of regions in advance, and each of the divided regions is searched as a search region. In the case of image data, the number of persons existing in an area including a person to be found and cut out by one search can be appropriately reduced, and the number of individual person information can be reduced. In addition, if the search processing is performed independently in each of the divided areas, the extraction of the person from the still image data can be performed in a shorter time.
[0035]
Further, when the still image data is divided in advance in this manner, the method further includes a division number determining unit that determines the number of persons included in the still image data and determines the division number of the still image data based on the determination. More preferably, the extracting means divides the image data by the number of divisions determined by the number of division determining means.
[0036]
Since the still image data is divided according to the number of persons included in the still image data, the division number of the still image data is equal to the number of persons included in the still image data. Thus, the number of pieces of individual person information can be more accurate than a configuration in which the number of divisions is fixed.
[0037]
Further, in the database construction apparatus of the present invention, it is determined whether or not the reproduction of the moving image data of the first recording medium in the first drive is the first time. A configuration may also be provided that includes start instruction means for starting acquisition of still image data.
[0038]
According to this, when the reproduction of the moving image data of the first recording medium in the first drive is the first time, the acquisition of the still image data by the still image data acquiring means is started by the start instruction means. In addition, the user can acquire the scene information database by only playing the first recording medium on the first drive without giving a special instruction.
[0039]
In order to solve the above problems, an image recording / reproducing apparatus according to the present invention includes a first drive for reproducing information recorded on a first recording medium, and a first drive for recording / reproducing information on a second recording medium. An image recording / reproducing apparatus provided with a second drive and a database construction apparatus according to the first to ninth aspects.
[0040]
As already described as the database construction apparatus, the database construction apparatus of the present invention is a database construction of scene information suitable for moving image data mainly composed of human images and capable of automatically acquiring scene information without bothering the user. Device.
[0041]
Therefore, the image recording / reproducing apparatus of the present invention equipped with such a database construction apparatus can easily determine a local scene change or the like, and can perform an image search or the like in accordance with the movement of the person. An excellent image recording / reproducing apparatus that can easily edit the contents of the moving image data described above.
[0042]
The image search method of the present invention is an image search method of the image recording / reproducing apparatus of the present invention, wherein the partial image data recorded on the second recording medium is used as scene information, Is searched for an arbitrary scene in the moving image data recorded in the.
[0043]
As described above, the partial image data recorded on the second recording medium includes, for example, the act of the person, the still image data already extracted therefrom, that is, the moving image having the still image data as a component. Since information that characterizes a certain scene is included, displaying partial image data makes the partial image data itself scene information. Also, when a plurality of pieces of person information can be obtained from the partial image data, the configuration of the plurality of persons is important scene information. In other words, these pieces of partial image data become scene information corresponding to information obtained for detecting a scene change for the user as it is.
[0044]
Therefore, since the search is performed using the scene information at a fine level including the motion of the person included in the image, a more efficient image search can be performed, for example, in a search at the time of reproducing moving image data centered on a person.
[0045]
The image search device of the present invention is an image search device provided in the image recording / reproducing device, and reproduces a partial image data group recorded on the second recording medium using the second drive. A partial image data display means to be displayed on the display means, an input means for receiving a selection from the user for the displayed partial image data, and a first image data based on the partial image data selected by the input means. Search means for searching for still image data from which the partial image data is to be extracted from the moving image data recorded on the recording medium.
[0046]
According to this, the partial image data reproduction / display means reproduces the partial image data group recorded on the second recording medium using the second drive, and causes the display means to display the partial image data group. When the user selects one of the displayed partial image data using the input means, the search means searches the moving image data for still image data from which the partial image data selected by the input means is extracted. I do.
[0047]
Therefore, it is possible to search for necessary image data (scene search) by directly looking at the motion, facial expression, and the like of a person recorded on the second recording medium. A more detailed search can be performed than when the display is performed. For example, even when the ratio of the area where the person is recorded in the original still image is small in the whole, the image search using the person information as a key can be performed. In other words, since the display is narrowed down on the screen, it is possible to perform a search that is easier for the user to see and that has a high information accuracy.
[0048]
In the image search device of the present invention, when the reproduction of the moving image data recorded on the first recording medium is instructed, the partial image data reproducing / displaying means may perform the reproduction of the partial image data before reproducing the moving image data. May be displayed.
[0049]
According to this, when the partial image data group of the moving image data of the first recording medium is present on the second recording medium, a part of the partial image data is automatically reproduced before the reproduction of the moving image data. Therefore, for example, it is possible for the user to view the partial image data and selectively reproduce only a desired scene.
[0050]
Further, a database construction program and a recording medium of the present invention are a program for causing a computer to function as each means in the above-described database construction apparatus of the present invention, and a recording medium recording the program.
[0051]
Further, an image search program and a recording medium according to the present invention are a program for causing a computer to function as each unit in the above-described image search apparatus according to the present invention, and a recording medium storing the program.
[0052]
Thereby, if the above-described database construction program or image search program is executed by a computer, the specific database construction apparatus, the image search apparatus, and the image recording / reproducing apparatus, but not the specific image recording / reproducing apparatus. This also makes it possible to realize the database construction device, the image search device, and the image recording / reproducing device of the present invention.
[0053]
Further, the present invention can also be expressed as follows. That is, the image recording / reproducing apparatus of the present invention includes a first drive capable of recording / reproducing a first recording medium, a second drive capable of recording / reproducing a second recording medium, and a reproducing apparatus capable of reproducing / recording from the first drive. A buffer memory for temporarily storing the reproduced image data, a memory for storing a program for image processing, and displaying at least one of data reproduced by the first drive and data reproduced by the second drive. A playback device capable of at least one of recording and playback of the first recording medium and the second recording medium at the same time, wherein reproduced image data from the first recording medium is stored in at least one buffer memory. A search is performed for an area including a person from the image data stored and stored in the buffer memory, and at least one area including the person is searched. Extracting the area as at least one or more partial image data, linking the partial image data with the address information of the original image data from which the partial image data has been extracted on the first recording medium, and It is characterized by recording.
[0054]
Also, here, the partial image data obtained by searching for an area including a person from the first recording medium is obtained by three pieces of information of number of people, color information, and shape information obtained at the time of the search. It is preferable that a plurality of pieces of information regarding the image data of the first recording medium, including at least the following, are recorded on the second recording medium so as to be associated with each of the partial image data in a tree structure.
[0055]
An operation of storing the reproduced image data from the first recording medium in at least one buffer memory; searching for an area including a person from the image data stored in the buffer memory; When at least one of the partial image data has been searched, the area is extracted as at least one or more partial image data, and the partial image data is the address information of the original image data from which the partial image data is extracted on the first recording medium. And the image recording / reproducing apparatus is capable of independently executing the operation of recording on the second recording medium, and sequentially changes the search area in the image data to search for an area including a person. It can also be characterized by performing.
[0056]
An operation of storing the reproduced image data from the first recording medium in at least one buffer memory; searching for an area including a person from the image data stored in the buffer memory; When at least one of the partial image data has been searched, the area is extracted as at least one or more partial image data, and the partial image data is the address information of the original image data from which the partial image data is extracted on the first recording medium. 2. The image recording / reproducing apparatus according to claim 1, wherein the image recording / reproducing apparatus is capable of independently executing an operation of recording on the second recording medium, and further searches a plurality of image data reproduced from the first recording medium in advance. After dividing into regions, a search for a region including a person may be performed.
[0057]
Further, the image retrieval system in the image recording / reproducing apparatus according to the present invention is characterized in that partial image data including a person recorded on the second recording medium is used as scene information of the image data on the first recording medium. .
[0058]
Further, the image retrieval system in the image recording / reproducing apparatus of the present invention reproduces the image data of the first recording medium, thereby rewriting a part of the partial image data group automatically recorded on the second recording medium. The second recording medium is also reproduced and displayed before the second and subsequent reproductions of the first recording medium, and by selecting the partial image data, the partial image data is converted to the image data recorded on the first recording medium. It is characterized as a search key.
[0059]
BEST MODE FOR CARRYING OUT THE INVENTION
[Embodiment 1]
An embodiment of the present invention will be described below with reference to FIGS.
[0060]
FIG. 1 is a configuration diagram of an image recording / reproducing apparatus of the present embodiment including a database construction apparatus and an image search apparatus of the present invention.
[0061]
The image recording / reproducing apparatus includes at least first and second two drives 11 and 12, a buffer memory 3, a display device 5, a CPU 6, and a memory 4.
[0062]
The first drive 11 is a device capable of recording and reproducing the first recording medium 1, and the second drive 12 is a device capable of recording and reproducing the second recording medium 2.
[0063]
The buffer memory (accumulating means) 3 temporarily stores still image data obtained at a predetermined timing from moving image data (consisting of a sequence of still image data) reproduced from the first drive 11. is there. The sampled still image data is further subdivided as necessary and transferred to the second drive 12, as described later.
[0064]
The display device (display means) 5 displays the image data reproduced by the first drive 11 and the image data reproduced by the second drive 12. The display of the reproduced image data is performed independently, and the display device 5 displays one of the image data reproduced by the first and second drives 11 and 12.
[0065]
The memory 4 is composed of a hard disk or the like, and stores various application programs including a program for image processing.
[0066]
The CPU 6 includes a RAM (not shown) as a work area, and is a control center for controlling various operations of the first and second drives 11 and 12, the buffer memory 3, and the display device 5. In addition, the application program is read from the memory 4 and executed, and embodies the database construction device and the image search device of the present invention. That is, the CPU 6 and the memory 4 have functions of a still image data acquisition unit, a search and extraction unit, a recording unit, a start instruction unit, a partial image data display unit, a search unit, and the like.
[0067]
Further, the first drive 11 and the second drive 12 can simultaneously perform at least one of recording and reproduction on the recording medium 1 or the second recording medium 2 as the corresponding recording medium. I have.
[0068]
The first and second recording media are not particularly limited, but in the present embodiment, as an example, a DVR disc (large-capacity phase-change optical disc) is used as the first recording medium 1, DVD-RW is used as the recording medium 2.
[0069]
When reproducing the moving image data of the first recording medium 1 with the first drive 11, the image recording and reproducing apparatus samples the still image data, which is a component of the moving image data, at a predetermined timing. The second drive 12 extracts partial image data, which is a part of still image data, and records the extracted partial image data on the second recording medium 2.
[0070]
Further, in the present image recording / reproducing apparatus, it is necessary to search for a specific scene or image included in the moving image data for the purpose of editing the contents of the moving image data recorded on the first recording medium 1, for example. When this occurs, the above-described partial image data group recorded on the second recording medium 2 is used as information for image search.
[0071]
Information used to search for a scene in moving image data is referred to as scene information. In the present invention, partial image data which is a part of still image data constituting moving image data is set as scene information. More specifically, partial image data including a person and surrounding images in the still image data is set as scene information.
[0072]
In the above-described conventional configuration, the scene information is configured to include information such as the position in the moving image, the meaning of the scene, the relationship between the scenes, and the information on the scene change. Therefore, it was difficult to automatically acquire scene information as complete information. However, if it is scene information including a person and surrounding images in still image data constituting moving image data, it is possible to automatically acquire the scene information using existing technology.
[0073]
Further, by narrowing down the scene information to a person and its peripheral image, search information in moving image data mainly composed of a person can be made more specific, including the motion of the person.
[0074]
In other words, the person information is extracted together with the surrounding images to some extent in FIGS. 2 (a) and 2 (b). When extracting the person information, the person is extracted not only with the person information itself but also with a certain peripheral image as shown by a broken line.
[0075]
Further, in order to obtain more accurate information as the content of the scene information, it is desirable that there be a plurality of partial image data so that all the persons included in the still image data are included in any of the partial image data. That is, it is desirable that the entire partial image data acquired from the still image data covers all the persons appearing in the still image data.
[0076]
Therefore, as shown in FIG. 2A, when there is no overlap between persons in the image, acquisition of partial image data is relatively easy. For example, starting from a large search area, one person may enter the search area and the person search may be successful. On the other hand, as shown in FIG. 2B, when there is an overlap of persons in the image, the shape and the size of the search area are changed. For example, when the search area is large, the partial image data of the two overlapping persons is changed. Are extracted and acquired as information on both of the two persons who overlap. Further, when the search area is small, it can be extracted and acquired as partial image data including one person.
[0077]
In the plurality of images included in the still image data, the determination as to whether or not the image is a person can be performed by, for example, capturing person information as a combination of basic frames as described below.
[0078]
Here, the basic frame includes a face frame and a body part frame (such as a hand, a torso, and a foot). With respect to the face frame, a plurality of face frames are stored in the memory 4 as criterion information corresponding to the change in the form by applying the fact that the planar form of the face changes according to the observation direction. The number of face frame patterns to be prepared as determination reference information, that is, the number N thereof varies depending on system conditions. However, as the number of face frames, the angular resolution in the desired observation direction is determined by (△ φ, Δθ: Δ When φ is tilted and △ θ is rotated, N = (180 / △ φ) × (360 / △ θ). The information form of the face frame is obtained by replacing the constituent elements (eyes, nose, mouth, etc.) with a plurality of pieces of line information.
[0079]
In determining whether a person is a person, first, image processing is performed to form a contour image of still image data (an image obtained by processing an image into several pieces of line information, which is formed by an existing technology). Then, the arrangement of each component of the face frame is determined between any one of several contour groups included in the contour image and a contour model for forming N face frames serving as the determination criterion. This is done by comparison. Check if there are close contours in the N face frames. This is the first step of determining whether or not the person is a person.
[0080]
Next, upon receiving the determination result of the first stage, a final determination is made as to whether or not there is a body part frame near the face frame. The body part frame for determination includes an arm frame, a torso frame, and the like. For example, when a plurality of arm frames are used for determination, it is determined whether or not the outline model of the arm frame is within a predetermined distance from the face frame. There is a method of judging that the image information obtained in the above judgment is a person. It is possible to determine whether a person is a person by such a two-stage determination method.
[0081]
Such a method of determining a person is described in, for example, “Extraction of a part of a moving image using a Labeled Graph Matching with a human head and a change in facial expression”, IEICE Transactions Vol. J85-D-II No. 11 pp. 1656-1663, November 2002, and the like.
[0082]
Further, when extracting these partial image data as image data information including a person, it is preferable to acquire information including at least the number of people as additional information regarding the partial image data. The number of people information is information indicating the number of persons included in the partial image data. More preferably, the additional information includes one or both (more preferable) of color information characterizing the color of each person or shape information characterizing the shape of each person.
[0083]
By acquiring the additional information together with the partial image data in this manner, even if the image has a person overlap and a plurality of persons are included in one extracted partial image data, the obtained partial image data is Simultaneously obtaining additional data (additional information), which is information managed in a tree structure as shown in FIG. 3, obtains a plurality of pieces of personal information included in the partial image data. It can be used as information.
[0084]
In more detail, the color information and the shape information are obtained by capturing the partial image data, and in the captured image, for each person, a face frame group and a body frame group, such as a face frame and a body part frame, which will be described later. This is information used to further characterize these after determining whether they are patterns. For example, color information is information representing the overall color tone of a frame (such as a face frame or body part frame) that has been captured and adapted for determination, and shape information is the exact color of the frame adapted for determination. It is information that represents the shape.
[0085]
Further, the additional information may include motion information. The motion information means that a frame (face frame, body part frame, etc.) adapted for judgment is transferred to another frame in a different frame group from the same frame group in the next still image data capture operation. This is information indicating that it has become compatible. Furthermore, by processing the components (eyes, nose, mouth, etc.) in the face frame into another frame group, it is possible to perform a process corresponding to a change in facial expression.
[0086]
In an image in which a plurality of people are overlapped, a plurality of face frames and body part frames for determining whether or not a person exists in an incomplete shape and in a close shape are used as determination criteria. Since the color information and shape information of the face frame and the body part frame and the respective color information and shape information are unique to the person, information of each of a plurality of persons can be obtained from the color and shape information of each frame. Further, as information amount for characterizing the partial image data, information on the partial image can be managed as an information amount of <number of people> × <number of person determination frames> × <shape information + color information>. , Easy to characterize image information. Also, by managing the information in a tree structure, the search can be easily performed.
[0087]
As such color information and shape information, a temporary storage device (not shown) dedicated to a contour model uses a contour model determined to be close to a face frame or a body frame, which is used to determine whether or not the person is a person, as information. The part corresponding to the position of the contour model by referring to the captured image data to determine whether or not the person is the source of the contour model, and information such as the color and shape in the original image Can be acquired by taking in as a temporary storage device (not shown) dedicated to color information and a temporary storage device (not shown) dedicated to shape information.
[0088]
Also, since the number-of-persons information is information on how many pieces of personal information determined to be a person in the extracted partial image data, a temporary storage device (not shown) dedicated to the number-of-persons information is also stored in the same part. The number of contour models acquired in the image data can be acquired by counting and storing this.
[0089]
Further, when searching for an area including a person from the still image data, it is preferable to appropriately change the search area, which is a unit of search, within a range equal to or less than the maximum size of the still image data. Changing the search area means changing the shape (usually a rectangle) and size of the search area. By appropriately switching the search area, the search area can be formed in an appropriate shape suitable for the human image, and highly accurate information about the human can be obtained in a short time.
[0090]
As a method of setting the search area, for example, there is a method of using a face frame. If the size of the same person changes between the still image data obtained at the next timing and the still image data obtained at the previous timing, the change in the size of the face frame is detected and adjusted according to the change rate. To reduce the search area. In this case, if the rate of change of the size of the face frame becomes 70% of the original, the search area is also made 70% of the original.
[0091]
When an area including a plurality of persons is searched, if there are five face frames, for example, the area is reset so that the number of the face frames becomes three. The individual information included is increased, and the accuracy is improved.
[0092]
In order to reduce the search area based on the face frame, it is effective to use the place where the face frame exists as the base point of the diagonal element of the search area.
[0093]
Specifically, as shown in FIG. 4, the size of the face frame is set as x1 × y1 and x2 × y2, and the distances X and Y between the face frames, and further provided as margins for sufficiently securing a search portion thereof. By combining ranges (m1, m2, n1, and n2 are arbitrary numbers) predetermined on the basis of the size of the original face frame, the search area size is {(m1 + 1) .x1 + X + (m2 + 1) .x2} × {(n1 + 1) ) .Y1 + Y + (n2 + 1) .y2}.
[0094]
By acquiring the scene information based on the partial image data including a person as described above, there is no concept that the scene is determined to be a different scene in consideration of various information of the scene and the change is managed in a hierarchical structure. Thus, it is possible to manage information focusing on what those persons do.
[0095]
Next, an operation of extracting partial image data including a person will be described with reference to the flowchart of FIG.
[0096]
FIG. 5 shows a flow up to the acquisition of partial image data in a case where a region including a person in the still image data is extracted as partial image data while the search area for searching for a person is appropriately changed.
[0097]
When a start signal is detected by the first drive 11 during reproduction of a moving image on the first recording medium 1, timing information for acquiring one frame of moving image data reproduced by the first drive 11 is generated. Then, acquisition of still image data is started (S1).
[0098]
When the timing information for acquiring the moving image data is given, in synchronization with the timing information, the still image data, which is one frame of the moving image data, is stored in the buffer memory 3 with its address data (address on the first recording medium 1). (S2, S3).
[0099]
Next, it is determined whether or not immediately after the start (S4), and other than immediately after the start, it is confirmed whether or not a trigger of the image data transfer request is continuously detected (S5). If the trigger of the image data transfer request is not detected except immediately after the start, S1 to S5 are repeated to continue the image storage.
[0100]
On the other hand, if it is immediately after the start, or if the trigger of the image data transfer request is detected, the still image data fetched into the buffer memory 3 is sent to the person information detection system (S7). The processing so far is the processing of the image storage system for storing still image data.
[0101]
Hereinafter, the person information detection system will be described. First, image processing is performed on the transferred still image data to form the above-described outline image (S8). Next, while reviewing the search area, an area including a person is searched (S9).
[0102]
In S9, the search area is shifted in the horizontal direction by a predetermined pixel (determined according to the resolution of the contour image) with respect to the contour image obtained from the still image data, and the line segment read in the search is defined as a vector. Detects viewing direction and length information. Then, the search proceeds while acquiring the direction and length information. When the search scan in one horizontal direction is completed, the search scan is started again in the horizontal direction, shifted by a predetermined pixel in the vertical direction. This search is continued for the entire contour image data until information that can be determined to be a person is obtained, and information on the presence or absence of a line segment and, if there is a line segment, information on the direction and length thereof are included in the contour image information. Are stored in the temporary storage device.
[0103]
In this search, information on the line segments in the contour image can be obtained quantitatively, and the similarity between the contour model of each frame, such as a face frame and a body part frame, for determining whether the person is a person can be determined. Numerical comparisons are made, and if they match the contour model of each frame, it is determined that a person is included (S10).
[0104]
If it is determined that a person is included, a certain range including the matching part is cut out as partial image data including the person (S12). At this time, field information is also acquired so as to know where the extracted partial image data was in the still image data (S11). The field information is information indicating at which position in the still image the person image exists.
[0105]
The cut-out partial image data and the field information are associated with the address information indicating the position on the first recording medium 1 of the still image data containing the partial image data, and the second drive 12 is used. Recording is performed on the second recording medium 2 (S13).
[0106]
Next, at the stage where the search of the entire contour image is not completed, the search for the unsearched portion is continued, and the search for the person image is continued while the search area is reviewed (S14, S15). The processes of S12 to S15 are repeated until it is confirmed in S16 that there is no other personal information.
[0107]
More preferably, the search is performed in a smaller search area within the range of the partial image data obtained once. This makes it possible to obtain information with a high degree of certainty that characterizes the scene, such as facial expressions, in a short time.
[0108]
On the other hand, in S10, if it is not determined that a person is included even when the entire still image data is searched, and if it is confirmed in S16 that there is no other person information, the processing of the person information detection system ends. I do.
[0109]
When the processing of the person information detection system is completed, a trigger of an image data transfer request for sending the next still image data to the person information detection system is generated (S17), and the trigger data is converted to the still image data. The data is sent to the stored buffer memory 3. As a result, the next still image data is transferred from the image storage system to the person information detection system (S7).
[0110]
In this search, a person information detection system that searches for an area including a person and an image storage system that stores image data make different movements. Therefore, some time after the end of the reproduction of the moving image data, the process of searching for an area including a person ends. At the end of the search, an end operation is started based on the transfer of the end information of the moving image data reproduced by the first drive 11 (S6).
[0111]
The time interval (sampling timing) at which the still image data constituting the reproduced moving image data is sampled and stored in the buffer memory 3 in S1 described above is required for searching for and recording an area including a person. It is preferable to set in consideration of time.
[0112]
Further, the user may be allowed to freely set the sampling timing. When reproducing the moving image data of the first recording medium 1 and acquiring the partial image data as the scene information, the user grasps the contents based on the moving image projected on the display device 5. Therefore, if the timing for acquiring still image data is appropriately set according to the content, sampling according to the content of the moving image can be performed.
[0113]
Next, an image search of moving image data in the present image recording / reproducing apparatus will be described.
[0114]
The partial image data obtained as described above includes, for example, the act of the person, the still image data already extracted from the partial image data, that is, information characterizing a scene with a moving image having the still image data as a component. Since the partial image data itself is displayed on the display device 5, the partial image data itself becomes scene information.
[0115]
Also, when a plurality of pieces of person information can be obtained from the partial image data, the configuration of the plurality of persons is important scene information. In other words, these partial image data provide the user with information obtained for detecting a scene change and considerable scene information even as it is.
[0116]
Therefore, as described above, in the present image recording / reproducing apparatus, when it is desired to search for a specific scene (image) from a number of scenes of moving image data recorded on the first recording medium 1, the main moving image A partial image data group acquired from the data and already recorded on the second recording medium 2 is used as information for image retrieval.
[0117]
In the image search, the partial image data from the partial image data group is displayed on the display device 5 in an order along the time axis of the moving image data. At this time, the partial image data may be displayed as a thumbnail. Further, it is not necessary to display all the partial image data, and a part thereof can be displayed by thinning it out temporally.
[0118]
Then, in a state where the partial image data is displayed in this manner, a selection from the user for the displayed partial image data is received via an input device (not shown), and based on the selected partial image data, The moving image data recorded on the first recording medium 1 is searched for still image data from which the partial image data is to be extracted. (Including before and after), the reproduction of the moving image data is started.
[0119]
Also, in the present image recording / reproducing apparatus, when the moving image data of the first recording medium 1 is reproduced by the present apparatus for the first time, partial image data including a person in the person information and its surrounding information is automatically acquired. It has become. Further, in the present image recording / reproducing apparatus, when the second or subsequent reproduction of the moving image data recorded on the first recording medium 1 is instructed, one of the acquired partial image data is reproduced before reproducing the moving image data. The unit is displayed as a thumbnail on the screen of the display device 5 as shown in a frame 20 in FIG.
[0120]
The display of the partial image data can be performed by simply displaying the acquired partial image data (still images) by a predetermined number (eight in FIGS. 1 and 6) for a predetermined time in the time axis direction of the moving image data. May be displayed after selecting only some of them. Further, as the selective display method, there is a method in which a display partial image is selected at regular time intervals in the entire original moving image, and the selected partial image data is displayed.
[0121]
Such partial image data is already narrowed down to a person's motion, facial expression, etc., so that it becomes easier for the user to view than a normal thumbnail display of the entire still image. Such a search based on the already narrowed partial image data is concealed in the thumbnail display of the entire still image, so that the information accuracy is high and a more detailed image search (scene search) is possible. become.
[0122]
For example, when the ratio of the area where the person is recorded in the original still image is small in the whole, it is difficult to identify the person in the thumbnail display of the entire still image image. In this case, since the person image is displayed in a close-up manner, an image search can be performed using the person information as a key.
[0123]
Further, since partial information is basically managed based on personal information, if, for example, color information on a body part frame is used, partial image data is used, for example, by using a costume worn by a specific person in the entire moving image as a key. It is also possible to selectively display only the partial image data including the costume from the group, and a more effective scene search can be performed.
[0124]
In another example, if the shape information on the body part frame is used, the information on the shape of the foot is combined with the information on the movement, and the information on the shape of the arm is combined with the information on the movement, so that the specific It is possible to search for a scene using the behavior of a person as a key. Also, if field information is used, a scene in which a specific person is moving can be searched while distinguishing whether it is walking or running.
[0125]
[Embodiment 2]
Another embodiment of the present invention will be described below with reference to FIGS. For convenience of explanation, members having the same functions as the members used in Embodiment 1 are given the same reference numerals, and descriptions thereof are omitted.
[0126]
The image recording / reproducing apparatus according to the first embodiment (FIG. 1) is different from the image recording / reproducing apparatus according to the first embodiment (FIG. 1) in that the image recording / reproducing apparatus according to the present embodiment includes a buffer memory group 30 that can be independently written and read. This is a big difference. In the image recording / reproducing apparatus according to the first embodiment, the CPU 6 uses one buffer memory 3 to search the entire still image data while moving the search area in the still image data. The still image data is divided into a plurality of regions in advance by using a buffer memory group 30 including a plurality of buffer memories, a person is independently searched in each divided region, and partial image data is extracted. ing. That is, the search area is not moved.
[0127]
An operation of extracting partial image data including a person will be described with reference to the flowchart of FIG.
[0128]
FIG. 7 shows a flow from when a region including a person in the still image data is extracted as partial image data, a search region for searching for a person is divided in advance, search is performed, and partial image data is acquired. .
[0129]
When a start signal is detected by the first drive 11 during reproduction of a moving image on the first recording medium 1, timing information for acquiring one frame of moving image data reproduced by the first drive 11 is generated, The acquisition of still image data is started (S21).
[0130]
After this acquisition, the division number N of the still image data is determined (S22). For example, information on whether a large number of people are present on the screen is roughly searched at the level of the outline image data of the still image data, and the state of the number of people in the entire image is analyzed. decide. Although not shown in FIG. 7, the determination includes a function that the CPU 6 starts a simple image analysis program stored in the memory 4 and performs an automatic determination operation. Note that the division number N is the maximum number of buffer memories constituting the buffer memory group 30.
[0131]
The still image data divided into N pieces is divided into N pieces of N pieces of still image data by dividing the still image data into 1 to N of the buffer memory group 30 which can perform N independent write and read operations simultaneously with the division. The data group is collectively stored together with the address data on the first recording medium 1 before the image division (S23, S24).
[0132]
Next, it is determined whether it is immediately after the start (S25), and other than immediately after the start, it is continuously confirmed whether or not a trigger of the image data transfer request is detected (S26). If the trigger of the image data transfer request is not detected except immediately after the start, S21 to S26 are repeated to continue the image storage of the divided still image data (hereinafter, referred to as divided image data) in the buffer memory group 30. To go.
[0133]
On the other hand, if it is immediately after the start or if the trigger of the image data transfer request is detected, the divided image data taken into the buffer memory group 30 is sent to the person information detection system in a lump (S27). The processing up to this point is the processing of the image storage system that stores the divided image data.
[0134]
Here, since the still image data is divided into N pieces and stored in buffer memories 1 to N, there are N pieces of person information detection systems. In each of the N person information detection systems, image processing is first performed on the transferred divided image data to form the above-described outline image (S28). Next, an area including a person is searched (S29). Here, the search is performed without reviewing the search area.
[0135]
Then, in the same manner as described in the first embodiment, when it is determined that a person is included in S30, a certain range including the matching part is cut out as partial image data including the person (S32). Also at this time, field information is also acquired so that the location of the cut-out partial image data in the still image data can be known (S31).
[0136]
The cut-out partial image data and the field information are associated with the address information indicating the position on the first recording medium 1 of the still image data including the partial image data, and the second drive 12 is used. Is recorded on the second recording medium 2 (S36).
[0137]
However, in this case, since the N person information detection systems are independent and perform processing in parallel, the communication with the CPU 6 via the processing of S34 and S35 is performed while the communication with the CPU 6 is performed. Waiting for permission for recording request. Although not shown individually in FIG. 6, the N person information detection systems have a temporary storage function.
[0138]
If it is determined in S30 that the divided image data does not include a person and the search and extraction unit is used, and if the recording on the second recording medium 2 is completed in S36, the process proceeds to S35 of the next person information detection system.
[0139]
As described above, in this embodiment, a re-search is not performed by changing the search area. For example, even if personal information in which a plurality of persons overlap is provisionally obtained, individual personal information can be supplemented by a method of separately recording as additional information.
[0140]
When the recording of the divided image data on the second recording medium 2 is completed in the final person information detection system, or when it is determined in S30 that no person is included, the processing of the N person information detection systems ends. I do.
[0141]
When the processing of the N person information detection systems is completed, a trigger of an image data transfer request for sending the next divided image data to each person information detection system is generated (S37), and the trigger data is transmitted to the divided image data. The data is sent to N buffer memories storing the data. Thus, the next divided image data is collectively transferred from the image storage system to the person information detection system (S27).
[0142]
Also in this search, a person information detection system that searches for an area including a person and an image storage system that stores image data move differently. Therefore, some time after the end of the reproduction of the moving image data, the process of searching for an area including a person ends. At the end of the search, an end operation is started based on the transfer of the end information of the original moving image data reproduced by the first drive 11 (S38).
[0143]
Note that the time interval (sampling timing) at which the still image data constituting the reproduced moving image data is sampled, divided, and stored in the buffer memory group 30 in S21 described above is determined by searching for an area including a person. It is preferable that the setting be made in consideration of the time required for the recording and the recording, and the user may be able to freely set the sampling timing.
[0144]
As described above, the image recording / reproducing apparatus of the present embodiment does not move the search area and does not change the search area, as compared with the image recording / reproducing apparatus of the first embodiment. Therefore, in the case where the still image data is moving image data including a large number of persons because there is no operation such as repetitive search, the processing of dividing such still image data into N pieces in advance and detecting person information Can search more efficiently.
[0145]
In the present image recording / reproducing apparatus, the image retrieval using the partial image data group for the second recording medium 2 is the same as that of the image recording / reproducing apparatus of the first embodiment, and the description is omitted.
[0146]
The image recording / reproducing apparatuses according to the first and second embodiments described above can also be recorded as a program on a computer-readable recording medium. For example, a recording medium on which a database construction program and an image retrieval program for causing a computer to function as a still image data acquisition unit, a search and extraction unit, a recording unit, a start instruction unit, a partial image data display unit, a retrieval unit, and the like can be considered.
[0147]
An object of the present invention is to record the program codes (executable program, intermediate code program, and source program) of a database construction program and an image search program, which are software for realizing such means on a computer, so that the computer can read them. The present invention can also be achieved by causing a computer to record the program code on a recording medium, supplying the recording medium to an image recording / reproducing apparatus, and reading and executing the program code recorded on the recording medium. In this case, the program code itself read from the recording medium implements the above-described procedure, and the recording medium on which the program code is recorded constitutes the present invention.
[0148]
Here, the recording medium as the program medium is a recording medium configured to be separable from the main body, such as a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a flexible disk or a hard disk, or a CD-ROM / MO / It carries a fixed program including a disk system of an optical disk such as an MD / DVD, a card system such as an IC card (including a memory card) / optical card, or a semiconductor memory such as a mask ROM, EPROM, EEPROM, flash ROM, or the like. It may be a medium.
[0149]
It should be noted that the present invention is not limited to the above-described embodiments, and various changes can be made within the scope of the claims, and the technical means disclosed in different embodiments can be appropriately combined. Embodiments included in the invention are also included in the technical means of the present invention.
[0150]
【The invention's effect】
As described above, the database construction apparatus of the present invention includes: a still image data acquisition unit configured to acquire still image data from moving image data of a first recording medium reproduced by a first drive; Means for accumulating the still image data obtained by the means, and searching for a person included in the still image data accumulated by the accumulating means, and if a person is included, a predetermined area including the person is partially searched. Search and extraction means for extracting the extracted partial image data as image data; and address information on the first recording medium of the still image data from which the extracted partial image data is to be extracted. And a recording means for recording on a second recording medium using a second drive in association with the recording means.
[0151]
According to this, partial image data obtained by extracting an area including a person is recorded on the second recording medium and is made into a database. By displaying such partial image data itself, the partial image data itself becomes scene information. Also, when a plurality of pieces of person information can be obtained from the partial image data, the configuration of the plurality of persons is important scene information. In other words, these pieces of partial image data become scene information corresponding to information obtained for detecting a scene change for the user as it is.
[0152]
In addition, since such scene information is partial image data obtained by cutting out a region including a person, the database can be constructed by automatically obtaining the scene information by using existing technology without bothering the user. .
[0153]
Therefore, this provides an effect of providing a scene information database construction apparatus suitable for moving image data mainly composed of human images and capable of automatically acquiring scene information without bothering the user.
[0154]
Also, by mounting such a database construction apparatus in an image recording / reproducing apparatus, it is possible to easily determine a local scene change and the like, and to perform an image search or the like in accordance with the movement of a person. An advantage is provided in that an image recording / reproducing apparatus that can easily edit the contents of moving image data can be provided.
[0155]
Further, in the database construction device of the present invention, the search and extraction means may further include, when extracting the partial image data, additional information including numerical information indicating the number of persons included in the partial image data. The acquired recording means may be characterized in that the recording means records the additional information corresponding to the partial image data together with the partial image data in the second recording medium so that the additional information can be associated with the partial image data in a tree structure.
[0156]
According to this, additional information including number information indicating the number of persons included in the partial image data is obtained, and the additional information corresponding to the partial image data can be associated with the partial image data in a tree structure. Since the data is recorded on the second recording medium, even if the cutout of the partial image data including the person is obtained as an image in which two persons overlap, the data recorded on the second recording medium has It also has the effect that it can be managed separately for two people.
[0157]
Further, the additional information may further include color information indicating the color characteristics of each person and / or shape information indicating the shape characteristics of each person.
[0158]
If color information is used, it is possible to search for a specific scene using, for example, a costume worn by a specific person in the entire moving image as a key. In another example, if shape information is used, a change in the shape can be captured, and a specific scene can be searched using the behavior of a specific person as a key.
[0159]
Therefore, when the moving image data is searched based on the partial image data extracted from the region including the person and extracted from the area, the key items of the search are increased, and a database that can perform a more effective search is constructed. It also has the effect of being able to do so.
[0160]
Further, in the database construction apparatus of the present invention, it is preferable that the search and extraction unit extracts the partial image data so that all the persons included in the still image data are included in any of the partial image data.
[0161]
According to this, since all the persons included in the still image data are acquired independently or together with other persons as partial image data, the effect that the information of the scene information becomes more accurate information is added. Play.
[0162]
Further, in the database construction device of the present invention, the search and extraction means may be characterized in that the search and extraction are performed while appropriately changing a search area when searching for a person.
[0163]
According to this, since the search area when searching for a person is appropriately changed, the search area can be set even for moving image data in which the ratio of personal information to still image data changes arbitrarily at the time of search. The search area can always be appropriately determined, for example, by gradually reducing the size from the screen maximum range, and the search can be performed more efficiently. In addition, even when a search is performed as a region including a plurality of persons in the first search, it is possible to easily perform another search by changing the search region in order to further increase the information accuracy.
[0164]
In the database construction device of the present invention, the search and extraction means may be characterized in that the still image data is divided into a plurality of regions in advance, and the divided regions are searched as search regions.
[0165]
According to this, the still image data is divided into a plurality of regions in advance, and each of the divided regions is searched as a search region. In the case of image data, the number of persons existing in an area including a person to be found and cut out by one search can be appropriately reduced, and the number of individual person information can be reduced. In addition, if the search processing is performed independently in each of the divided areas, the effect of extracting the person from the still image data in a shorter time can be achieved.
[0166]
Further, when the still image data is divided in advance in this manner, the method further includes a division number determining unit that determines the number of persons included in the still image data and determines the division number of the still image data based on the determination. More preferably, the extracting means divides the image data by the number of divisions determined by the number of division determining means.
[0167]
Since the still image data is divided according to the number of persons included in the still image data, the division number of the still image data is equal to the number of persons included in the still image data. Thus, an effect that the number of individual person information can be more accurately achieved than the configuration in which the number of divisions is fixed is also provided.
[0168]
Further, in the database construction apparatus of the present invention, it is determined whether or not the reproduction of the moving image data of the first recording medium in the first drive is the first time. A configuration may also be provided that includes start instruction means for starting acquisition of still image data.
[0169]
According to this, when the reproduction of the moving image data of the first recording medium in the first drive is the first time, the acquisition of the still image data by the still image data acquiring means is started by the start instruction means. In addition, there is an effect that the user can acquire the database of the scene information without playing a special instruction only by playing the first recording medium with the first drive.
[0170]
In order to solve the above problems, an image recording / reproducing apparatus according to the present invention includes a first drive for reproducing information recorded on a first recording medium, and a first drive for recording / reproducing information on a second recording medium. An image recording / reproducing apparatus provided with a second drive and a database construction apparatus according to the first to ninth aspects.
[0171]
As already described as the database construction apparatus, the database construction apparatus of the present invention is a database construction of scene information suitable for moving image data mainly composed of human images and capable of automatically acquiring scene information without bothering the user. Device.
[0172]
Therefore, the image recording / reproducing apparatus of the present invention equipped with such a database construction apparatus can easily discriminate a local scene change and the like, and can perform an image search or the like in accordance with the movement of the person. This makes it possible to provide an excellent image recording / reproducing apparatus which can easily edit the contents of moving image data.
[0173]
The image search method of the present invention is an image search method of the image recording / reproducing apparatus of the present invention, wherein the partial image data recorded on the second recording medium is used as scene information, Is searched for an arbitrary scene in the moving image data recorded in the.
[0174]
As described above, the partial image data recorded on the second recording medium includes, for example, the act of the person, the still image data already extracted therefrom, that is, the moving image having the still image data as a component. Since information that characterizes a certain scene is included, displaying partial image data makes the partial image data itself scene information. Also, when a plurality of pieces of person information can be obtained from the partial image data, the configuration of the plurality of persons is important scene information. In other words, these pieces of partial image data become scene information corresponding to information obtained for detecting a scene change for the user as it is.
[0175]
Therefore, since the search is performed using scene information at a fine level including even the motion of a person included in the image, more efficient image search can be performed, such as in a search when reproducing moving image data centered on a person. Is also played.
[0176]
The image search device of the present invention is an image search device provided in the image recording / reproducing device, and reproduces a partial image data group recorded on the second recording medium using the second drive. A partial image data display means to be displayed on the display means, an input means for receiving a selection from the user for the displayed partial image data, and a first image data based on the partial image data selected by the input means. Search means for searching for still image data from which the partial image data is to be extracted from the moving image data recorded on the recording medium.
[0177]
According to this, the partial image data reproduction / display means reproduces the partial image data group recorded on the second recording medium using the second drive, and causes the display means to display the partial image data group. When the user selects one of the displayed partial image data using the input means, the search means searches the moving image data for still image data from which the partial image data selected by the input means is extracted. I do.
[0178]
Therefore, it is possible to search for necessary image data (scene search) by directly looking at the motion, facial expression, and the like of a person recorded on the second recording medium. A more detailed search can be performed than when the display is performed. For example, even when the ratio of the area where the person is recorded in the original still image is small in the whole, the image search using the person information as a key can be performed. In other words, since the display is narrowed down on the screen, it is possible to perform a search that is easier for the user to view and that has a high information accuracy.
[0179]
In the image search device of the present invention, when the reproduction of the moving image data recorded on the first recording medium is instructed, the partial image data reproducing / displaying means may perform the reproduction of the partial image data before reproducing the moving image data. May be displayed.
[0180]
According to this, when the partial image data group of the moving image data on the first recording medium is present on the second recording medium, a part of the partial image data is automatically reproduced before reproducing the moving image data. Therefore, for example, the user can view the partial image data and selectively reproduce only the scenes desired to be viewed.
[0181]
Further, a database construction program and a recording medium of the present invention are a program for causing a computer to function as each means in the above-described database construction apparatus of the present invention, and a recording medium recording the program.
[0182]
Further, an image search program and a recording medium according to the present invention are a program for causing a computer to function as each unit in the above-described image search apparatus according to the present invention, and a recording medium storing the program.
[0183]
Thus, if the above-described database construction program or image search program is executed by a computer, the present invention can be applied not only to a specific database construction apparatus, image search apparatus, and image recording / reproducing apparatus but also to an unspecified image recording / reproducing apparatus. The database construction device, the image search device, and the image recording / reproducing device of the present invention can also be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an image recording / reproducing apparatus according to an embodiment of the present invention.
FIGS. 2A and 2B are explanatory diagrams showing an example of acquiring scene information in the image recording / reproducing apparatus. FIG.
FIG. 3 is an explanatory diagram showing an information addition structure in a person information search in the image recording / reproducing apparatus.
FIG. 4 is an explanatory diagram showing a method of changing a search area when searching for a person in the image recording / reproducing apparatus.
FIG. 5 is a flowchart showing a procedure for automatically acquiring personal information in the image recording / reproducing apparatus.
FIG. 6 is a block diagram illustrating a configuration of an image recording / reproducing apparatus according to another embodiment of the present invention.
FIG. 7 is a flowchart showing a procedure for automatically acquiring personal information in the image recording / reproducing apparatus.
FIG. 8 is a diagram showing a scene information input system of the invention disclosed in the conventional publication.
FIG. 9 is a flowchart showing a procedure for inputting scene information according to the invention disclosed in the above-mentioned publication.
[Explanation of symbols]
1 First recording medium
2 Second recording medium
3 buffer memory (accumulation means)
4 memory (still image data acquisition means, search and extraction means, recording means, start instruction
Means, partial image data display means, search means)
5 Display means
6 CPU (still image data acquisition unit, search and extraction unit, recording unit, start instruction
Means, partial image data display means, search means)
11 First drive
12 Second drive
30 buffer memory group (accumulation means)

Claims (17)

A still image data acquisition unit for acquiring still image data from moving image data of a first recording medium reproduced by a first drive;
Accumulation means for accumulating the still image data acquired by the still image data acquisition means,
Search and extraction means for searching for a person included in the still image data accumulated by the accumulation means, and extracting a predetermined area including the person as partial image data when a person is included,
When the partial image data is extracted by the search and extraction means, the extracted partial image data is associated with the address information on the first recording medium of the still image data from which the extraction is performed, and the second drive is used. And a recording means for recording on a second recording medium. The search and extraction means, when extracting the partial image data, additionally obtains additional information including number information indicating the number of persons included in the partial image data,
2. The database construction according to claim 1, wherein the recording means records the additional information corresponding to the partial image data together with the partial image data on the second recording medium so that the additional information can be associated with the partial image data in a tree structure. apparatus. 3. The database construction apparatus according to claim 2, wherein the additional information includes color information indicating a color characteristic of each person. The database construction apparatus according to claim 2, wherein the additional information includes shape information representing a shape characteristic of each person. The method according to claim 1, wherein the search and extraction unit extracts the partial image data such that all persons included in the still image data are included in any of the partial image data. Database construction device. The database construction apparatus according to any one of claims 1 to 4, wherein the search and extraction unit performs the search while appropriately changing a search area when searching for a person. The database construction apparatus according to any one of claims 1 to 4, wherein the search and extraction unit divides the still image data into a plurality of regions in advance and searches the divided regions as search regions. The state of the number of persons included in the still image data is determined, and a division number determination unit that determines the division number of the still image data based on the determination,
8. The database construction apparatus according to claim 7, wherein said search and extraction means divides the image data by the number of divisions determined by said number of division determination means. It is determined whether or not the reproduction of the moving image data of the first recording medium in the first drive is the first time, and if it is the first time, the start instruction to start the acquisition of the still image data by the still image data acquiring means The database construction apparatus according to claim 1, further comprising means. A first drive for reproducing information recorded on the first recording medium;
An image recording / reproducing apparatus including a second drive for recording / reproducing information on / from a second recording medium;
An image recording / reproducing apparatus comprising the database construction apparatus according to claim 1. An image retrieval method for the image recording / reproducing apparatus according to claim 10, wherein
An image search characterized by performing a search for an arbitrary scene in moving image data recorded on the first recording medium using partial image data recorded on the second recording medium as scene information. Method. An image retrieval device provided in the image recording / reproducing device according to claim 10,
A partial image data display unit for reproducing the partial image data group recorded on the second recording medium using the second drive and displaying the partial image data group on a display unit;
Input means for receiving selection from the user for the displayed partial image data;
Based on the partial image data selected by the input means, a search is performed on the moving image data recorded on the first recording medium to retrieve still image data from which the partial image data is to be extracted. An image search device comprising: When the reproduction of the moving image data recorded on the first recording medium is instructed, the partial image data display means displays a part of the partial image data before reproducing the moving image data. The image search device according to claim 11, wherein: A database construction program for causing a computer to function as each means in the database construction apparatus according to any one of claims 1 to 9. A computer-readable recording medium on which the database construction program according to claim 14 is recorded. An image search program for causing a computer to function as each unit in the image search device according to claim 12. A computer-readable recording medium recording the image search program according to claim 16.

Images