JP4334898B2 - Database construction device, database construction program, image retrieval device, image retrieval program, and image recording / reproducing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for easily searching for a specific scene from moving image data recorded on a recording medium, a database construction device, a database construction program, an image retrieval device, an image retrieval program, and an image recording / reproduction. Relates to the device.
[0002]
[Prior art]
First, terms used in this specification will be defined. A moving image is composed of a sequence of still images, and an arbitrary sequence of still images constituting a part of the entire moving image is a scene. The scene information is information that is distinguished from the image data itself of the still images that constitute the scene, is given to each scene, and can identify the scene. If the image data of the moving image is moving image data, the image data of the still image is still image data, and it is not necessary to distinguish between the moving image data and the still image data, they are simply referred to as image data.
[0003]
Nowadays, it is possible to easily record a moving image on a recording medium, and it is desired to develop a technique that can efficiently edit the recorded moving image. However, it is redundant to find a scene that includes a specific object or event included in the moving image while the user is watching the screen while playing the moving image, and finds the target scene in a short time. It is difficult.
[0004]
Therefore, conventionally, information relating to a scene (hereinafter referred to as scene information) is given to each scene constituting a moving image, this is stored in a database, and moving image data is searched using the scene information. ing.
[0005]
As scene information, for example, Patent Document 1 discloses a position in a moving image (for example, start / end frame number, time code), meaning of a scene (for example, keyword, attribute, representative frame), and a relationship between scenes (for example, Information such as the identifier of the parent or child scene), information on the scene change (for example, the position of the change point in the moving image, the type of change, the probability).
[0006]
FIG. 8 shows a configuration diagram of a scene information input system described in the document. In this case, the scene information editor 112 takes out still image data representing a plurality of scenes from the representative frame file 122, and displays the still images along with the time axis in the time order along the time axis. To display. When an instruction from the user is received through the input device 115, moving image data is taken out from the LD 116 and displayed on the TV monitor 118 for a period corresponding to a part of the designated time axis.
[0007]
Also, the scene information editor 112 takes out scene information, which is information given to those scenes, from the scene information file 121 and displays it graphically on the screen of the display 114 at the same time. When the user inputs an editing command through the input device 115, the editing command is executed and the scene information file 121 is edited.
[0008]
FIG. 9 shows a procedure for constructing the scene information described in the document. This involves dividing a moving image composed of temporally continuous frame sequences into scenes that are partial frame sequences, confirming and correcting the division into scenes, inputting scene information, and inputting the input information. The flow of processing until registration in a moving image database is shown.
[0009]
Step a) for detecting a scene change point, step b) for generating scene information based on the detected scene change point and storing it in a scene information file, step c) for displaying the scene information graphically on a display, and input Step d) for editing the scene information file by executing an editing command instructed by the user using the means, and step e) for registering the scene information of the scene information file in the database means.
[0010]
[Patent Document 1]
JP-A-5-334374 (December 17, 1993)
[0011]
[Problems to be solved by the invention]
In searching for moving images, the creation of a database of scene information is a significant core technology, but the technology described in the above-mentioned literature is to construct a database by the user himself / herself entering scene information. It must be said that this is a difficult task for the user.
[0012]
Therefore, a system capable of automatically inputting scene information is required, but it is very difficult to automatically input complete scene information. This is because the contents of the scene information are very diverse, such as a person, the names of all the objects existing there, the location of each component, and the brightness of each component, all of which are input targets.
[0013]
The technique of the above document is an invention that helps to construct a database of scene information, but it is the same as the prior art of that document in that it needs to handle all information related to the contents of image data as scene information. And automation is difficult.
[0014]
Further, in the technique of the above-described literature, scenes are divided by capturing scene changes. However, it is difficult to judge changes in video with small changes, and even when changes occur locally (part of the screen), including the search for the location of the change, it is very difficult to judge changes. Required.
[0015]
For this reason, the database (moving image database) constructed by the above-described document system cannot be an effective database for any kind of image while the content of the scene information is precise. .
[0016]
For example, when there is a request to extract a still image from a moving image like a snapshot in a moving image mainly of a person, a plurality of scenes whose scene changes are local or small are used for scene information retrieval. Depending on the selection of keywords to be used, it will be searched at the same time.
[0017]
The present invention has been made in view of the above problems, and further subdivides still image data constituting moving image data, and acquires scene information as a person included in a still image or a combination of a person and its additional information. For moving image data mainly composed of person images, scene information can be automatically acquired as image information including a person without bothering the user, thereby facilitating the creation of a database of scene information and local PROBLEM TO BE SOLVED: To provide a database construction device, a database construction program, an image retrieval device, an image retrieval program, and an image recording / reproducing device that can easily discriminate a scene change etc. It is an object.
[0018]
[Means for Solving the Problems]
In order to solve the above problems, the database construction apparatus of the present invention includes still image data acquisition means for acquiring still image data from moving image data of a first recording medium reproduced by a first drive, and the still image data A storage unit for storing still image data acquired by the image data acquisition unit, and a person included in the still image data stored by the storage unit are searched. Search extraction means for extracting a region as partial image data; and when the partial image data is extracted by the search extraction means, the extracted partial image data is stored on the first recording medium of still image data from which the extraction is performed. And recording means for recording on a second recording medium using a second drive in association with the address information.
[0019]
According to this, the still image data acquisition means acquires still image data from the moving image data of the first recording medium reproduced by the first drive. The acquired still image data is accumulated in the accumulating unit, and the search and extracting unit searches for a person included in the still image data accumulated by the accumulating unit, and includes a person when the person is included. A predetermined area is extracted as partial image data. The partial image data extracted by the search extraction means is associated with the second partial image data associated with the address information on the first recording medium of the still image data that is the extraction source by the recording means. Recording is performed on the second recording medium using the drive.
[0020]
The partial image data recorded on the second recording medium is characterized by the action of the person, such as still image data that has already been extracted, that is, a scene with a moving image having the still image data as a component. Since information is included, the partial image data itself becomes scene information by displaying the partial image data. In addition, when a plurality of pieces of person information are obtained from the partial image data, the person configuration of the plurality of persons is important scene information. That is, even if these partial image data are used as they are, they become information obtained for detecting a scene change and corresponding scene information.
[0021]
Since such scene information is partial image data obtained by cutting out an area including a person, the database can be constructed by automatically acquiring the existing technology without bothering the user. .
[0022]
Accordingly, it is possible to provide a scene information database construction device that can automatically acquire scene information suitable for moving image data mainly composed of human images without bothering the user.
[0023]
By installing such a database construction device in an image recording / playback device, it is possible to easily identify local scene changes and the like, and it is possible to search for images according to the movement of a person, etc. It is possible to provide an image recording / reproducing apparatus capable of easily editing the contents of image data.
[0024]
Further, in the database construction device of the present invention, the search and extraction means further extracts additional information including number information representing the number of persons included in the partial image data when extracting the partial image data. Acquiring and recording, the recording means records the additional information corresponding to the partial image data on the second recording medium so as to be related in a tree structure connected to the partial image data.
[0025]
According to this, the additional information including the number information indicating the number of persons included in the partial image data is acquired, and the corresponding additional information together with the partial image data can be related in a tree structure connected to the partial image data. Since it is recorded on the second recording medium, even if the partial image data including the person is obtained as an image in which two persons overlap, on the data side recorded on the second recording medium, It can be managed separately in two persons.
[0026]
The additional information may further include color information representing the color characteristics of each person and / or shape information representing the shape characteristics of each person.
[0027]
If the color information is used, for example, it is possible to search for a specific scene using a costume worn by a specific person as a key in the entire moving image. In another example, if shape information is used, it is possible to search for a specific scene by capturing the change in shape and using the action of a specific person as a key.
[0028]
Therefore, when searching for moving image data based on the partial image data extracted from the region including people as the scene information, the number of search key items increases and a database that can perform more effective search is constructed. can do.
[0029]
In the database construction device of the present invention, it is preferable that the search and extraction means extract partial image data so that all persons included in still image data are included in any partial image data.
[0030]
According to this, since all the persons included in the still image data are acquired as partial image data independently or together with other persons, the information of the scene information is more accurate.
[0031]
Further, in the database construction device of the present invention, the search extraction means can be characterized in that it is performed while appropriately changing the search area when searching for a person.
[0032]
According to this, since the search area at the time of searching for a person is changed as appropriate, the search area can be set even for moving image data or the like in which the ratio of the person information to the still image data changes arbitrarily during the search. The search area can always be appropriately determined by reducing the maximum range of the screen step by step, and the search can be performed more efficiently. In addition, when searching for an area including a plurality of persons in the initial search, it is possible to easily perform a search again by changing the search area in order to further improve information accuracy.
[0033]
In the database construction device of the present invention, the search and extraction means may divide still image data into a plurality of areas in advance, and search the divided areas as search areas.
[0034]
According to this, since still image data is divided into a plurality of regions in advance and each divided region is searched as a search region, a moving image that is expected to have a large number of people recorded in the cut out still image data in advance. In the case of image data, it is possible to appropriately reduce the number of persons existing in an area including a person to be found and extracted by a single search, and to reduce the number of individual person information. Further, if the search process is performed independently in each divided area, it is possible to extract a person from still image data in a shorter time.
[0035]
In addition, when dividing still image data in advance, the above search is provided with a division number determining means for determining the state of the number of persons included in the still image data and determining the number of divisions of the still image data based on the determination. More preferably, the extracting means divides the image data by the division number determined by the division number determining means.
[0036]
Thereby, since the division of the still image data is performed according to the number of persons included in the still image data, the number of divisions of the still image data was in the state of the number of persons included in the still image data. Thus, the number of pieces of individual person information can be made more accurate than the configuration in which the number of divisions is fixed.
[0037]
In the database construction device of the present invention, it is determined whether or not the moving image data of the first recording medium is reproduced for the first time in the first drive. It can also be configured to include a start instruction means for starting acquisition of still image data.
[0038]
According to this, when the reproduction of the moving image data of the first recording medium in the first drive is the first time, the acquisition of the still image data by the still image data acquisition unit is started by the start instruction unit. The user can acquire a database of scene information by only reproducing the first recording medium with the first drive without giving a special instruction.
[0039]
In order to solve the above problems, an image recording / reproducing apparatus of the present invention includes a first drive for reproducing information recorded on a first recording medium, and a first drive for recording / reproducing information on a second recording medium. An image recording / reproducing apparatus including two drives includes the database construction apparatus according to any one of the first to ninth aspects.
[0040]
As already described as a database construction apparatus, the database construction apparatus of the present invention is a database construction of scene information suitable for moving image data mainly of human images and capable of automatically obtaining scene information without bothering the user. Device.
[0041]
Therefore, the image recording / reproducing apparatus of the present invention equipped with such a database construction apparatus is capable of easily discriminating local scene changes and the like, and can search for images according to the movement of the person. Thus, an excellent image recording / reproducing apparatus capable of easily editing the contents of the moving image data is provided.
[0042]
The image search method of the present invention is the image search method of the image recording / reproducing apparatus of the present invention, wherein the partial image data recorded on the second recording medium is used as scene information, and the first recording medium is used. An arbitrary scene is searched in the moving image data recorded in the video.
[0043]
As described above, the partial image data recorded on the second recording medium includes, for example, the action of the person, still image data that has already been extracted, that is, a moving image having the still image data as a component. Therefore, by displaying partial image data, the partial image data itself becomes scene information. In addition, when a plurality of pieces of person information are obtained from the partial image data, the person configuration of the plurality of persons is important scene information. That is, even if these partial image data are used as they are, they become information obtained for detecting a scene change and corresponding scene information.
[0044]
Therefore, since the search is performed using scene information at a fine level including the motion of the person included in the image, a more efficient image search can be performed, for example, in a search when reproducing moving image data centered on a person.
[0045]
The image search apparatus of the present invention is an image search apparatus provided in the image recording / reproducing apparatus, and reproduces a partial image data group recorded on the second recording medium using the second drive. Partial image data display means to be displayed on the display means;
An input unit that accepts selection of the displayed partial image data from the user, and a moving image data recorded on the first recording medium based on the partial image data selected by the input unit And retrieval means for retrieving still image data from which the partial image data is extracted.
[0046]
According to this, the partial image data reproduction display means reproduces the partial image data group recorded on the second recording medium using the image second drive, and causes the display means to display it. When the user selects one of the displayed partial image data using the input means, the search means searches the moving image data for still image data from which the partial image data selected by the input means is extracted. To do.
[0047]
Accordingly, it is possible to search for necessary image data (scene search) by directly looking at the movement, facial expression, etc. of the person recorded on the second recording medium. Finer search is possible than when displayed. For example, even when the ratio of the area where the person is recorded in the original still image is small in the whole, the image search using the person information as a key can be performed. In other words, since the screen is narrowed and displayed, a search that is easier for the user to see and has high information accuracy becomes possible.
[0048]
In the image search device according to the present invention, the partial image data reproduction / display means may further execute partial image data before reproduction of the moving image data when the reproduction of the moving image data recorded on the first recording medium is instructed. It is also possible to display a part of the screen.
[0049]
According to this, when the partial image data group of the moving image data on the first recording medium is on the second recording medium, a part of the partial image data is automatically reproduced before the moving image data is reproduced. Therefore, for example, the user can use the partial image data by selectively reproducing only the desired scene.
[0050]
The database construction program and recording medium of the present invention are a program for causing a computer to function as each means in the above-described database construction apparatus of the present invention, and a recording medium recording the program.
[0051]
The image search program and recording medium of the present invention are a program for causing a computer to function as each means in the above-described image search apparatus of the present invention and a recording medium on which the program is recorded.
[0052]
Thus, if the above-described database construction program or image retrieval program is executed by a computer, it is not intended for a specific specific database construction device, image retrieval device, or image recording / reproduction device, but for an unspecified image recording / reproduction device. In addition, the database construction device, image search device, and image recording / playback device of the present invention can be realized.
[0053]
The present invention can also be expressed as follows. That is, the image recording / reproducing apparatus of the present invention reproduces data from the first drive capable of recording / reproducing the first recording medium, the second drive capable of recording / reproducing the second recording medium, and the first drive. Displays at least one of a buffer memory for temporarily storing the image data, a memory for storing a program for image processing, data reproduced by the first drive, and data reproduced by the second drive In a playback device capable of simultaneously recording and playing back at least one of the first recording medium and the second recording medium, the playback image data from the first recording medium is stored in at least one buffer memory. When an area including a person is searched from the stored image data stored in the buffer memory, and at least one area including a person can be searched. In addition, the region is extracted as at least one partial image data, the partial image data is combined with address information in the first recording medium of the original image data from which the partial image data is extracted, and the second recording medium It is characterized by recording.
[0054]
Further, here, for the partial image data obtained by searching an area including a person from the first recording medium, three pieces of information obtained at the time of searching, number information, color information, and shape information. It is preferable that a plurality of pieces of information relating to image data of the first recording medium including at least are recorded on the second recording medium so as to be related in a tree structure connected to the individual partial image data.
[0055]
Also, an operation for accumulating reproduced image data from the first recording medium in at least one buffer memory, an area including a person is searched from the image data stored in the buffer memory, and an area including a person is included. When at least one or more of them can be searched, the area is extracted as at least one partial image data, and the partial image data is address information in the first recording medium of the original image data from which the partial image data is extracted. The image recording / reproducing apparatus is capable of independently executing the operation of recording on the second recording medium, and sequentially searches the area including the person by changing the search area in the image data. It can also be characterized by performing.
[0056]
Also, an operation for accumulating reproduced image data from the first recording medium in at least one buffer memory, an area including a person is searched from the image data stored in the buffer memory, and an area including a person is included. When at least one or more of them can be searched, the area is extracted as at least one partial image data, and the partial image data is address information in the first recording medium of the original image data from which the partial image data is extracted. The image recording / reproducing apparatus according to claim 1, wherein the image recording / reproducing apparatus is capable of independently executing an operation of recording on the second recording medium, and a plurality of searches for image data reproduced from the first recording medium in advance. It is also possible to search for an area including a person after dividing the area.
[0057]
The image search system in the image recording / playback apparatus of the present invention is characterized in that partial image data including a person recorded on the second recording medium is used as scene information of the image data on the first recording medium. .
[0058]
Further, the image search system in the image recording / reproducing apparatus of the present invention reproduces the image data of the first recording medium, thereby reconstructing a part of the partial image data group automatically recorded on the second recording medium. The second recording medium is also reproduced and displayed before the second reproduction of the first recording medium, and the partial image data is selected, whereby the partial image data is recorded on the first recording medium. It is a search key.
[0059]
DETAILED DESCRIPTION OF THE INVENTION
[Embodiment 1]
One embodiment of the present invention will be described with reference to FIGS. 1 to 5 as follows.
[0060]
FIG. 1 is a configuration diagram of an image recording / playback apparatus according to the present embodiment, which includes a database construction apparatus and an image search apparatus according to the present invention.
[0061]
The image recording / reproducing apparatus includes at least first and second drives 11 and 12, a buffer memory 3, a display device 5, a CPU 6, and a memory 4.
[0062]
The first drive 11 is a device capable of recording / reproducing the first recording medium 1, and the second drive 12 is a device capable of recording / reproducing the second recording medium 2.
[0063]
The buffer memory (storage means) 3 temporarily stores still image data acquired at a predetermined timing from moving image data (consisting of a sequence of still image data) reproduced from the first drive 11. is there. The sampled still image data is further subdivided as described below and transferred to the second drive 12 as necessary.
[0064]
The display device (display unit) 5 displays the image data reproduced by the first drive 11 and the image data reproduced by the second drive 12. The reproduced image data is displayed independently, and the display device 5 displays either one of the image data reproduced by the first and second drives 11 and 12.
[0065]
The memory 4 is composed of a hard disk or the like, and stores various application programs including a program for image processing.
[0066]
The CPU 6 includes a RAM (not shown) as a work area, and is a control center that controls various operations of the first and second drives 11 and 12, the buffer memory 3, and the display device 5. Further, the application program is read from the memory 4 and executed, and the database construction apparatus and the image search apparatus according to the present invention are also embodied. That is, the CPU 6 and the memory 4 have functions such as still image data acquisition means, search extraction means, recording means, start instruction means, partial image data display means, and search means.
[0067]
Further, the first drive 11 and the second drive 12 described above can simultaneously perform at least one of recording and reproduction with respect to the recording medium 1 or the second recording medium 2 which is a corresponding recording medium. Yes.
[0068]
The first and second recording media are not particularly limited, but in the present embodiment, as an example, the first recording medium 1 is a DVR disc (large-capacity phase change optical disc), the second The recording medium 2 uses a DVD-RW.
[0069]
In the image recording / reproducing apparatus, when the first drive 11 reproduces the moving image data of the first recording medium 1, the still image data which is a component of the moving image data is sampled at a predetermined timing. The partial image data, which is a part of still image data, is extracted and recorded on the second recording medium 2 by the second drive 12.
[0070]
Further, in this image recording / reproducing apparatus, for example, for the purpose of editing the contents of the moving image data recorded on the first recording medium 1, it is necessary to search for a specific scene or image included in the moving image data. When it occurs, the partial image data group recorded on the second recording medium 2 is used as information for image retrieval.
[0071]
Information used to search a scene of moving image data is referred to as scene information. In the present invention, partial image data that is a part of still image data constituting moving image data is used as scene information. More specifically, the partial image data composed of a person in the still image data and surrounding images is used as scene information.
[0072]
In the above-described conventional configuration, the scene information is configured from information such as the position in the moving image, the meaning of the scene, the relationship between the scenes, and information on the scene change. For this reason, it is difficult to automatically obtain scene information as complete information. However, if the scene information is composed of a person in the still image data constituting the moving image data and the surrounding image, it can be automatically acquired by the existing technology.
[0073]
Further, by narrowing down the scene information to the person and the surrounding image, the search information in the moving image data mainly of the person can be made more specific including the action of the person.
[0074]
That is, in FIG. 2 (a) and FIG. 2 (b), the person information is extracted together with the surrounding image to some extent. When extracting the person information, not only the person information itself is extracted, but the person is extracted together with surrounding images around to some extent as indicated by a broken line.
[0075]
Further, in order to obtain more accurate information as the contents of the scene information, it is desirable that the partial image data is plural so that all the persons included in the still image data are included in any partial image data. That is, it is desirable that the entire partial image data acquired from the still image data covers all persons appearing in the still image data.
[0076]
Therefore, as shown in FIG. 2A, when there are no overlapping persons in the image, it is relatively easy to acquire partial image data. For example, starting from a large search area, one person enters the search area, and the person search increases. On the other hand, as shown in FIG. 2B, when there are overlapping persons in the image, the shape and size of the search area are changed. For example, when the search area is large, partial image data of two overlapping persons Is extracted and acquired as information on both of the two people who overlap. Further, when the search area is small, it can be extracted and acquired as partial image data including one person.
[0077]
The determination as to whether or not the image is a person among the plurality of images included in the still image data can be performed by capturing the person information as a combination of basic frames as follows, for example.
[0078]
Here, the basic frame is composed of a face frame and a body part frame (hand, torso, legs, etc.). As for the face frame, by applying the fact that the shape of the face changes depending on the observation direction, a plurality of patterns of face frames are stored in the memory 4 as determination reference information corresponding to the change of the shape. The number of face frame patterns to be prepared as judgment reference information, that is, the number N varies depending on the system conditions, but the number of face frames can be determined by setting the angle resolution in the desired observation direction (Δφ, Δθ: Δ When φ is tilted and Δθ is rotation, N = (180 / Δφ) × (360 / Δθ). The information format of the face frame is obtained by replacing the constituent elements (eyes, nose, mouth, etc.) with a plurality of line information.
[0079]
In determining whether or not a person is present, first, image processing is performed to form a contour image of still image data (an image obtained by processing an image into a plurality of pieces of line information, which is formed by existing technology). Then, the arrangement of each component of the face frame is made between any one of several contour groups included in the contour image and the contour model for forming the N face frames as the determination criteria. Do by comparing. Check if there are close contours in the N face frames. This is the first stage determination as to whether or not the person is a person.
[0080]
Next, in response to the determination result of the first stage, a final determination is made based on whether there is a body part frame in the vicinity of the face frame. The body part frame for judgment includes an arm frame and a torso frame. For example, when a plurality of arm frames are used for determination, it is determined whether the contour model of the arm frame is within a predetermined distance range from the face frame, and if this distance condition is satisfied, the face frame There is a determination method in which the image information obtained in this determination is determined as a person. Whether or not a person is a person can be determined by such a two-stage determination method.
[0081]
As for such a person determination method, for example, “Extraction of a human head and a part with a change in facial expression from a moving image using Labeled Graph Matching”, IEICE Transactions Vol.J85-D-II No. .11 pp.1656-1663 It is described in November 2002.
[0082]
In addition, when extracting these partial image data as image data information including a person, it is preferable to acquire information including at least the number information as additional information regarding the partial image data. The number information is information indicating the number of persons included in the partial image data. More preferably, the additional information includes one or both (more preferable) of color information that characterizes the color of each person or shape information that characterizes the shape of each person.
[0083]
By acquiring the additional information together with the partial image data in this way, even if there are overlapping persons in the image and the extracted partial image data includes a plurality of persons, the obtained partial image data A plurality of pieces of personal information included in the partial image data can be acquired by simultaneously acquiring additional data (additional information) whose information is managed in a tree structure as shown in FIG. It can be used as information.
[0084]
More specifically, color information and shape information are obtained by capturing partial image data, and in the captured image, for each person, which face frame group or body frame frame, such as a face frame or body part frame described later, This information is used to further characterize the pattern after it is determined. For example, color information is information that expresses the overall color tone of a frame (such as a face frame or body part frame) that has been captured and adapted for judgment, and shape information is exactly what the frame adapted for judgment is. It is information that shows its shape.
[0085]
Further, movement information may be included in the additional information. Motion information refers to a frame (face frame, body part frame, etc.) adapted for judgment being transferred to another frame in a different frame group from the frame group in the next still image data capture operation. It is information indicating that it has become compatible. Furthermore, by processing the constituent elements (eyes, nose, mouth, etc.) in the face frame into another frame group, processing corresponding to facial expression changes and the like can be performed.
[0086]
In an image in which a plurality of persons are overlapped, there are a plurality of face frames and body part frames for determining whether or not a person is in an incomplete shape and close to each other. Since the face frame and body part frame, each color information, and shape information are unique to a person, the information of each of a plurality of persons can be obtained from the color and shape information of each frame. Further, as information amount for characterizing the partial image data, information on the partial image can be managed as information amount of <number information> × <number of person determination frames> × <shape information + color information>. Easy to characterize image information. In addition, by managing with a tree structure, the search can be easily performed.
[0087]
Such color information and shape information is obtained by using a contour model dedicated temporary storage device (not shown) as the information used to determine whether the person is a person or the face frame or body frame. Information such as color and shape in the original image in the part corresponding to the position of the contour model, with reference to the image data captured for the determination of whether or not the person is the basis of the contour model Are acquired as information by a temporary storage device (not shown) dedicated to color information and a temporary storage device (not shown) dedicated to shape information.
[0088]
In addition, since the number information is information on how many pieces of person information determined to be persons in the extracted partial image data, a temporary storage device (not shown) dedicated to the number information also has the same part. It can be acquired by counting and storing how many times the contour model is acquired in the image data.
[0089]
Further, when searching for a region including a person from still image data, it is preferable to appropriately change the search region as a unit of search within a range not exceeding the maximum size of still image data. In other words, changing the search area means changing the shape (usually rectangular) and size of the search area. By appropriately switching the search area, the search area can have an appropriate shape that matches the person image, and highly accurate information about the person can be acquired in a short time.
[0090]
As a method of setting the search area, for example, there is a method using a face frame. If the size of the same person changes between the still image data acquired at the next timing and the still image data acquired at the previous timing, the change in the size of the face frame is detected and matched to the rate of change. To reduce the search area. In this case, if the change rate of the size of the face frame becomes 70%, the search area is also changed to 70%.
[0091]
In addition, when an area including a plurality of persons is searched, if there are five face frames, for example, the area is reset so that the number of the face frames is three. Individual person information included increases, and the accuracy increases.
[0092]
Thus, in the case where the search area is made smaller based on the face frame, a method of using the place where the face frame exists as the base point of the diagonal element of the search area is effective.
[0093]
Specifically, as shown in FIG. 4, the face frame sizes x1 × y1 and x2 × y2 and the distances X and Y between the face frames, and further provided as a margin for sufficiently securing the search portion, The search area size is {(m1 + 1) · x1 + X + (m2 + 1) · x2} × {(n1 + 1) by combining predetermined ranges (m1, m2, n1, and n2 are arbitrary numbers) based on the size of the original face frame. ) · Y1 + Y + (n2 + 1) · y2}.
[0094]
By acquiring scene information from partial image data that includes people as described above, there is no concept of determining changes in scenes considering various information and managing the changes in a hierarchical structure. Information that pays attention to what those persons do, etc. can be managed.
[0095]
Next, an operation of extracting partial image data including a person will be described using the flowchart of FIG.
[0096]
FIG. 5 shows a flow up to acquisition of partial image data in the case where a region including a person in still image data is extracted as partial image data while changing a search area for searching for a person as appropriate.
[0097]
When the first drive 11 detects a start signal during moving image reproduction on the first recording medium 1, timing information for acquiring one frame of moving image data reproduced by the first drive 11 is generated. Then, acquisition of still image data is started (S1).
[0098]
When timing information for acquiring moving image data is given, in synchronization with this, still image data, which is one frame of moving image data, is stored in the buffer memory 3 with its address data (address on the first recording medium 1). It accumulates with (S2, S3).
[0099]
Next, it is determined whether or not it is immediately after the start (S4), and it is subsequently confirmed whether or not a trigger for the image data transfer request is detected except for immediately after the start (S5). If an image data transfer request trigger is not detected except immediately after the start, S1 to S5 are repeated and image accumulation is continued.
[0100]
On the other hand, when it is immediately after the start and when the trigger for the image data transfer request is detected, the still image data captured in the buffer memory 3 is sent to the person information detection system (S7). The processing so far is processing of an image storage system that stores still image data.
[0101]
The person information detection system will be described below. First, image processing is performed on the transferred still image data to form the above-described contour image (S8). Next, an area including a person is searched while reviewing the search area (S9).
[0102]
In S9, with respect to the contour image obtained from the still image data, the search area is shifted in the horizontal direction by a predetermined pixel (determined according to the resolution of the contour image), and the line segment read in the search is defined as a vector. Detect direction and length information. Then, the search is advanced while obtaining the direction and length information. When the search scan in one horizontal direction is completed, the search scan is started again in the horizontal direction after shifting by a predetermined pixel in the vertical direction. This search is continued until the information that can be determined as a person is obtained for the entire contour image data, and the presence / absence information of the line segment and, if there is a line segment, the direction and length information thereof are the contour image information. Are stored in the temporary storage device.
[0103]
This search provides quantitative information about the line segments in the contour image, so the similarity between the contour model of each frame to determine whether it is a person, such as a face frame or body part frame, can be determined. When numerical comparison is made and it matches the contour model of each frame, it is determined that a person is included (S10).
[0104]
If it is determined that a person is included, a certain range including the matching portion is cut out as partial image data including the person (S12). At this time, the field information is also acquired so that the extracted partial image data can be seen in the still image data (S11). The field information is information indicating in which position in the still image the person image exists.
[0105]
The extracted partial image data and field information are associated with address information indicating the position on the first recording medium 1 of the still image data in which the partial image data was included, and the second drive 12 is used. Recording is performed on the second recording medium 2 (S13).
[0106]
Next, at the stage where the search for the entire contour image has not been completed, the search for the unsearched portion is continued, and the search for the person image is continued while the search area is reviewed (S14, S15). The processes in S12 to S15 are repeated until it is confirmed in S16 that there is no other person information.
[0107]
Further, it is more preferable to search within the range of the partial image data obtained once in a search area smaller than that. This makes it possible to obtain information such as facial expressions with high accuracy that can characterize the scene in a short time.
[0108]
On the other hand, if it is not determined in S10 that a person is included even if the entire still image data is searched, and if it is confirmed in S16 that there is no other person information, the process of the person information detection system is completed. To do.
[0109]
When the processing of the person information detection system is completed, an image data transfer request trigger for sending the next still image data to the person information detection system is generated (S17), and the trigger data is converted into still image data. The data is sent to the accumulated buffer memory 3. Thus, the next still image data is transferred from the image storage system to the person information detection system (S7).
[0110]
At the time of this search, the person information detection system that searches for an area including a person and the image storage system that stores image data perform different movements. Therefore, after a while after the reproduction of the moving image data is finished, the process of searching for an area including a person is finished. At the end of the search, the end operation is started based on the transfer of the end information of the moving image data reproduced by the first drive 11 (S6).
[0111]
Note that the time interval (sampling timing) at which the still image data constituting the reproduced moving image data is sampled and stored in the buffer memory 3 in S1 described above is required for searching and recording the area including the person. It is preferable to set in consideration of time.
[0112]
Further, the user may be able to freely set the sampling timing. When the moving image data of the first recording medium 1 is reproduced and partial image data serving as scene information is acquired, the user grasps the content based on the moving image displayed on the display device 5. Therefore, if the timing for acquiring still image data is appropriately set according to the content, sampling according to the content of the moving image can be performed.
[0113]
Next, image retrieval of moving image data in the image recording / reproducing apparatus will be described.
[0114]
The partial image data acquired as described above includes, for example, the actions of the person, still image data that has already been extracted, that is, information that characterizes a scene with a moving image having the still image data as a component. Therefore, by displaying the partial image data itself on the display device 5, the partial image data itself becomes scene information.
[0115]
In addition, when a plurality of pieces of person information are obtained from the partial image data, the person configuration of the plurality of persons is important scene information. That is, even if these partial image data are used as they are, they provide information obtained for detecting a scene change and corresponding scene information.
[0116]
Therefore, as described above, in the present image recording / reproducing apparatus, when it is desired to search for a specific scene (image) from a number of scenes of moving image data recorded in the first recording medium 1, the main moving image A partial image data group acquired from the data and already recorded on the second recording medium 2 is used as information for image retrieval.
[0117]
In the image search, the display device 5 displays the partial image data from the partial image data group in the order along the time axis of the moving image data. At this time, the partial image data may be displayed as a thumbnail. Further, it is not necessary to display all the partial image data, and a part of the partial image data can be displayed by being thinned in time.
[0118]
And in the state where the partial image data is displayed in this way, the selection for the displayed partial image data is received from the user via an input device (not shown), and based on the selected partial image data, The moving image data recorded on the first recording medium 1 is searched for still image data from which the partial image data is extracted, and from the still image data from which the selected partial image data is extracted ( (Including before and after that), playback of moving image data is started.
[0119]
Further, in the present image recording / reproducing apparatus, when moving image data of the first recording medium 1 is reproduced for the first time by this apparatus, partial image data including a person in person information and its peripheral information is automatically acquired. It is like that. Further, in the present image recording / reproducing apparatus, when the second or subsequent reproduction of the moving image data recorded on the first recording medium 1 is instructed, one of the acquired partial image data is reproduced before the moving image data is reproduced. Are displayed as thumbnails on the screen of the display device 5 as shown by a frame 20 in FIG.
[0120]
Partial image data can be displayed even if partial image data (still images) obtained simply in the time axis direction of the moving image data is displayed for a predetermined number of times (eight in FIGS. 1 and 6) for a predetermined time. Only some of them may be selected and displayed. Further, as the selection display method, there is a method in which a partial image for display is selected at regular time intervals in the entire original moving image, and the selected partial image data is displayed.
[0121]
Since such partial image data has already been narrowed down to human actions, facial expressions, etc., it is easier for the user to view than a normal thumbnail display of the entire still image. Such a search based on already narrowed partial image data is hidden in the normal thumbnail display of the entire still image, and the information accuracy is high and finer image search (scene search) is possible. become.
[0122]
For example, if the ratio of the area in which the person is recorded in the original still image is small in the whole, it is difficult to specify the person in the thumbnail display of the entire still image. In this case, since the person image is displayed in close-up, an image search using person information as a key is possible.
[0123]
In addition, since partial information is basically managed in the partial image, for example, if color information about the body part frame is used, the partial image data is used, for example, with a costume worn by a specific person in the entire moving image as a key. It is also possible to selectively display only partial image data including the costume from the group, thereby enabling more effective scene search.
[0124]
In another example, if the shape information about the body part frame is used, the information on the shape of the foot is combined with the information on the movement, or the information on the shape of the arm is combined with the information on the movement. It is possible to search for a scene by using a person's action as a key. Further, by using field information, it is possible to search for a scene in which a specific person is moving by distinguishing whether it is walking or running.
[0125]
[Embodiment 2]
The following will describe another embodiment of the present invention with reference to FIGS. For convenience of explanation, members having the same functions as those used in the first embodiment are denoted by the same reference numerals and description thereof is omitted.
[0126]
As shown in FIG. 6, the image recording / reproducing apparatus of the present embodiment is provided with a buffer memory group 30 that can be written and read independently, as the image recording / reproducing apparatus of the first embodiment (FIG. 1). This is a very different point. In the image recording / reproducing apparatus of the first embodiment, the CPU 6 uses one buffer memory 3 to search the entire still image data while moving the search area in the still image data. Using the buffer memory group 30 composed of a plurality of buffer memories, still image data is divided into a plurality of areas in advance, a person is searched independently in each divided area, and partial image data is extracted. ing. That is, the search area is not moved.
[0127]
The operation of extracting partial image data including a person will be described using the flowchart of FIG.
[0128]
FIG. 7 shows a flow from when a region including a person in still image data is extracted as partial image data until a search region for searching for a person is divided in advance and a partial image data is acquired. .
[0129]
When the first drive 11 detects a start signal during moving image reproduction of the first recording medium 1, it generates timing information for acquiring one frame of moving image data reproduced by the first drive 11, Acquisition of still image data is started (S21).
[0130]
After this acquisition, the number N of still image data divisions is determined (S22). For example, information on whether there are a large number of people on the screen is roughly searched at the contour image data level of still image data, the state of the number of people in the entire image is analyzed, and the number of divisions is calculated based on the analysis result. decide. Although not shown in FIG. 7, the determination includes a function in which the CPU 6 starts a simple image analysis program stored in the memory 4 and performs automatic determination work. The division number N is the maximum number of buffer memories constituting the buffer memory group 30.
[0131]
The still image data divided into N pieces is divided into N pieces of N pieces of image data obtained by dividing the still image data into N pieces of 1 to N of the buffer memory group 30 capable of performing N pieces of independent writing and reading operations simultaneously with the division. As a data group, it is accumulated together with the address data in the first recording medium 1 before image division (S23, S24).
[0132]
Next, it is determined whether or not it is immediately after the start (S25), and it is confirmed whether or not an image data transfer request trigger has been detected except for immediately after the start (S26). If a trigger for an image data transfer request is not detected except immediately after the start, S21 to S26 are repeated, and image storage in the buffer memory group 30 of the divided still image data (hereinafter referred to as divided image data) is continued. To go.
[0133]
On the other hand, when it is immediately after the start and when the trigger for the image data transfer request is detected, the divided image data fetched into the buffer memory group 30 is sent to the person information detection system all at once (S27). The processing so far is the processing of the image storage system that stores the divided image data.
[0134]
Here, since the still image data is divided into N pieces and stored in the buffer memories 1 to N, there are N person information detection systems. In each of the N person information detection systems, first, image processing is performed on the transferred divided image data to form the above-described contour image (S28). Next, an area including a person is searched (S29). However, here, the search is performed without reviewing the search area.
[0135]
If it is determined in the same manner as described in the first embodiment and it is determined in S30 that a person is included, a certain range including the matching portion is cut out as partial image data including the person (S32). Also in this case, field information is also acquired so that the extracted partial image data can be seen where in the still image data (S31).
[0136]
The extracted partial image data and field information are associated with address information indicating the position on the first recording medium 1 of the still image data including the partial image data, and the second drive 12 is used. Is recorded on the second recording medium 2 (S36).
[0137]
However, since the N person information detection systems are independent and perform processing in parallel here, the processing to S2 and S35 is performed through the processing of S34 and S35, and the second drive 12 is connected to the second drive 12. This is done after permission for the recording request. Although not shown in FIG. 6 individually, the N person information detection systems have a temporary storage function.
[0138]
If it is determined in S30 that the divided image data does not include a person and search extraction means is used, and if recording on the second recording medium 2 is completed in S36, the process proceeds to S35 of the next person information detection system.
[0139]
As described above, in this embodiment, the re-search is not performed by changing the search area. For example, even if personal information obtained by overlapping a plurality of persons is acquired, individual personal information can be supplemented by a method of separately recording additional information.
[0140]
When the recording of the divided image data on the second recording medium 2 is completed in the final person information detection system, or when it is determined that no person is included in S30, the processing of the N person information detection systems ends. To do.
[0141]
When the processing of the N person information detection systems is completed, an image data transfer request trigger for sending the next divided image data to each person information detection system is generated (S37). Send data to N buffer memories that store data. Thus, the next divided image data is collectively transferred from the image storage system to the person information detection system (S27).
[0142]
Also in this search, the person information detection system that searches for an area including a person and the image storage system that stores image data perform different movements. Therefore, after a while after the reproduction of the moving image data is finished, the process of searching for an area including a person is finished. At the end of the search, the end operation is started based on the transfer of the end information of the original moving image data reproduced by the first drive 11 (S38).
[0143]
Note that the time interval (sampling timing) at which the still image data constituting the reproduced moving image data is sampled, divided and stored in the buffer memory group 30 in S21 described above is a search for an area including a person. It is preferable to set in consideration of the time required for recording, and the user may be able to freely set the sampling timing.
[0144]
As described above, the image recording / reproducing apparatus according to the present embodiment does not move the search area and does not change the search area as compared with the image recording / reproducing apparatus according to the first embodiment. Therefore, in the case of moving image data in which the still image data includes a large number of people because there is no operation such as repeated search, a process of detecting the person information by dividing such still image data into N pieces in advance. More efficient search is possible.
[0145]
In the present image recording / reproducing apparatus, the image search using the partial image data group for the second recording medium 2 is the same as that of the image recording / reproducing apparatus of the first embodiment, and the description thereof will be omitted.
[0146]
The image recording / reproducing apparatus according to the first and second embodiments described above can also be recorded as a program on a computer-readable recording medium. For example, a database construction program for causing a computer to function as still image data acquisition means, search extraction means, recording means, start instruction means, partial image data display means, search means, and a recording medium on which an image search program is recorded can be considered.
[0147]
An object of the present invention is to record a database construction program, which is software for realizing such means on a computer, and a program code (execution format program, intermediate code program, source program) of an image search program so that the computer can read it. This can also be achieved by recording on a medium, supplying the recording medium to an image recording / reproducing apparatus, and reading and executing the program code recorded on the recording medium by the computer. In this case, the program code itself read from the recording medium realizes the above-described procedure, and the recording medium recording the program code constitutes the present invention.
[0148]
Here, the recording medium as the program medium is a recording medium configured to be separable from the main body, such as a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a flexible disk or a hard disk, or a CD-ROM / MO /. It carries a fixed program including disk systems for optical disks such as MD / DVD, card systems such as IC cards (including memory cards) / optical cards, or semiconductor memories such as mask ROM, EPROM, EEPROM, flash ROM, etc. It may be a medium.
[0149]
The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the technical means disclosed in different embodiments can be appropriately combined. Embodiments to be described are also included in the technical means of the present invention.
[0150]
【The invention's effect】
As described above, the database construction apparatus of the present invention includes still image data acquisition means for acquiring still image data from moving image data of the first recording medium reproduced by the first drive, and the still image data acquisition described above. The storage means for storing still image data acquired by the means, and a person included in the still image data stored by the storage means are searched, and if a person is included, a predetermined area including the person is partially Search extraction means for extracting as image data, and when partial image data is extracted by the search extraction means, address information on the first recording medium of still image data from which the extracted partial image data is extracted And a recording means for recording on a second recording medium using a second drive.
[0151]
According to this, partial image data obtained by extracting an area including a person is recorded on the second recording medium and is made into a database. Such partial image data is displayed by itself, and the partial image data itself becomes scene information. In addition, when a plurality of pieces of person information are obtained from the partial image data, the person configuration of the plurality of persons is important scene information. That is, even if these partial image data are used as they are, they become information obtained for detecting a scene change and corresponding scene information.
[0152]
Moreover, since such scene information is partial image data obtained by cutting out an area including a person, the database can be constructed by automatically acquiring the existing technology without bothering the user. .
[0153]
Therefore, it is possible to provide an apparatus for constructing a database of scene information that can automatically acquire scene information suitable for moving image data mainly composed of human images without bothering the user.
[0154]
In addition, by installing such a database construction device in an image recording / playback device, local scene changes can be easily identified, and image search in accordance with a person's movement is possible. There is an effect that it is possible to provide an image recording / reproducing apparatus capable of easily editing the contents of moving image data.
[0155]
Further, in the database construction device of the present invention, the search and extraction means further extracts additional information including number information representing the number of persons included in the partial image data when extracting the partial image data. Acquiring and recording, the recording means records the additional information corresponding to the partial image data on the second recording medium so as to be related in a tree structure connected to the partial image data.
[0156]
According to this, the additional information including the number information indicating the number of persons included in the partial image data is acquired, and the corresponding additional information together with the partial image data can be related in a tree structure connected to the partial image data. Since it is recorded on the second recording medium, even if the partial image data including the person is obtained as an image in which two persons overlap, on the data side recorded on the second recording medium, It also has the effect of being able to be managed separately by two persons.
[0157]
The additional information may further include color information representing the color characteristics of each person and / or shape information representing the shape characteristics of each person.
[0158]
If the color information is used, for example, it is possible to search for a specific scene using a costume worn by a specific person as a key in the entire moving image. In another example, if shape information is used, it is possible to search for a specific scene by capturing the change in shape and using the action of a specific person as a key.
[0159]
Therefore, when searching for moving image data based on the partial image data extracted from the region including people as the scene information, the number of search key items increases and a database that can perform more effective search is constructed. It also has the effect of being able to.
[0160]
In the database construction device of the present invention, it is preferable that the search and extraction means extract partial image data so that all persons included in still image data are included in any partial image data.
[0161]
According to this, since all the persons included in the still image data are acquired as partial image data independently or together with other persons, the effect that the information of the scene information becomes more accurate can be obtained. Play.
[0162]
Further, in the database construction device of the present invention, the search extraction means can be characterized in that it is performed while appropriately changing the search area when searching for a person.
[0163]
According to this, since the search area at the time of searching for a person is changed as appropriate, the search area can be set even for moving image data or the like in which the ratio of the person information to the still image data changes arbitrarily during the search. The search area can always be appropriately determined by reducing the maximum range of the screen step by step, and the search can be performed more efficiently. In addition, when searching for an area including a plurality of persons in the initial search, there is an effect that it is possible to easily perform a search again by changing the search area in order to further improve the information accuracy.
[0164]
In the database construction device of the present invention, the search and extraction means may divide still image data into a plurality of areas in advance, and search the divided areas as search areas.
[0165]
According to this, since still image data is divided into a plurality of regions in advance and each divided region is searched as a search region, a moving image that is expected to have a large number of people recorded in the cut out still image data in advance. In the case of image data, it is possible to appropriately reduce the number of persons existing in an area including a person to be found and extracted by a single search, and to reduce the number of individual person information. In addition, if the search processing is performed independently in each divided area, there is also an effect that a person can be extracted from still image data in a shorter time.
[0166]
In addition, when dividing still image data in advance, the above search is provided with a division number determining means for determining the state of the number of persons included in the still image data and determining the number of divisions of the still image data based on the determination. More preferably, the extracting means divides the image data by the division number determined by the division number determining means.
[0167]
Thereby, since the division of the still image data is performed according to the number of persons included in the still image data, the number of divisions of the still image data was in the state of the number of persons included in the still image data. Thus, there is an effect that the number of pieces of individual person information can be made more accurate than the configuration in which the number of divisions is fixed.
[0168]
In the database construction device of the present invention, it is determined whether or not the moving image data of the first recording medium is reproduced for the first time in the first drive. It can also be configured to include a start instruction means for starting acquisition of still image data.
[0169]
According to this, when the reproduction of the moving image data of the first recording medium in the first drive is the first time, the acquisition of the still image data by the still image data acquisition unit is started by the start instruction unit. The user can also obtain the scene information database by reproducing the first recording medium with the first drive without giving a special instruction.
[0170]
In order to solve the above problems, an image recording / reproducing apparatus of the present invention includes a first drive for reproducing information recorded on a first recording medium, and a first drive for recording / reproducing information on a second recording medium. An image recording / reproducing apparatus including two drives includes the database construction apparatus according to any one of the first to ninth aspects.
[0171]
As already described as a database construction apparatus, the database construction apparatus of the present invention is a database construction of scene information suitable for moving image data mainly of human images and capable of automatically obtaining scene information without bothering the user. Device.
[0172]
Therefore, the image recording / reproducing apparatus of the present invention equipped with such a database construction apparatus is capable of easily discriminating local scene changes and the like, and can search for images according to the movement of the person. As a result, it is possible to provide an excellent image recording / reproducing apparatus capable of easily editing the contents of the moving image data.
[0173]
The image search method of the present invention is the image search method of the image recording / reproducing apparatus of the present invention, wherein the partial image data recorded on the second recording medium is used as scene information, and the first recording medium is used. An arbitrary scene is searched in the moving image data recorded in the video.
[0174]
As described above, the partial image data recorded on the second recording medium includes, for example, the action of the person, still image data that has already been extracted, that is, a moving image having the still image data as a component. Therefore, by displaying partial image data, the partial image data itself becomes scene information. In addition, when a plurality of pieces of person information are obtained from the partial image data, the person configuration of the plurality of persons is important scene information. That is, even if these partial image data are used as they are, they become information obtained for detecting a scene change and corresponding scene information.
[0175]
Accordingly, since the search is performed using scene information at a fine level including the motion of the person included in the image, it is possible to perform a more efficient image search such as a search when reproducing moving image data centered on a person. Play together.
[0176]
The image search apparatus of the present invention is an image search apparatus provided in the image recording / reproducing apparatus, and reproduces a partial image data group recorded on the second recording medium using the second drive. A first image based on the partial image data displayed on the display means, an input means for receiving selection of the displayed partial image data from the user, and the partial image data selected by the input means; It is characterized by comprising search means for searching still image data from which the partial image data is extracted from moving image data recorded on a recording medium.
[0177]
According to this, the partial image data reproduction display means reproduces the partial image data group recorded on the second recording medium using the image second drive, and causes the display means to display it. When the user selects one of the displayed partial image data using the input means, the search means searches the moving image data for still image data from which the partial image data selected by the input means is extracted. To do.
[0178]
Accordingly, it is possible to search for necessary image data (scene search) by directly looking at the movement, facial expression, etc. of the person recorded on the second recording medium. Finer search is possible than when displayed. For example, even when the ratio of the area where the person is recorded in the original still image is small in the whole, the image search using the person information as a key can be performed. In other words, since the display is narrowed down, it is possible to perform a search that is easier for the user to see and has high information accuracy.
[0179]
In the image search device according to the present invention, the partial image data reproduction / display means may further execute partial image data before reproduction of the moving image data when the reproduction of the moving image data recorded on the first recording medium is instructed. It is also possible to display a part of the screen.
[0180]
According to this, when the partial image data group of the moving image data on the first recording medium is on the second recording medium, a part of the partial image data is automatically reproduced before the moving image data is reproduced. Therefore, for example, the user sees the partial image data, and also has an effect that it is possible to selectively reproduce only the scene to be viewed.
[0181]
The database construction program and recording medium of the present invention are a program for causing a computer to function as each means in the above-described database construction apparatus of the present invention, and a recording medium recording the program.
[0182]
The image search program and recording medium of the present invention are a program for causing a computer to function as each means in the above-described image search apparatus of the present invention and a recording medium on which the program is recorded.
[0183]
Thus, if the above-described database construction program or image retrieval program is executed by a computer, the present invention can be applied not only to a specific database construction device, image retrieval device, and image recording / reproduction device but also to an unspecified image recording / reproduction device. This also brings about the effect that the database construction device, the image search device, and the image recording / reproducing device of the invention can be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an image recording / reproducing apparatus according to an embodiment of the present invention.
FIGS. 2A and 2B are explanatory diagrams illustrating an example of scene information acquisition in the image recording / playback apparatus.
FIG. 3 is an explanatory diagram showing an information addition structure in person information search in the image recording / reproducing apparatus.
FIG. 4 is an explanatory diagram showing a search area changing method when searching for a person in the image recording / reproducing apparatus.
FIG. 5 is a flowchart showing a procedure for automatically obtaining personal information in the image recording / reproducing apparatus.
FIG. 6 is a block diagram showing a configuration of an image recording / reproducing apparatus according to another embodiment of the present invention.
FIG. 7 is a flowchart showing a procedure for automatically obtaining personal information in the image recording / reproducing apparatus.
FIG. 8 is a diagram showing a scene information input system of the invention disclosed in the conventional publication.
FIG. 9 is a flowchart showing a procedure of scene information input according to the invention disclosed in the above-mentioned publication.
[Explanation of symbols]
1 First recording medium
2 Second recording medium
3 Buffer memory (storage means)
4 memory (still image data acquisition means, search extraction means, recording means, start instruction means, partial image data display means, search means)
5 display means
6 CPU (still image data acquisition means, search extraction means, recording means, start instruction means, partial image data display means, search means)
11 First drive
12 Second drive
30 buffer memory group (storage means)

Claims (17)

Still image data acquisition means for acquiring still image data from moving image data of the first recording medium reproduced by the first drive;
Storage means for storing still image data acquired by the still image data acquisition means;
A person included in the still image data stored by the storage means is searched by comparing a contour image obtained by processing the still image data into a plurality of line information, and a face frame and a body frame for determining the person. Search and extraction means for extracting a predetermined area including a person as partial image data for each of all persons included in the still image data;
When the partial image data is extracted by the search extraction means, the extracted partial image data is associated with the address information on the first recording medium of the still image data as the extraction source, and the second drive is used. And a recording unit for recording a contour model indicating a contour determined by the search and extraction unit to be highly similar to the face frame and the body frame. . The search extraction means, when extracting the partial image data, the number of times said recording means has recorded the contour model, counted as the number of persons included in the partial image data, the number information indicating the number Acquire additional information including
2. The database construction according to claim 1, wherein the recording means records the additional information corresponding to the partial image data on the second recording medium so as to be related in a tree structure connected to the partial image data. apparatus. The additional information as the color information representing the color characteristics of each person, in the still image data the search extracting means has extracted partial image data, information including indicating the color of a portion corresponding to the position of the contour model The database construction device according to claim 2 characterized by things. The additional information, the shape information representing the shape characteristics of each person, in the still image data the search extracting means has extracted partial image data, the contour model including information indicating the shape of a portion corresponding to the position The database construction device according to claim 2 or 3 characterized by things.   The said search extraction means extracts partial image data so that all the persons included in still image data may be included in any partial image data, The any one of Claims 1-4 characterized by the above-mentioned. Database construction device.   The database construction device according to claim 1, wherein the search extraction unit performs the search while appropriately changing a search area when searching for a person.   The database construction apparatus according to claim 1, wherein the search extraction unit divides still image data into a plurality of areas in advance, and searches the divided areas as search areas. Determining the number of persons included in the still image data, and having a division number determining means for determining the number of divisions of the still image data based on the determination;
8. The database construction apparatus according to claim 7, wherein the search extraction unit divides the image data by the number of divisions determined by the division number determination unit.   It is determined whether or not the moving image data of the first recording medium is reproduced for the first time in the first drive, and if it is the first time, a start instruction for starting the acquisition of still image data by the still image data acquisition unit The database construction device according to claim 1, further comprising means. A first drive for reproducing information recorded on the first recording medium;
In an image recording / reproducing apparatus comprising a second drive for recording / reproducing information on a second recording medium,
An image recording / reproducing apparatus comprising the database construction apparatus according to claim 1. An image search method for an image recording / reproducing apparatus according to claim 10,
An image search comprising: searching for an arbitrary scene in the moving image data recorded on the first recording medium using the partial image data recorded on the second recording medium as scene information. Method. An image search apparatus provided in the image recording / reproducing apparatus according to claim 10,
Partial image data display means for reproducing the partial image data group recorded on the second recording medium using the second drive and displaying the reproduced data on the display means;
An input means for receiving a selection for the displayed partial image data from the user;
Search for searching for still image data from which partial image data is extracted from moving image data recorded on the first recording medium based on the partial image data selected by the input means. An image search device comprising means.   The partial image data display means displays a part of the partial image data before reproducing the moving image data when an instruction to reproduce the moving image data recorded on the first recording medium is given. The image search device according to claim 12.   A database construction program for causing a computer to function as each means in the database construction device according to claim 1.   The computer-readable recording medium which recorded the database construction program of Claim 14.   14. An image search program for causing a computer to function as each means in the image search device according to claim 12 or 13.   The computer-readable recording medium which recorded the image search program of Claim 16.

Images