JP4556739B2 - Image management apparatus, image management method, and image management program - Google Patents

Description

  The present invention relates to an image management apparatus, an image management method, and an image management program, and more particularly to an image management apparatus, an image management method, and an image management program capable of easily managing images.

  With the spread of digital cameras, the number of photographic images that individuals should manage has increased. Considering 10 to 20 years or more, it is expected that it will become increasingly difficult to reach an image to be seen from a group of photographic images such as mountains owned by individuals. Especially for old photos more than 10 to 20 years ago, the memory of the date and time is vague, and it is difficult to search only from the shooting date and time information, so a search technique based on the contents of the photo (especially a specific subject such as a person) is required. It has been.

  Patent Document 1 below discloses an image data search method that automatically determines whether or not a specific person appears in a photographic image.

The following Patent Document 2 adds and records information for dividing a plurality of images into a plurality of groups, reproduces representative images for each of the plurality of image groups, displays a plurality of images simultaneously, and based on the displayed plurality of representative images. An electronic image filing device that designates a desired image group and designates and displays a desired image from the image group is disclosed.
JP 2001-357067 A Japanese Patent Laid-Open No. 5-128166

  As described above, stored images may be left unorganized, and it may take effort to arrive at the desired image even if it is to be viewed.

  SUMMARY An advantage of some aspects of the invention is that it provides an image management apparatus, an image management method, and an image management program capable of easily managing recorded images.

In order to achieve the above object, according to one aspect of the present invention, an image management apparatus is an image management apparatus that manages a plurality of images as a plurality of groups, and inputs information that specifies a person to be searched as a predetermined subject. and means, in each image within each group of a plurality of groups, and detecting means for detecting a predetermined subject, for each group, in the group number of images, as well as in the group given detected in each image The distance and area from the image center of the subject is detected, and the smaller the number of images in the group, the closer the subject is to the image center, and the larger the area of the subject, the representative in each group of subjects detected by the detecting means determining means for the degree of large determining, based on the size of the degree of the determined by the determining means representative displays a representative image of each group And a shown means.

  Preferably, the determining unit determines the representative degree based on the number of images in which the subject is detected in the image group in the group.

  Preferably, the subject is a predetermined person.

  Preferably, the determining unit determines the representative degree based on at least one of a position and a size of the subject detected in each image of the image group in the group.

According to another aspect of the invention, the image management apparatus includes a holding unit that holds management information for managing an image group divided into a plurality of groups, and a detection unit that detects a predetermined subject in each image in the group. for each group, the number of images, and detects the distance and the area from the image center of the predetermined subject detected in each image in the group in the group, as the number of images in the group is small, Based on the degree of representation determined by the determination unit, the determination unit that determines that the closer the subject is to the center of the image, the larger the area of the subject, the greater the degree of representation in each group of the subject detected by the detection unit, Changing means for changing the management information.

  Preferably, the determining unit determines the representative degree based on the number of images in which the subject is detected in the image group in the group.

  Preferably, the subject is a predetermined person.

  Preferably, the determining means determines the representative degree based on the position and size of the subject detected in each image in addition to the number of images in which the subject is detected in the image group in the group.

  Preferably, the management information includes metadata defined according to a predetermined subject.

  Preferably, the management information includes a specific image in the group or information indicating the specific image.

According to yet another aspect of the present invention, an image management method, an image management method for managing a plurality of images stored in the storage device in the computer as a plurality of groups, the input device of the computer to identify the person to be searched An input step for receiving an input of information as an input of a predetermined subject ; a detection step in which a computing device of a computer accesses a storage device and detects a predetermined subject in each image in each of a plurality of groups; and a computing device There, for each group, the number of images, and detects the distance and the area from the image center of the predetermined subject detected in each image in the group in the group, enough number of images in the group is small , as the subject is close to the image center, the larger the area of the subject, the guru detected in the detection step subject Comprising a determining step of determining the degree of representative large in-flop, and a display step display device of the computer, which on the basis of the magnitude of the degree of the determined at decision step representative, displaying a representative image of each group.

According to yet another aspect of the present invention, an image management method, an image management method that holds management information for managing divided image groups stored in the storage device in the computer into a plurality of groups, calculation of the computer device, a detecting step of detecting a predetermined subject in each image in the group, computing device, for each group, the number of images in the group as well as the predetermined subject detected in each image in the group, The distance and area from the center of the image is detected, and the smaller the number of images in the group, the closer the subject is to the center of the image, and the larger the area of the subject, the representative of each group of subjects detected in the detection step. a determining step of determining the degree is large, the arithmetic unit, based on the degree of the determined at decision step representative management information varying And a changing step of.

According to yet another aspect of the present invention, an image management program is a program for executing a process for managing a plurality of images stored in the storage device to the computer as a plurality of groups, want to find the input device of the computer detecting step an input step of accepting input of information identifying a person as an input of a predetermined subject, the arithmetic unit of a computer may access the storage device, each image in each group of a plurality of groups, for detecting a predetermined subject If, computing device, for each group, the number of images, and detects the distance and the area from the image center of the predetermined subject detected in each image in the group in the group image in the group as a small number, as the object is closer to the image center, the larger the area of the object, detected by the detecting step Display step and determining step, the display device of the computer, which on the basis of the magnitude of the degree of the determined at decision step representative, displaying a representative image of each group to determine that a large degree of representatives of each group of the subject And let the computer run.

According to still another aspect of the present invention, an image management program is a program for causing a computer to execute a process of holding management information for managing an image group stored in a storage device in a plurality of groups. arithmetic unit computer, a detection step of detecting a predetermined subject in each image in the group, computing device, for each group, are detected in the respective images in the number of images as well as a group, in the group and detecting the distance and the area from the image center of a predetermined subject, as the number of images in the group is small, as the subject is close to the image center, the larger the area of the object, each of the detected at the detecting step subject a determining step of determining the degree of representatives of the group is large, arithmetic unit, the degree of the determined at decision step representative Hazuki, to execute a changing step of changing the management information to the computer.

  According to these inventions, since management based on the degree of representation of a subject can be performed, it is possible to provide an image management apparatus, an image management method, and an image management program that can easily manage recorded images. is there.

[First Embodiment]
FIG. 1 is a diagram showing a configuration of an image management system (archive system) according to the first embodiment of the present invention.

  This image management system is configured to automatically organize a large number of stored images and easily find an image to be viewed. The image management system employs a method of dividing an image into groups and managing it as a large image management method. Images are grouped and managed for each shooting date and event.

  Referring to FIG. 1, the image management system includes an image forming apparatus 1 such as an MFP (Multi Function Peripherals), client PCs 2a, 2b,. The image forming apparatus 1, client PCs 2a, 2b,... And the server 6 are connected via a network. Note that a network-connected PC, printer, or the like may be employed instead of the MFP.

  The image forming apparatus 1 is an apparatus that forms a copy image of a scanned original image, client PCs 2a, 2b,... And an image generated from print data transmitted from the server 6 on a sheet.

  FIG. 2 is a block diagram illustrating a hardware configuration of the image forming apparatus 1 of FIG.

  Referring to the figure, an image forming apparatus 1 includes a control unit 106 that controls the entire apparatus, an image reader unit 101 that reads image data from a document, a printer unit 102 that prints an image on a sheet, and a short-range wireless communication. A communication unit 103 for performing communication and connecting a printing apparatus to a network or a telephone line, a storage unit 104 for storing job data, an operation panel 105 serving as an interface with a user, and a remaining consumable item And a sensor unit 107 that detects the amount and the like.

  FIG. 3 is a block diagram showing a hardware configuration of the client PC or one server in FIG.

  Referring to the figure, a client PC (or server) includes a CPU 601 for controlling the entire apparatus, a display 605, and a LAN (local area network) card 607 (or a modem for connecting to a network or communicating with the outside). Card), an input device 609 including a keyboard and a mouse, a flexible disk drive 611, a CD-ROM drive 613, a hard disk drive 615, a RAM 617, and a ROM 619.

  Data such as programs and images recorded on the flexible disk F can be read by the flexible disk drive 611, and data such as programs and images recorded on the CD-ROM 613a can be read by the CD-ROM drive 613. Is possible.

  In the present embodiment, managed images are stored in the image forming apparatus 1, the client PCs 2 a, 2 b,.

  Note that a DVD drive, a memory card reader, or the like can be provided as an interface for inputting an image such as a photograph to the system. A hard disk drive is preferably used for image storage.

  In addition, a video signal output terminal to a television may be provided for displaying an image such as a photograph. In order to simplify the operation of the system, a remote controller may be provided. Furthermore, an image management device in which a display screen, a storage device, an image capturing device, a control device, and a remote control transmission / reception unit are integrally configured may be provided.

  Registered images such as photographs are grouped at the time of registration and stored in a directory for each group. As described above, when managing a large number of images, it is convenient to manage them in groups.

  The group classification unit may be based on activities such as a specific trip or hobby circle to which the group belongs, or may be based on time such as date. In any case, the frequency of the specific subject detected in the image group within the unit group includes useful information.

  For example, if you try to search for photos of trips that you made with your good friend A with two individual photos, if there are a lot of trips that included A, there will be many unintended photos. Will be included. However, if the search is performed in consideration of the detection frequency for each group, it is considered that a trip with two people is more likely to see Mr. A than a large number of trips. Is likely to be able to.

  Also, when you want to find a group of photos when you go to a car exhibition, you can search for a group of photos that include a car in the background by accident by detecting the vehicle in the same way. Further, when the detection frequency of a specific subject is used as in the case of search, better metadata (label) can be attached to an image group, or a better representative image can be selected from each group.

  Furthermore, by evaluating not only the detection frequency of a specific subject but also the size of the detected subject and the position in the image, photos containing subjects that appear accidentally in the background are more effectively excluded, You can search and select.

  FIG. 4 is a flowchart showing an operation performed by the image management system.

  According to this flowchart, a subject (such as a specific person) to be searched is searched from a plurality of image groups, and the representative degree of the subject in each image group is determined. Based on the degree of representation, the search result image is determined and displayed.

  Here, the representative degree is determined based on the number of images in which the subject is searched in the image group in the group and the position and size of the subject in the image.

  Referring to FIG. 4, the search is performed according to the following procedure.

・ Grouping (interactively executed by programs running on PC)
(1) The user creates a folder for each event in the root folder, and distributes images to be registered in each folder (S101). As a modification, the image distribution may be automatically performed based on information such as the photographing time as disclosed in Japanese Patent Laid-Open No. 5-128166.

・ Search (Process automatically executed by the program running on PC)
(2) The user designates a specific person A to be searched and instructs the program to execute the search (S103). This designation is performed when the user inputs an image of the person A as image data or selects an image of the person A from a plurality of images prepared in advance.

  (3) The program executed by the image management system is a person A created in advance for all images in the folders of each group in the root folder by a method disclosed in, for example, Japanese Patent Laid-Open No. 2001-357067. It is determined whether or not the person A is included in each image with reference to the face feature amount dictionary (S105). This is done by preparing templates of a plurality of sizes in the dictionary and performing template matching with each part of the target image. At this time, the size (area) S and the center position C of the face of the person A in the image are specified.

  (4) The program calculates the following evaluation value Rg (A) for each group (folder) (S107).

  Here, Rg (A) represents an average of the evaluation values ri (A) of each of a plurality of images included in a certain group g (the degree that the image is represented by the person A). ri (A) is determined by whether or not the person A is included in the image, the face area of the person A in the image, and the deviation of the face of the person A from the image center position.

  (5) The program displays representative images from each group in a predetermined layout in descending order of the evaluation value Rg (A) by the predetermined number of groups (S109). The representative image of each group is an image corresponding to the one having the largest value of ri (A) in each group.

  (6) The user browses the displayed image group and clicks on a display image of a group that seems to be a desired group to make a selection (S111).

  (7) The program displays all images of the selected group on the display in a predetermined layout (S113).

  In the above embodiment, the grouping can also be performed dynamically during the search.

[Second Embodiment]
Since the configuration of the image management system in the second embodiment is the same as that in the first embodiment, description thereof will not be repeated here.

  The present embodiment is characterized in that automatic indexing using metadata and representative images is performed by the system.

  The image grouping procedure can be performed in the same manner as in the first embodiment. In the present embodiment, it is assumed that group folder management information referred to by the program is stored in the root folder as a file.

  FIG. 5 is a diagram showing a configuration of group folder management information.

  It is assumed that the management information includes a plurality of text metadata for each group folder and a pointer to the representative images. The pointer to the representative image is expressed as a management number of a certain image in the group folder.

  FIG. 6 is a flowchart illustrating an operation performed by the image management system according to the second embodiment.

  According to this flowchart, when a new group folder g including a plurality of images is registered, metadata is assigned to the folder g and a representative image is automatically selected, and management information is updated based on this.

  Referring to FIG. 6, the management information is updated by the following procedure.

-Automatic indexing at the time of image registration (processing automatically executed by a program running on a PC)
(1) The user designates the newly registered group folder g and instructs the program to execute indexing (S201).

  (2) The program executes the steps (3) to (4) (S105 to S107) of the first embodiment for each of the persons A, B, C,... Registered in advance, and Rg (A) , Rg (B), Rg (C),... Are calculated (S203). For those persons whose values exceed a predetermined threshold t, the corresponding person name (s) is obtained by referring to a predetermined dictionary, and the person names are added to the group metadata of the management information (S205). . However, at this time, metadata is registered in descending order of value.

  (3) The program calculates the following evaluation values ri (A, B, C,...) For each image for each group (S207).

  (4) The image (s) whose evaluation value ri (A, B, C,...) Exceeds the predetermined threshold value t ′ is added to the management information as the group representative image (S209). However, at this time, representative images are registered in descending order of value.

  FIG. 7 is a flowchart showing processing at the time of browsing after image registration.

  This flowchart is automatically executed by a program operating on the PC.

  Referring to FIG. 7, in S301, the user instructs the program to execute browsing.

  In S303, the program displays the first representative image and group metadata of each group as a pair on the display with a predetermined layout.

  FIG. 8 is a diagram showing a specific example of display on the display during browsing.

  Referring to the figure, a set of the first representative image and group metadata of each group is displayed on a plurality of displays.

  Note that the process of calculating the evaluation value ri (A, B, C,...) May be based on any of the following expressions.

[Modification]
In the present embodiment, text data of a person name is used as metadata. However, it is possible to use arbitrary data associated with a target subject such as arbitrary text data or voice instead. It is.

  In this embodiment, the display of the representative image is limited to one for one registered subject. However, the screen can be divided and displayed at the same time, or the representative image displayed by clicking the image can be displayed. It is also possible to take a configuration in which the image is switched to another one.

  In addition, it is possible to perform indexing using only metadata or only representative images.

  When indexing is performed on a group that has already been indexed, the metadata and the representative image may be updated.

  Furthermore, a high-speed search can be performed using the result of automatic indexing. In this method, for example, an image group of a predetermined person is searched by text-searching the metadata provided by automatic indexing using the person name. In addition, when metadata is not added, high-speed search is possible by performing subject (person) detection using only representative images.

[Effects of the embodiment]
As described above, according to the embodiment of the present invention, a group of photographic images closely related to a specific subject (person) can be searched (or metadata representing a subject in the group of photographic images). Or a representative image of a group can be selected).

  Photo images may be managed in units of groups, or may be grouped when performing a search. At this time, there may be images belonging to the groups overlapping each other. Management groups are usually organized in units such as semantic and temporal events.

  When searching for personal image data using the search key, it is better to search for “image groups” that are closely related to the search key than to search for “individual” images that are closely related to the search key. It is thought that there are many. In the present embodiment, it is possible to provide a search method that meets such a demand.

  In addition, a group that is closely related to the search key can be detected based on the number (frequency) of images that are closely related to the search key in the group.

  The above-described processing may be performed by a PC computer, an image forming apparatus computer, or a server computer.

  Note that the processing in the above-described embodiment may be performed by software or by using a hardware circuit.

  In addition, a program for executing the processing in the above-described embodiment can be provided, and the program is recorded on a recording medium such as a CD-ROM, a flexible disk, a hard disk, a ROM, a RAM, and a memory card and provided to the user. You may decide to do it. The program may be downloaded to the apparatus via a communication line such as the Internet.

  Further, the present invention can be applied to a system connected to a network and a system not connected to a network environment.

  In addition, it should be thought that the said embodiment is an illustration and restrictive at no points. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a figure which shows the structure of the image management system in the 1st Embodiment of this invention. FIG. 2 is a block diagram illustrating a hardware configuration of the image forming apparatus 1 in FIG. 1. FIG. 2 is a block diagram showing a hardware configuration of one client PC or server in FIG. 1. It is a flowchart which shows the operation | movement which an image management system performs. It is a figure which shows the structure of the management information of a group folder. It is a flowchart which shows the operation | movement which the image management system in 2nd Embodiment performs. It is a flowchart which shows the process at the time of browsing after image registration. It is a figure which shows the specific example of the display of the display at the time of browsing.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Image forming apparatus, 2a, 2b Client PC, 6 server, 102 Printer part, 103 Communication part, 104 Memory | storage part, 105 Operation panel, 106 Control part, 601 CPU.

Claims (11)

An image management apparatus that manages a plurality of images as a plurality of groups,
Input means for inputting information for identifying a person to be searched as a predetermined subject;
Detecting means for detecting the predetermined subject in each image in each of a plurality of groups;
For each group, the number of images in the group, as well as to detect the distance and the area from the image center of the predetermined object detected in each image in the group, as the number of images in the group is small, Determining means for determining that the closer the subject is to the center of the image, the greater the area of the subject, and the greater the degree of representation in each group of subjects detected by the detecting means;
An image management apparatus comprising: display means for displaying a representative image of each group based on the magnitude of the representative degree determined by the determining means.   The image management apparatus according to claim 1, wherein the determining unit determines a representative degree based on a number of images in which a subject is detected in an image group in the group. Holding means for holding management information for managing an image group divided into a plurality of groups;
Detecting means for detecting a predetermined subject in each image in the group;
For each group, the number of images in the group, as well as to detect the distance and the area from the image center of the predetermined object detected in each image in the group, as the number of images in the group is small, Determining means for determining that the closer the subject is to the center of the image, the greater the area of the subject, and the greater the degree of representation in each group of subjects detected by the detecting means;
An image management apparatus comprising: changing means for changing the management information based on the degree of representation determined by the determining means. The image management apparatus according to claim 3 , wherein the determination unit determines a representative degree based on the number of images in which a subject is detected in an image group in the group. The determination means determines a representative degree based on the position and size of a subject detected in each image in addition to the number of images in which the subject is detected in an image group in the group. Item 5. The image management device according to Item 3 or 4 . The image management apparatus according to claim 3 , wherein the management information includes metadata defined according to a predetermined subject. The image management apparatus according to claim 3 , wherein the management information includes a specific image in the group or information indicating the specific image. In a computer, an image management method for managing a plurality of images stored in the storage device as a plurality of groups,
An input step of accepting input of information, the input device of the computer to identify the person to be searched as an input of a given object,
Computing device of the computer, to access the storage device, each image in each group of a plurality of groups, a detection step for detecting the predetermined object,
The arithmetic unit, for each group, the number of images, and detects the distance and the area from the image center of the predetermined object detected in each image in the group in the group image in the group A determination step of determining that the smaller the number of images, the closer the subject is to the center of the image, and the greater the area of the subject, the greater the degree of representation in each group of subjects detected in the detection step;
An image management method , comprising: a display step in which the display device of the computer displays a representative image of each group based on the magnitude of the representative degree determined in the determining step. An image management method for holding management information for managing a group of images stored in a storage device in a computer in a plurality of groups,
A detection step of detecting a predetermined subject in the arithmetic unit of the computer, each image in the group,
The arithmetic unit, for each group, the number of images, and detects the distance and the area from the image center of the predetermined object detected in each image in the group in the group image in the group A determination step of determining that the smaller the number of images, the closer the subject is to the center of the image, and the greater the area of the subject, the greater the degree of representation in each group of subjects detected in the detection step;
An image management method, comprising: a changing step in which the arithmetic device changes the management information based on a representative degree determined in the determining step. A computer, a program for executing processing for managing a plurality of images stored in the storage device as a plurality of groups,
An input step of accepting input of information, the input device of the computer to identify the person to be searched as an input of a given object,
A detecting step in which an arithmetic device of the computer accesses the storage device and detects the predetermined subject in each image in each of a plurality of groups;
The arithmetic unit, for each group, the number of images, and detects the distance and the area from the image center of the predetermined object detected in each image in the group in the group image in the group A determination step of determining that the smaller the number of images, the closer the subject is to the center of the image, and the greater the area of the subject, the greater the degree of representation in each group of subjects detected in the detection step;
Display device of the computer, on the basis of the magnitude of the degree of representative determined in the determining step, to execute a display step of displaying the representative image of each group in the computer, image management program. A program for causing a computer to execute processing for holding management information for managing an image group stored in a storage device in a plurality of groups,
A detecting step in which the computing device of the computer detects a predetermined subject in each image in the group;
The arithmetic unit, for each group, the number of images, and detects the distance and the area from the image center of the predetermined object detected in each image in the group in the group image in the group as a small number, the determination step of the subject is closer to the image center, as the area of the object is large, it is determined that a large degree of representatives of each group of the subject detected by the detecting step,
The arithmetic unit, based on the degree of representative determined in the determining step, to execute a changing step of changing the management information to the computer, an image management program.

Images