JPH10124655A - Device for preparing digital album and digital album device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract an image suited to a retrieval condition by automatically retrieving an image photographed by a digital camera. SOLUTION: An image photographed by using a digital still camera and an index simultaneously recorded on a film at the time of photographing are registered and retrieved, and an image suited to a retrieval condition is displayed or recorded in a digital album device. The data of an APS film 7 are scanned and read by using large amount of film automatic reading device 30 of the APS, and transmitted to a digital album device 10. A film 8 after scan is ejected to a housing box or the like. The automatic film reading device 30 successively scans the plural films, and transmits the data to the digital album device 10 only when the plural films are inputted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for creating a digital album using a digital still camera, and more particularly, to automatically search for an image taken by a digital camera and extract an image that meets the search conditions. The present invention relates to an apparatus for creating a digital album using a digital still camera.

[0002]

[Prior art]

(Digital Album) Recently, digital still cameras have been widely used as image input devices for computers. Such digital still cameras (hereinafter, abbreviated as digital cameras) are expected to increase their share in the future and to significantly increase the usage rate in ordinary households. As software related to the use of digital cameras in ordinary households, it can be guessed that the most common use is to use an image taken by a digital camera on a personal computer instead of the conventional album. Here, this “system for viewing a snapshot taken by a digital camera on a personal computer” is referred to as a digital album. Compared to a general-purpose image database system such as the conventional Ricoh "Kuraku", the digital album is different in that the target is limited to a snapshot photo album. Such digital albums are expected to be increasingly commercialized by various companies in the future. For example, Kodak Japan sells software that allows you to edit a personal computer album with a simple operation called "PC album editing software: music box with feelings". Also, as a system that has already been commercialized, a child's growth record is edited with photos and voices, and a CD-R
Some are converted to OM and viewed on a personal computer. The market for such digital albums will continue to expand as hardware advances are expected to further lower the price per photo.

[0003]

[Problems to be solved by the invention]

Digital album search problem:
How do you search for a digital album? For analog cameras (the ones we currently use), DP
After submitting to E, I see the photos but it is troublesome to organize them.
Often, you don't know which photos go where without putting them on the album. If you have time to organize your album and add comments,
In many cases, it takes quite a while to accumulate photos, and it is often impossible to find them later. The same problem will occur with digital cameras. Even though photos taken with a digital camera are stored on a personal computer, they are not organized, and the photos accumulate too soon, making it difficult to find them later. In order to solve these problems, a search mechanism on a digital album is required.

Problems with image search: Image search is generally difficult, and is not at a practical level at present. In the case of a still image, when searching for a target image from an image database, features such as color histogram, texture, and contour extraction are extracted, the similarity between the images is expressed using the extracted features, and the similarity is used. Although a method has been adopted, it is not effective at present. Various search methods have been proposed for moving images such as videos, but none are effective. The cheapest and most reliable way to do this today is to add a character index to the image. For example, the name of the leading actor or director, the date of creation, etc. are added to the videotape of the movie. In the case of a digital album, the user inputs the shooting date, place, comments, etc. on a personal computer. The search is performed by using these character string indexes, and the search by the image itself is difficult, so that the search is not performed.

[0005] The problems of this existing system can be summarized as follows. -The probability of finding the desired image is low even if the image search is performed on the image itself (image search technology is not yet mature). -It is recommended to use a character string index instead, but it is troublesome to add a character string index one by one, and there is a high possibility that an image is left unattended.

[0006]

According to a first aspect of the present invention, an image photographed using a digital still camera and an index simultaneously recorded on a film at the time of photographing are registered and searched, and a photograph matching the search condition is registered. It is characterized by being displayed or recorded on a digital album device.

According to a second aspect of the present invention, in the first aspect of the present invention, the digital album apparatus includes a face portion detecting section for detecting a face portion from the image, and a hair portion detecting section for detecting a hair portion from the image. A clothing part detection unit that detects a clothing part from the image, and a human body part detection unit that detects a human body part from the image.

According to a third aspect of the present invention, in the second aspect of the present invention, the digital album apparatus further comprises a face portion detecting section for detecting a face portion from the image, and further detecting a face from the portion detected by the face portion detection. It is characterized by including a face part detecting section for detecting a portion and a glasses detecting section for detecting the presence or absence of glasses.

According to a fourth aspect of the present invention, there is provided an index creating section for creating an index from information obtained from a digital still camera, an image index creating section for creating an image obtained from the digital still camera by image recognition. A combination index creating unit for creating a new index by combining two indexes is provided, and a target picture is efficiently searched from a huge number of pictures.

According to a fifth aspect of the present invention, in the invention of the fourth aspect, the combination index creating section sets an index creating rule in advance and creates a new index using the rule. It is a characteristic.

According to a sixth aspect of the present invention, in the fourth aspect of the present invention, the digital album apparatus further comprises a face portion detecting section for detecting a face portion from the image, and a hair portion detecting section for detecting a hair portion from the image. A clothing part detection unit that detects a clothing part from the image, and a human body part detection unit that detects a human body part from the image.

According to a seventh aspect of the present invention, in the sixth aspect of the present invention, the digital album apparatus further comprises a face portion detecting section for detecting a face portion from the image, and further detecting a face from the portion detected by the face portion detection. It is characterized by including a face part detecting section for detecting a portion and a glasses detecting section for detecting the presence or absence of glasses.

An eighth aspect of the present invention is a digital camera having an external communication function and a digital album apparatus having a function of receiving external communication, wherein images and indexes are transferred from the digital camera to the digital album apparatus. It is characterized in that the photographed information can be sent at any time by transmitting.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention is characterized in that a digital camera creates an index on behalf of a human, and enhances the effectiveness of searching by an image itself by narrowing down an object to a snapshot of a human. There is.
The following two types of indexes are created.

1. Index Created Using Image Recognition Technology For example, there is use of a technology for detecting a human face from a natural image. The following indexes can be created from human face detection technology. Existing technologies for face detection include "Kure, Yauchida, Chen: Face Detection from Color Natural Images: Face Detection from Color Natural Images Using Fuzzy Pattern Matching, Image Lab, November 1995, pp. 19-22" For example, the following information can be obtained. (A) Number of Faces Included in Photo The number of faces can be indexed as the number of people in the picture. (B) Face Size An index indicating whether the image is a close-up photograph of the face, the lower body, or a photograph of the entire body. (C) Length of Hair Assuming that the black portion around the face is hair, the length can be determined. From there, you can create an index for hair length. (D) Face Color After detecting the face part, the average value of the color of the part recognized as the face can be determined. From there, an index called face color can be created. (E) Presence or absence of glasses After detecting the face portion, it can be determined whether there is an extraneous portion corresponding to the glasses above the portion recognized as the face. From there,
You can create an index of the presence or absence of glasses. (F) Color of clothes After detecting the face part, the area of the clothes can be estimated in the area under the face, and it is possible to estimate the position of the neck of the clothes from the size of the face. It is possible to infer the area of the clothes. From there, we create an index for the color of the clothes.

2. Index created based on information added at the time of photographing by the digital camera: For example, it is assumed that the following photographing information is stored together with image data at the time of photographing. (A) Date and time of photography (b) Location of photography (c) Focal length at the time of photography (d) Brightness (amount of light) at the time of photography (e) Position on focused photograph There are two types of usage. (I) Use as it is as a search condition. For example, a search can be performed under a search condition of “degree of brightness at the time of shooting”. (Ii) Use that information for image recognition technology. For example,
The “focused position on the photograph” data can be used to detect a face portion under the assumption that “often focuses on a human face”.

Next, a digital album search function will be described. The user inputs search conditions as shown in Table 1 on a personal computer, and obtains a corresponding photograph as a result. Note that there are search conditions as shown in Table 2 for the target person.

[0018]

[Table 1]

[0019]

[Table 2]

Configuration of Digital Album Device: Photo data registration module Registers film data taken with a digital camera. 2. Search index creation module Creates the above two types of indexes from film data. As a result, a search index is created and added to each photo. This module is further divided into: (A) Face detection module (b) Hair detection module (c) Clothes detection module Search module Search conditions are input from the screen interface of the digital album device. After that, a photo matching the search condition is searched using the search index. 4. Search result layout module Lay out photos (generally multiple photos) of search results on the screen.

Hereinafter, a mechanism for performing the above search will be described. Table 3 shows the hardware functions for digital cameras.
It is as follows. Table 3 shows the feasibility, but most can be obtained with the current hardware technology.

[0022]

[Table 3]

As a method of determining the position of the photographing place, a position determining system used in a car navigation system may be suitably applied. In a car navigation system, the position is specified using a combination of satellites, geomagnetic sensors, signposts along the road, and the like. Since it is expected that various devices will be provided with a “position identification function” in the future as an application of a global positioning system (GPS) using satellites, it is expected that the addition to a digital camera will be possible in the near future.

Search Mechanism: The key point of the digital album search of the present invention is to obtain a high search hit rate that could not be realized by general objects by narrowing down the objects to snapshots of people. Representative snapshots can be classified as follows.・ People's photos Upper body 2. Adult whole body 3. 3. The whole child (for example, a baby snap) Arrangement (for example, relatives photo of wedding) For each of the above types, the number of people is added as a parameter.・ Photos of buildings (for example, five-story pagoda of Horyu-ji Temple) ・ Photos of landscapes (for example, Mt. Fuji) ・ Photos of flowers

For example, only a photograph of a person is extracted as follows. Other photos are excluded from the search. Narrow down sequentially in the following order. 1. If the focal length is 10 m or more, it is regarded as a building / landscape and deleted. 2. If the focal length is 80 cm or less, it is regarded as a close-up of a flower or the like and deleted. 3. The outline of the portrait is extracted in advance, and several types of outline templates of the portrait are prepared. The contour of the target photo is extracted, and the position on the focused photo is compared with the portrait photo template described above, and a suitable one is selected. For the comparison of the contours, a technique of contour extraction studied in image processing and its comparison technique are used.

In the following, an internal search mechanism will be described for each search condition. Date of shooting: This is obtained from a digital camera. This index is the most effective search condition. Shooting location: This is obtained from the navigation system function of the digital camera. This index is the most effective search condition. Number of persons: Determined by contour extraction using a person contour template.

The following search conditions are first determined by contour extraction.
It is performed after determining which part is a person. In particular, in an autofocus camera, a human face is often set as the focal position, and the following is performed based on the assumption that an area around the human face is a face. Gender: The following are prepared as evaluation functions for determining gender. 1. The length of the hair. 2. Face color. 3. Lip redness. 4. The vividness of clothes. 5. With or without necklace. 6. Wearing pants. 7. Height and physique. These can be calculated from the focal length, and the factors of such an evaluation function are parameterized so that they can be further increased by experiments. Further, the weighting of each parameter is made to be a mechanism for learning by a neurosystem so as to be sure as the learning is performed.

A mechanism for determining a search condition will be described below. Face color: The position of the face is determined in some cases by the following method. 1. The position of the focal position is assumed to be a face. 2. The position of the face is determined from the person outline template. Makeup: Check the following for the face. 1. The presence or absence of the eye shadow is determined based on whether the space between the eyes and the eyebrows is brown, blue, or the like. 2. The presence or absence of lipstick is determined by whether the lips are red. 3. It is determined based on whether the shape of the eyebrows is thin and not shabby. Presence of glasses: Determined by whether there is a frame such as black around the eyes. Clothes color: Judgment is made based on colors below the face.

(A) Modules for creating an index using an image recognition technique include the following.・ Face part detector ・ Detector of face parts (eyebrows, lips, eyes, etc.) ・ Hair part detector ・ Clothing part detector ・ Human body part detector ・ Eyeglass presence detector

The following indexes are created secondarily from this module. The value in parentheses indicates the attribute value.・ Number and number of faces in the picture ・ Face size (up face, upper body, whole body) ・ Length of hair (bald, short hair, long hair) ・ Body (obesity, tall, etc.) ・ Face color (white) , Black) ・ Clothing (skirt, pants) ・ Color of lips ・ Color around eyes (with or without eye shadow) ・ Brows thickness

(B) Search index based on the above combination An example of a rule prepared in advance when the following search conditions are given is given. * Search for photos of all of your wedding relatives. In this method, candidates can be narrowed down by the following rules for combining indexes.・ The whole body is shown.・ The number of people is 5 or more.・ Bright when shooting.

* Search for a picture of a child blowing out the candles on a birthday cake. In this method, candidates can be narrowed down by the following rules for combining indexes.・ One person.・ Up face. -The brightness at the time of shooting is low.

* Search for a commemorative photo of a male student at a seaside school. In this method, candidates can be narrowed down by the following rules for combining indexes.・ The number of people is 20 or more.・ Whole body.・ The clothes are flesh-colored.・ The complexion is black.

* Search for bromide pictures of a professional baseball player with the Hanshin Tigers. In this method, candidates can be narrowed down by the following rules for combining indexes.・ One person.・ Whole body or half body. -The pattern of the clothes is black and white stripes.・ The complexion is black.・ Brows are thick.

Hereinafter, a method for registering data in a digital album apparatus according to the present invention will be described with reference to the drawings. Hereinafter, reference numeral 10 indicates a digital album apparatus throughout the drawings.

FIG. 1 shows a RAM card method.
FIG. 4 is a diagram showing an example of registering data in the digital album device 10 using the card 1, which is a memory card (SRA) such as a PCMCIA standard card of a PC;
M, a flash memory, and a hard disk) 1. Film data is stored on a card 1 and the card is passed through the card and registered in a PC as a digital album apparatus.

Method by serial interface: FIG.
The digital camera 2 has a serial interface attached thereto, is connected to a PC as the digital album device 10 by a cable, and sends data photographed by the digital camera to the digital album device 10.

Method by Modem: FIG. 3 is a diagram showing data taken by the digital camera 2 using a modular jack (RJ4).
5) The digital album device 10 is sent via the public telephone line network 4, telephone 5, and modem 6. The telephone network may be an ISDN network.
Digital cameras require TA.

Method using cordless telephone function: FIG.
Transmits data captured by the digital camera 11 with a cordless telephone function to the base station 12, the ISDN network 13, the DS
Digital album device 10 via U14, TA15, etc.
It is intended to be sent to. The feature of this method is that the other data registration method stores data in the secondary memory, but this method does not store the data in the secondary memory but directly transfers the data from the main memory to the PC. The transfer is performed by a method such as ftp. The advantage is that the film memory is no longer full and you can no longer take pictures. Because it is a method of sending after shooting, it is possible to take any number of pictures.

FIG. 5 shows an APS film scanning method using an APS film scanning apparatus 20.
The data recorded on the film is sent to the digital album device 10 via a cable through the serial interface of the APS film scanning device 20.

FIG. 6 shows a method in which the APS film data is read using the APS automatic film reading device 30 and sent to the digital album device 10 after the scanning. Is discharged to a storage box or the like. Although the existing APS film scan can load only one film as shown in FIG. 5, the APS mass-film automatic reading apparatus 30 shown in FIG. Scan it in turn and send the data to the digital album device.

[0042]

As is apparent from the above description, according to the present invention, an image photographed by a digital still camera is automatically searched for the characteristics of the image, so that a search condition can be selected from a large number of images. Automatically search only matching images,
Since it can be extracted, even if a large number of films have been taken, they can be easily and quickly arranged.

[Brief description of the drawings]

FIG. 1 is a diagram showing an example of a case where data on the RAM is sent to a digital album device using a RAM card.

FIG. 2 is a diagram illustrating an example in which data of a digital camera is transmitted to a digital album device using a serial interface.

FIG. 3 is a diagram illustrating an example of a case where data of a digital camera is transmitted to a digital album device using a modem.

FIG. 4 is a diagram illustrating an example of a case where data of a digital camera is transmitted to a digital album device using a cordless telephone function.

FIG. 5 is a diagram illustrating an example in which film data is transmitted to a digital album apparatus using an APS film scanning apparatus.

FIG. 6 is a diagram illustrating an example of a case where data is transmitted to a digital album apparatus using an APS mass-film automatic reading apparatus.

[Explanation of symbols]

1. RAM card, 2. Digital camera, 6. Modem,
10: Digital album device, 11: Digital camera with cordless telephone function, 20: APS film scanning device, 30: APS automatic mass film reading device.

Claims (8)

[Claims] 1. An image photographed by using a digital still camera and an index simultaneously recorded on a film at the time of photographing are registered and searched, and a photograph meeting a search condition is displayed or recorded on a digital album device. An apparatus for creating a digital album using a digital still camera. 2. The digital album apparatus according to claim 1, wherein the digital album apparatus includes a face part detecting unit for detecting a face part from the image, a hair part detecting unit for detecting a hair part from the image, and a clothes part detecting unit for detecting a clothes part from the image. The digital album creating apparatus using a digital still camera according to claim 1, further comprising a human body part detecting unit configured to detect a human body part from the image. 3. A digital album apparatus, comprising: a face part detection unit for detecting a face part from the image; a face part detection unit for further detecting a face part from the part detected by the face part detection; The digital album creating apparatus using a digital still camera according to claim 2, further comprising a spectacles detecting unit for detecting presence / absence. 4. An index creation unit for creating an index from information obtained from a digital still camera, an image index creation unit for creating an image obtained by recognizing an image obtained from the digital still camera, and a set of the two indexes. Creation of a digital album using a digital still camera, which has a combination index creation unit that creates a new index in accordance with the combination, and efficiently searches for a desired photo from a huge number of photos. apparatus. 5. The digital still camera according to claim 4, wherein the combination index creation unit sets an index creation rule in advance, and creates a new index using the rule. Digital album creation device used. 6. A digital album apparatus, comprising: a face part detecting unit that detects a face part from the image; a hair part detecting unit that detects a hair part from the image; and a clothes part detecting unit that detects a clothes part from the image. The digital album creating apparatus using a digital still camera according to claim 4, further comprising a human body part detecting unit that detects a human body part from the image. 7. A digital album apparatus, comprising: a face part detection unit for detecting a face part from the image; a face part detection unit for further detecting a face part from the part detected by the face part detection; The digital album creating apparatus using a digital still camera according to claim 6, further comprising a spectacles detecting unit for detecting presence / absence. 8. A digital camera having an external communication function and a digital album apparatus having a function of receiving communication from outside, wherein an image and an index are transmitted from the digital camera to the digital album apparatus. A digital album apparatus characterized in that photographed information can be sent at any time.