JP2009171627A - Image sorting apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology suitable for a purpose of easily sorting out image data in an electronic camera or the like. <P>SOLUTION: An apparatus includes an identification unit, a recording destination selection unit and a sorted recording unit. The identification unit identifies a subject to be imaged by an imaging unit. In accordance with subject identification results of the identification unit, the recording destination selection unit selectively determines a destination to record image data output from the imaging unit. The sorted recording unit records the image data in the recording destination selected by the recording destination selection unit. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image classification apparatus and an image classification program for classifying and recording image data.

  The present invention relates to an electronic camera equipped with an image classification device.

  2. Description of the Related Art Conventionally, there are known multi-slot compatible electronic cameras that can simultaneously mount a plurality of memory cards.

  For example, Patent Document 1 discloses a technique for classifying image data according to the data size and recording the data on each memory card.

JP 2001-245248 A (Claim 1 etc.)

  By the way, it is conceivable that the image data captured by the electronic camera is classified for each acquaintance in the image, stored in a recording medium, and the recording medium is transferred to an acquaintance.

  Usually, such an image data classification operation is difficult to perform with an electronic camera alone, and is often performed on a computer. For this reason, it was not possible to easily complete the classification work of the image data on the spot where it was taken, and to immediately transfer the created recording medium.

  Further, even in the classification work on the computer, the user has to select the image data while visually checking the acquaintances reflected in the image data. Therefore, when the number of shooting frames is large, the operation is complicated and time-consuming.

  By the way, recently, an inexpensive memory card of a write-once type has been proposed. With this type of memory card, it is easy to photograph an acquaintance with an electronic camera and transfer the memory card to the acquaintance on the spot. However, since this memory card is write-once (recording only once), it is desirable to avoid recording extra image data other than acquaintances as much as possible. That is, it is required that the electronic camera alone selects the image data showing the acquaintance and records only the selected image data on the recording medium for transfer.

  Therefore, an object of the present invention is to provide a technique for making the usage mode such as classifying image data for each acquaintance more comfortable.

  The image classification device according to the first technique is an image classification device having a plurality of individual recording destinations, and includes an identification unit, a recording destination selection unit, and a classification recording unit.

  The identification unit identifies a subject to be imaged by the imaging unit.

  The recording destination selection unit selects and determines the recording destination of the image data output from the imaging unit in accordance with the identification result of the subject by the identification unit.

  The classification recording unit records the image data in the recording destination selected by the recording destination selection unit.

  The image classification device of the second technology is the image classification device of the first technology, and the identification unit has a function of identifying each of a plurality of human subjects from the shooting angle of view of the imaging unit. On the other hand, the recording destination selection unit divides a plurality of persons identified in this way into groups, and selects and determines the recording destination of image data for each group. The classification recording unit duplicates and records image data in accordance with the number of recording destinations to the recording destination selected by the recording destination selection unit.

  In the image classification device of the third technology, in the image classification device of the first technology or the second technology, the recording destination selection unit always has a recording destination and a classification recording destination as options for the recording destination.

  Image data is always recorded in this constant recording destination regardless of the identification result.

  On the other hand, the image data is selected or sorted according to the identification result and recorded in the classification recording destination.

  In the image classification device according to the fourth technique, in the image classification apparatuses according to the first technique to the third technique, when the identification unit detects the condition data for object comparison from the recording destination, the object is compared with the condition data. On the other hand, if the subject matches the condition data as a result of the collation, the recording destination selection unit selects the recording destination where the condition data is detected as the recording destination of the image data.

  The image classification device of the fifth technology is the image classification device of any one of the first technology to the fourth technology, and the identification unit includes a database and a new registration unit.

  The database holds condition data for subject verification.

  When the subject is identified as an unknown subject, the new registration unit creates condition data for the subject and newly registers it in the database.

  With this configuration, the subject can be identified for the second and subsequent times with respect to an unknown subject.

  The image classification device of the sixth technology is the image classification device of any one of the first technology to the fifth technology, and the identification unit includes a detection unit and an image specifying unit.

  The detection unit detects a region satisfying the condition of the subject image from the image data.

  The image specifying unit identifies the subject image detected by the detection unit and specifies the subject.

  The image classification program of the seventh technique is an image classification program for causing a computer to function as the identification unit, the recording destination selection unit, and the classification recording unit of any of the first to sixth techniques.

  An electronic camera according to an eighth technology includes the image classification device according to any one of the first technology to the sixth technology, and an imaging unit that provides image data to the image classification device.

  The electronic camera of the ninth technology is the electronic camera of the eighth technology, in which the recording destination selection unit has at least a recording medium that can be attached to and detached from the electronic camera as a recording destination option. When the condition data for subject collation is designated from the outside, the classification recording unit of the mounted image classification apparatus selects image data that matches the condition data and records it on this removable recording medium.

The electronic camera of the tenth technology is an electronic camera of the eighth technology or the ninth technology, and the recording destination selecting unit is a recording destination,
(1) a recording medium that is detachably attached to the electronic camera;
(2) Recording medium with built-in electronic camera,
(3) Select at least one of the group consisting of recording media capable of wirelessly or wiredly communicating with the electronic camera.

  In the image classification device of the first technique, the subject is identified, and the recording destination of the image data is classified according to the identification result.

  For example, the subject can be identified by detecting a face region from the image data and extracting the feature amount of the face. In this case, the recording destination of the image data is changed according to face identification.

  Further, for example, a clothing part can be detected from image data, and the subject can be identified from its color, pattern, or shape. In this case, the recording destination of the image data is changed according to the characteristics of the clothes.

  Further, for example, whether the subject is a landscape center or a person center may be identified from the luminance distribution and color characteristics such as image data and divided photometry results. In this case, the recording destination of the image data is changed according to the distinction such as a person scene / landscape scene.

  Further, for example, when the subject side has information submitting means for identification information such as a tag and a barcode, the identification information may be acquired from the subject at the time of imaging and subject identification may be performed. In this case, the recording destination of the image data is changed in accordance with accurate subject identification based on the identification information.

  As described above, the image classification apparatus identifies the subject and dynamically changes the recording destination of the image data, so that it is possible to greatly reduce the time and effort required for the user to sort and organize the image data.

  When there are a plurality of human subjects within the shooting angle of view, the image classification device according to the second technique performs identification for each human subject. Next, the apparatus divides the plurality of identified persons into groups, and selects and determines the image data recording destination for each group. At this time, if there are a plurality of recording destinations of image data, the image data is duplicated and recorded in accordance with the plurality of recording destinations.

  For example, by grouping a plurality of persons after identification into “Mr. A's family” and “Ms. B's family” registered in advance, it becomes possible to classify and organize the image data for each family unit. .

  Furthermore, if “One of Mr. A's family” and “One of Mr. B's family” are shown together in one piece of image data at this time, this one piece of image data is stored in the recording destination of both families. A duplicate is recorded.

  By having the grouping function in this way, the problem of excessively fine classification such as subject identification in individual units is solved, and image data can be classified into practical units that can be easily organized later. As a result, a practical and easy-to-use image classification device is realized.

  The image classification device according to the third technique has both a constant recording destination and a classification recording destination as options for the recording destination. In this constant recording destination, image data is always recorded regardless of the identification result. On the other hand, the classification recording destination selects and sorts the image data based on the identification result and records it.

  Accordingly, all image data is temporarily stored in the constant recording destination. On the other hand, image data selected or distributed by subject identification is copied to the classification recording destination.

  By properly using the recording destination, all the image data can be stored just in case. On the other hand, a transfer recording medium can be easily created by selecting image data of a specific subject and collecting them in different recording destinations.

  In the image classification device of the fourth technique, it is assumed that the condition data for subject collation exists at the recording destination. When this condition data is detected from the recording destination, the apparatus collates the subject with the condition data and records image data that matches the condition in the recording destination.

  For example, if Mr. A brings a memory card in which his / her family's condition data is stored, the image data of Mr. A's family can be collected and stored in the memory card without any trouble.

  In this case, it is only necessary to prepare a memory card that stores the condition data, so that it is not necessary to manually set the condition data for each electronic camera. In addition, by replacing the memory card with a plurality of electronic cameras, an interesting usage method is possible in which image data of Mr. A's family can be collected from the plurality of electronic cameras without trouble. .

  When an unknown subject is identified, the image classification device of the fifth technique automatically registers a new subject in the database, and uses this condition data from the next and subsequent subject identification. Therefore, the user can automatically identify an unknown subject from the next time onward, without registering the subject in the database.

  For example, in order to determine that the subject is an unknown subject for the first time, it is necessary to collate with all the condition data, which takes time. However, after the second time, since there is condition data for unknown subjects, it is not necessary to collate with all condition data. As a result, it is possible to rationally reduce the processing time of the unknown subject.

  The image classification device of the sixth technique identifies a subject by detecting a region of the subject image from the image data and identifying the subject image. In this case, if there is only image data for the time being, the subject identification is completed without requiring other information (such as divided photometric information).

  Therefore, even in a state where image data of different electronic cameras are mixed, it is possible to identify subjects uniformly with respect to these image data, and to classify and record image data for each subject. In other words, an image classification apparatus with higher versatility is realized.

  By executing the image classification program of the seventh technique on a computer, the computer can function as any one of the image classification devices of the first technique to the sixth technique.

  The electronic camera of the eighth technology is equipped with the image classification device of any one of the first technology to the sixth technology. Therefore, the above-described operation can be executed in the electronic camera.

  For example, by performing the above-described operation at the time of imaging, it is possible to dynamically change the recording destination in parallel with imaging. As a result, it is possible to use the recording medium after image classification is transferred immediately after imaging.

  Further, for example, a plurality of image data is temporarily recorded during imaging, and then the operation of the above-described image classification device is performed, whereby the image data can be classified and arranged as needed at a necessary timing.

  In the electronic camera of the ninth technology, a detachable recording medium is used as at least one recording destination. Therefore, if the transfer recording medium is mounted in advance on the electronic camera, the image data of the specific subject can be classified and recorded on the transfer recording medium. As a result, it is possible to take out the recording medium from the electronic camera and transfer it as it is after classification and recording.

  The electronic camera of the tenth technology has, as recording destinations, “a recording medium that is detachably attached to the electronic camera”, “a recording medium with a built-in electronic camera”, and “a recording medium that can communicate with the electronic camera wirelessly or by wire” Select at least one of the group consisting of Therefore, it is possible to classify and record image data captured by the electronic camera in a wider range and various recording destinations.

2 is a block diagram illustrating a configuration of an electronic camera 11. FIG. It is a flowchart explaining operation | movement of an image classification process. It is a flowchart explaining operation | movement of a subject registration process.

<< Description of Configuration of this Embodiment >>
FIG. 1 is a diagram showing an electronic camera 11 that supports multi-slots.

  In FIG. 1, a photographing lens 12 is attached to the electronic camera 11. In the image space of the photographic lens 12, the light receiving surface of the image sensor 13 is arranged.

  An image generated by the image sensor 13 is temporarily stored in the buffer memory 17 via the A / D converter 15 and the signal processor 16.

  The buffer memory 17 is connected to the bus 18. An image processing unit 19, a card interface 20, a microprocessor 22, a built-in nonvolatile memory 30, a communication interface 31, and a compression / decompression unit 23 are connected to the bus 18.

  Among these, a plurality of memory cards A, B,... Are detachably attached to the card interface 20. The microprocessor 22 is connected with an operation unit 22a and a divided photometry unit 22c.

  Further, the communication interface 31 communicates with an external image recording server 32 via a wireless LAN, a cable, or the like.

  The built-in nonvolatile memory 30 is a memory that holds data when the main power of the electronic camera 11 is turned off. For this reason, not only a so-called non-volatile memory, but a memory that holds data in a state where the main power is turned off by battery backup or the like may be used.

<Description of image classification processing>
FIG. 2 is a flowchart for explaining the operation of the image classification process.

This image classification processing may be performed every time immediately after or in parallel with the imaging operation of the electronic camera 11, or may be performed later on the image data stored in the recording medium. Hereinafter, the image classification process will be described along the step numbers shown in FIG.
[Step S1] The microprocessor 22 scans the image data to be classified, searches for facial features (color, shape, arrangement thereof, face image pattern, etc.), and detects a face region. When a plurality of persons are shown in the image data, the microprocessor 22 detects a plurality of face areas.
[Step S <b> 2] The microprocessor 22 detects a facial feature amount unique to the individual from the detected face area. For example, triangle shape data connecting both eyes and mouth, mandibular shape data, face contour shape data, position data such as moles, eye height in the face, shape data such as eyes, nose and mouth Color data, hairstyle data (hair length and color), and the like are detected.

The microprocessor 22 refers to the subject database in the built-in nonvolatile memory 30 for these facial feature amounts, and identifies a human subject having a high degree of condition matching. If a plurality of face areas are detected, this specifying operation is repeated by the microprocessor 22 to specify a plurality of persons in the image data.
[Step S3] If the condition data is not met and the person in the face area cannot be identified, the microprocessor 22 determines that the subject is an unknown subject, and proceeds to step S4.

On the other hand, when the identification of the person is completed for all the face regions, the microprocessor 22 shifts the operation to step S5.
[Step S4] The microprocessor 22 newly registers the feature quantity of the unknown subject in the subject database as condition data. This makes it possible to identify the second and subsequent unknown subjects and avoid double registration of unknown subjects. For an unknown subject registered in this way, a recording destination is always registered as a default for the time being as a recording destination. It is also possible to change the classification recording destination of the unknown subject later by user input.
[Step S5] If the group attribute exists in the subject database for the human subject in the image data, the microprocessor 22 moves the operation to step S6.

On the other hand, in other cases, the microprocessor 22 shifts the operation to step S7.
[Step S6] The microprocessor 22 groups a plurality of persons in the image data according to the group attribute. As a result, even for a large number of people, it is only necessary to select a recording destination after dividing it into rough groups, and the processing load for selecting the recording destination can be efficiently reduced.
[Step S7] The microprocessor 22 inquires of the subject database for the person subject (or group) specified in the image data in this way, and reads a pre-registered recording destination. At this time, a case where a plurality of human subjects (or groups) exist in one frame of image data and the recording destinations thereof are different is assumed. In such a case, a plurality of recording destinations are set for one frame of image data.
[Step S8] The microprocessor 22 determines whether or not a constant recording destination is preset in accordance with user settings.

  Here, when the recording destination always exists, the microprocessor 22 shifts the operation to step S9.

In other cases, the microprocessor 22 shifts the operation to step S10.
[Step S9] The microprocessor 22 always adds a recording destination to the recording destination of the image data.
[Step S10] The microprocessor 22 determines whether or not the recording capacity of the image data can be secured at the recording destination of the image data. For example, in the case of the memory cards A and B and the built-in nonvolatile memory 30, the determination can be made by managing the remaining capacity in the camera. When the image recording server 32 is the recording destination, the remaining capacity may be inquired to the image recording server 32 by communication.

  Here, if the recording destination has free space, the microprocessor 22 shifts the operation to step S12.

On the other hand, when the recording destination has insufficient free space, the microprocessor 22 shifts the operation to step S11.
[Step S11] The microprocessor 22 moves the recording destination for the recording destination that has insufficient free space, or temporarily records image data shortcuts (data indicating the location or link of the image data). Change to an opportunity.
[Step S12] The microprocessor 22 records the image data to be classified (see step S1) in the selected recording destination. Note that control may be performed so that the same image data is not recorded repeatedly for one recording destination.

  By the series of operations described above, automatic classification of image data by subject identification is realized.

<Subject registration process>
FIG. 3 is a flowchart for explaining the operation of subject registration processing.

  This subject registration process is performed as needed by the user operating the operation unit 22a between the imaging operations of the electronic camera 11. This is also carried out as needed when a memory card A or the like is newly installed in the electronic camera 11. Furthermore, it is also performed whenever the electronic camera 11 starts communication with the image recording server 32 via the communication interface 31.

Hereinafter, the subject registration process will be described according to the step numbers shown in FIG.
[Step S31] First, the microprocessor 22 searches for whether or not the condition data exists in the image data recording destination. For example, in the case of the memory cards A and B and the built-in nonvolatile memory 30, the file identifier defined in the condition data may be searched. If the image recording server 32 is the recording destination, the image recording server 32 may be queried for condition data.

  Here, when the condition data exists at the recording destination, the microprocessor 22 shifts the operation to step S32.

On the other hand, in other cases, the microprocessor 22 shifts the operation to step S33.
[Step S32] The microprocessor 22 registers the “recording destination where the condition data exists” and the “condition data existing at the recording destination” in association with each other in the subject database.

Thus, by storing the condition data in advance in the recording destination, the electronic camera 11 can automatically register the relationship between the condition data and the recording destination.
[Step S33] The microprocessor 22 determines image data for registration according to a user instruction. At this time, the user can select the following three types of user instruction methods by operating the operation unit 22a.
(A) The user captures new image data for registration.
(B) The user gives image data for registration to the electronic camera 11 from the outside via the recording medium or the communication interface 31.
(C) The user selects image data for registration from the image data group captured by the electronic camera 11.
[Step S34] The microprocessor 22 detects a face area from the registration image data. The microprocessor 22 analyzes the detected face area and extracts condition data.
[Step S35] The microprocessor 22 receives a recording destination setting input from the user via the operation unit 22a.

  If there is this setting input, the microprocessor 22 shifts the operation to step S37.

On the other hand, when there is no setting input, the microprocessor 22 shifts the operation to step S36.
[Step S36] Since there is no recording destination setting input, the microprocessor 22 always sets the recording destination as the default recording destination.
[Step S37] The microprocessor 22 associates the condition data with the recording destination and registers them in the subject database. Note that group attributes are also registered in the subject database when a setting is input.

  The subject registration process is completed by the series of operations described above.

<< Effects of this embodiment >>
As described above, in the present embodiment, the face of the subject is identified, and the recording destination of the image data is changed according to the person identification. Therefore, it is possible to change the recording destination for each person shown in the image data and classify the image data without trouble.

  In the present embodiment, a plurality of persons can be classified into groups, and a recording destination can be determined for each group. As a result, it is possible to classify and organize image data according to family units, circle units, regional units, and the like.

  Furthermore, in this embodiment, by setting a constant recording destination, it is possible to always record image data in this constant recording destination.

  In this embodiment, a removable recording medium such as a memory card B (especially a write-once type is preferable) can be set as the classification recording destination. In this case, the memory card B and the like can be transferred as they are to the subject.

  Furthermore, in this embodiment, by storing the condition data in the recording destination, the recording destination and the condition data can be automatically registered in the subject database without trouble.

  Further, in this embodiment, unknown subjects are automatically registered in the database, so that unknown subjects can be determined for the second and subsequent unknown subjects at the time of comparison with this new condition data. As a result, for the second and subsequent unknown subjects, it is not necessary to collate with all the condition data in the database, and wasteful time for collating the condition data can be shortened.

  By using an external device such as a computer to edit and create more complicated identification information, store the identification information in a memory card, and attach the memory card storing the identification information to the electronic camera. You can also organize more complicated files.

<< Additional items of embodiment >>
In the present embodiment, the case where the image classification function is installed in the electronic camera 11 has been described. However, the present invention is not limited to this. For example, the operations shown in FIG. 2 and FIG. 3 may be programmed and executed on a computer. In this case, the subject of the image data can be identified on the computer, and the image data can be classified and recorded without trouble according to the identification result.

  In this embodiment, a group attribute item is provided in the subject database. However, the present invention is not limited to this. For example, this group attribute item may be omitted by setting the same recording destination for the condition data of the same group.

  In the present embodiment, subject identification is performed using the face area of the image data. However, the present invention is not limited to this.

  For example, a clothing part of image data may be detected and subject identification may be performed based on the color, pattern, or shape.

  In addition, subject identification may be performed based on luminance distribution and color characteristics such as the divided photometry result of the electronic camera 11. In this case, since the subject identification can be started prior to the time of imaging, the recording destination determination timing can be advanced. As a result, a more light image classification operation can be realized.

  In order to further improve the subject identification accuracy, more complicated file organization may be performed by combining the face area of the image data, the clothing portion of the image data, the luminance distribution, the color characteristics, and the like.

  Further, when a person having a predetermined area or more is not detected by subject identification, it may be determined that the image data is centered on the landscape. In this case, it is possible to automatically classify the landscape image into a landscape recording destination.

  In addition, when the electronic camera can read information from a means for outputting identification information such as an IC tag or a barcode possessed by the subject, the subject identification is performed accurately and reliably by acquiring the identification information at the time of imaging. Also good.

  As described above, the present invention is a technique that can be used for an electronic camera or a program having an image classification function.

A, B Memory card 11 Electronic camera 12 Imaging lens 13 Image sensor 16 Signal processing unit 17 Buffer memory 18 Bus 19 Image processing unit 20 Card interface 22 Microprocessor 22c Split photometry unit 23 Compression / decompression unit 30 Built-in non-volatile memory 31 Communication interface 32 Image Recording server

Claims (1)

An image classification device having a plurality of individual recording destinations,
An identification unit for identifying a subject to be imaged by the imaging unit;
A recording destination selection unit that selects and determines a recording destination of image data output from the imaging unit in response to the identification result of the subject by the identification unit;
An image classification apparatus comprising: a classification recording unit that records the image data at the recording destination selected by the recording destination selection unit.

Images