JP2007288409A - Imaging apparatus with image data classifying function and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To resolve the problem that the time spent in search is increased to make operations complicated when searching only image scene of reminiscent scenes from numerous information though image data stored in a storage medium may be numerous by the increase in capacity of the recording medium. <P>SOLUTION: An imaging apparatus which converts an output of an imaging device 13 for converting incident light to an electric signal, to a digital value to obtain image data incudes: a feature point information processing part 17 for extracting feature point information included in the image data; an operation key switch 31 which inputs classification items for classification of the feature point information and classification conditions corresponding to the classification items; a CPU 21 for classifying the image data in accordance with the classification conditions; and a liquid crystal monitor 30 for displaying classified image data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to processing of image data obtained by an imaging device using a photoelectric conversion element.

  Image data captured by an imaging apparatus such as a digital camera is stored in an internal storage medium of the imaging apparatus or a storage medium inserted in the imaging apparatus. In recent years, the capacity of the storage medium has increased, and accordingly, the number of image data stored in the storage medium has become enormous.

  Conventionally, in order to find a target image from an enormous number of image data stored in a large-capacity storage medium, a list of image data is displayed on a personal computer (hereinafter referred to as a PC), or Print the image data of the photographed subject directly on the imaging device using a printing device such as a printer that has the function of printing, or output the image to a display device such as a liquid crystal monitor provided in the imaging device body. The user searches for the target image by switching the images in order by the operation member.

Further, index image data for search is generated using image data and reduced image data stored in an internal storage medium of the imaging apparatus or a storage medium inserted in the imaging apparatus, and image data search is performed using the index image data for search. Has been proposed (Patent Document 1).
JP 2000-175147 A

  However, although the ease of search is improved by generating the search image data, the act of the user selecting the target image remains unchanged.

  As a case where selection of image data by the user is required, for example, a case where an image taken when traveling by a plurality of people is distributed or printed can be considered. Images taken when traveling with multiple people include scenery, only those who have traveled, those that are different from those who have traveled, and only people who are different from those who have traveled It is conceivable that there are things that are reflected in the image, or that only the scenery is reflected and those in various situations. From the image data that contains these images, only the image data of the scene that remains in memory is printed, divided into folders for each person in the image, stored in a portable storage medium, or transferred In the conventional method in which the user searches for the target image data while checking the image displayed on the display device, the time spent by the user for the search operation is large when the total number of image data is large. Increases and the operation becomes complicated.

  In addition, if a long time has elapsed since the target image was taken, the search operation becomes more difficult. For example, when a plurality of trips are performed before and after the above trip, the memory where the target image is taken at which place during the trip, the memory becomes ambiguous over time, and the conventional In the search by this method, the time that the user spends for the search work is further increased, and the operation is further complicated.

  The present invention has been made in view of such problems, and when searching for image data, by using shooting conditions when shooting the image data and feature point information extracted from the image data. The main object is to provide an imaging apparatus having a function of classifying images and making it easy to search for a target image.

  It is another object of the present invention to provide a program that operates on a PC for executing the classification processing based on the use of the image capturing conditions and feature point information of the image data.

  In order to achieve the above object, an image pickup apparatus according to the present invention obtains image data by converting the output of an image pickup element that converts incident light into an electrical signal into a digital value. And image data is classified and displayed.

  Further, the image data shooting conditions and the feature point information may be stored in association with the image data as a comparison target with the classification conditions corresponding to the classification items. In addition to the feature point information extracted from the image data, by classifying in consideration of the shooting conditions when shooting the image data, it becomes possible to search for the target image even from more ambiguous memory.

  Further, the imaging apparatus of the present invention may include a printing unit that prints the classified image data and / or a transmission unit that transmits the classified image data to the outside. Further, a storage unit that stores the classified image data in a storage medium may be provided. As a result, the classified image data can be printed as it is, transmitted to the outside, or saved in a storage medium, so that the user's trouble can be reduced.

  In addition, as an index associated with the classified image data, the name of the image data, the logical position in the storage medium of the classified image data, or the physical position in the storage medium of the classified image data May be saved. By holding the classified image data as an index, the capacity of the storage medium can be saved.

  Further, in order to classify the classified image data more finely, classification may be repeatedly performed.

  The program of the present invention is a program for causing a computer to function as image data classification means for classifying image data according to shooting conditions and / or feature point information accompanying the image data. As a result, the PC can be used as the image data classification means, and the application of the present invention is greatly expanded.

  According to the present invention, an enormous number of image data stored in a large-capacity storage medium is automatically classified according to classification items without displaying each piece of image data on a display device or the like. Is possible.

  Furthermore, an enormous number of image data stored in a large-capacity storage medium can be automatically generated based on an ambiguous impression of target image data without displaying each image data on a display device or the like. It becomes possible to classify into.

  Because of these actions, when searching for target image data from a large number of image data stored in a large-capacity storage medium, not only the user's operation is simplified, but also the processing is automated. It also leads to a reduction in time.

  Embodiments of the present invention will be described below with reference to the drawings.

Example 1
FIG. 1 is a block diagram simply showing the configuration of a digital camera which is an image pickup apparatus of the present embodiment.

  Reference numeral 11 denotes an imaging lens for forming a subject image on the imaging element 13, and 12 denotes a stop. Reference numeral 13 denotes an image sensor such as a CCD that converts incident light into an electrical signal. An A / D converter 14 converts the output signal of the image sensor 13 into a digital value and obtains image data. An image processing unit 15 performs processing such as γ correction on the image data output from the A / D conversion unit 14. A compression processing unit 16 compresses the image data output from the image processing unit 15. Reference numeral 17 refers to the image data output from the compression processing unit 16 and extracts feature point information included in the image data, such as acquisition of image color distribution and face recognition processing to acquire personal identification information. It is a point information processing unit. The feature point information processing unit 17 may be included in the image processing unit 15. A memory 18 temporarily stores image data obtained by the compression processing unit 16 and feature point information obtained by the feature point information processing unit 17. The feature point information will be described later. Reference numeral 24 denotes a storage medium for storing feature point information and image data in the memory 18, which may be fixed to the imaging apparatus or removable. Reference numeral 19 denotes a drive unit that drives the image sensor 13 and the A / D converter 14. Reference numeral 22 denotes an aperture controller that controls and drives aperture blades that adjust the amount of light incident on the image sensor 13. A lens control unit 23 drives the imaging lens 11 to adjust the focal length. Reference numeral 32 denotes a release switch for instructing start of photographing. Reference numeral 20 denotes an overall control unit that controls operation of the entire digital camera. The overall control unit 20 includes a GPS (Global Positioning System) 25, an RTC (Real Time Clock) 26 for obtaining the current date and time, and a current temperature. A thermometer 27 for obtaining the current humidity, a hygrometer 28 for obtaining the current humidity, a microphone 29 for obtaining the volume of the surrounding sound, etc. are connected, and a CPU (Central Processing Unit) 21 in the overall control unit 20 is connected. Can be read out as part of the shooting conditions. Also connected to the overall control unit 20 are a liquid crystal monitor 30 for extracting and displaying classified image data, and an operation key switch 31 for inputting classification conditions and selecting start / stop of classification. Has been.

  The overall control unit 20 performs, for example, control according to the operation of the release switch 32, adjustment of the focal length and exposure amount, and control of writing and reading data to and from the memory 18 and the storage medium 24. The CPU 21 in the overall control unit 20 reads out and executes a predetermined program, thereby executing functions in this embodiment.

  The feature point information processing unit 17 has a face detection function. Further, it has a function of acquiring the luminance distribution or color distribution (histogram) of the designated area included in the image data.

  A flowchart from when the release switch 32 of the digital camera in this embodiment is pressed until image data is stored will be described with reference to FIG.

  When the user operates the release switch 32, the overall control unit 20 detects this and starts a photographing operation (step 101).

  The output of the image sensor 13 that has received the light flux from the subject that has passed through the imaging lens 11 and the aperture 12 is converted into digital signal image data (RAW format image data) by the A / D converter 14 (step 102). .

  The image processing unit 15 performs a blackening process, white balance gain value calculation, aperture correction, luminance γ correction, and the like on the output data of the A / D converter 14 to create a luminance signal Y, and simultaneously performs color interpolation and matrix conversion. Then, color γ correction is performed to create a color signal C, and a YC format signal is formed (step 103).

  The formed YC format image data is output to the compression processing unit 16 to generate JPEG format image data, which is one of the image compression formats, and stores it in the memory 18 (step 104). The data may be stored in the memory 18 in the RAW format without being compressed.

  In addition, the imaging conditions are acquired from the control parameters controlled by the GPS 25, the RTC 26, the thermometer 27, the hygrometer 28, the microphone 29, the overall control unit 20, and the like while creating the image data in the YC format and stored in the memory 18 ( Step 105).

  Further, the image data in YC format is referred to by the feature point information processing unit 17, and the feature points in the image are acquired by the above-described face detection function and the function of acquiring the luminance distribution or color distribution (histogram) of the designated area, Store in the memory 18 (step 106).

  The shooting conditions and feature point information stored in the memory 18 are created as header information of image data such as, for example, Exif (Exchangeable Image File Format) information (step 107), and this header information is added to the image data in JPEG format or the like. Then, it is stored in the storage medium 24 (step 108).

  Note that the acquisition of the feature point information is not necessarily performed in step 106. If the processing speed when classifying image data is sufficiently high, it is possible to classify images while extracting feature point information included in the image data.

  In the above flowchart, the feature point information is extracted from the YC format image data. However, as described above, the feature point information can also be applied to the RAW format image data obtained by digitizing the output of the image sensor 13. .

  Next, a flowchart until the image data stored in the storage medium 24 is classified by the above-described means will be described with reference to FIG.

  The user operates the operation key switch 31 while watching the guidance displayed on the liquid crystal monitor 30 to enter a mode for executing classification (step 201).

  Next, the user operates the operation key switch 31 while watching the guidance displayed on the liquid crystal monitor 30 to select the item to be classified (step 202), and the classification according to the classification item. When conditions are input (step 203), an image data search operation for image data classification is started (step 204). FIG. 4 is an example of guidance displayed on the liquid crystal monitor 30.

  If all the images have not been searched (step 205), the image data are sequentially read out (step 206), and the imaging conditions or feature point information associated with the image data are compared according to the classification conditions of the input classification items. (Step 207), it is automatically determined whether or not the classification condition is met (Step 208). If the classification condition is not met, the next image data is read and the same processing is performed, and the process returns to step 205. If the classification condition is met, the index of the image data is stored and the next data is read out, and the process returns to step 205. The image data index will be described later.

  When all the images have been searched (step 205), it is determined whether or not the index of the image data is empty (step 210). If it is empty, a message indicating that there is no image that matches the classification condition is displayed on the LCD monitor. 30 (step 212). If it is not empty, the list of classified image data is displayed on the liquid crystal monitor 30 with reference to the index of the image data (step 211).

  When the index of the image data is empty, when the classification item is selected again (step 213), the process returns to step 202. In the case of narrowing down classification, the process returns to step 202 in the same manner. However, the image data to be searched in steps 205 to 209 is only the image data already stored in the index.

  If not executed in step 213, the classification process is terminated (step 214). FIG. 5 is an example in which a list of classified image data is displayed. The classified image data can be arbitrarily selected by the user, and can be printed, transmitted to the outside, or stored in a storage medium.

  Next, the above-described image data index will be described with reference to FIG.

  In general, data stored in the storage medium 24 has a unique (unique) file name, and logical position information associated with the file name and physical position information associated with the logical position information. Managed based on location information. The image data stored in the storage medium 24 is also managed and stored in the storage medium 24 using a unique file name, logical position information, and physical position information.

  Image data (image 1 to image 5) stored in the image folder as shown in FIG. 6 is also managed by a unique file name, logical position information, and physical position information. If one can be known, it is possible to read out a specific piece of image data. The index stores any one of file name, logical position information, and physical position information. By holding the classified image data as an index, the capacity of the storage medium 24 can be saved. . It is also possible to store the index in the memory 18 instead of the storage medium 24.

  As an example in which no index is used, image data classified as shown in FIG. 7 can be copied and stored in another folder. In this case, the capacity of the storage medium 24 is compressed, but when it is intended to distribute image data, it is not necessary to select the image data while referring to the index, and the user's operation is simplified. Can do.

  The image data capturing conditions and feature point information extraction, the operation of classifying image data using the image data capturing conditions and feature point information, and the method of storing the classified data have been described above. Further, the feature point information extraction described above may be performed on image data already stored in the storage medium 24 or image data transferred from an external electronic device and stored in the memory 18 or the storage medium 24. Is possible.

(Example 2)
The present embodiment is characterized in that image data is classified on an image processing apparatus or an information processing apparatus such as a PC using feature point information of image data in JPEG format or RAW format.

  For example, feature point information of image data that has not been classified is read out, and based on this, it is assigned to a plurality of arbitrary folders created by the user, or image data is duplicated.

  A method for classifying image data using color distribution for each area as feature point information will be described with reference to FIGS.

  When it is desired to search for image data as shown in FIG. 8, for example, if a long time has passed since this image data was shot, the user may have forgotten the shooting time or shooting location. However, even in that case, if only the impression that the image data “the sky is blue and the green of the trees is beautiful” remains, the area is divided into upper and lower areas as shown in FIG. Only can be classified. In FIG. 9, “region: upper and lower” in the upper left indicates that the image is divided into upper and lower parts. From the palette in which the color names are described in the left nine divided regions, “up” and “ Select the color corresponding to “Bottom”.

  However, if the image data of FIG. 8 shown in this example is classified based on a strict color distribution for each region, the classification condition may not be met. In the case of the image data in FIG. 8, for example, the color data may be saturated by sunlight, and the upper color may be white most, and the lower part is most brown, which is the color of soil or wood, rather than green. It may be. Therefore, the color distribution in the region has the top two or third colors as feature point information.

  In addition, by assigning a certain range to the hue selected as the classification condition, it is possible to accurately specify a color close to an impression.

  In addition, by dividing the area up and down, up and down and center, left and right, left and right and center, 4 divisions and 4 divisions and centering, an ambiguous impression can be accurately specified as a classification condition without complicated region division. it can. When taking an image with an imaging device such as a digital camera, it is often the case that the subject is placed near the center, so it is possible to specify a classification condition that is close to the impression by acquiring the color distribution at the center separately. it can. For example, when the region is divided into four divisions and the center, the image data as shown in FIG.

  FIG. 11 shows an area where a color distribution is acquired as feature point information. By extracting the upper 2nd or 3rd color for each region from region 1 to region 5 as feature point information, for example, the number of upper colors is the sum of region 1 and region 2, the left color When the number is the sum of region 1 and region 3, the color distribution of all divided regions can be obtained.

  The method for classifying image data using the color distribution as feature point information and using it as a classification condition has been described above.

(Example 3)
The present embodiment is characterized in that the user can easily and easily specify the classification items and the classification conditions using the shooting conditions and the feature point information of the image data in JPEG format or RAW format.

  For users of imaging devices such as digital cameras, the meaning of exposure time, lens F value, ISO sensitivity, exposure correction amount, strobe emission amount, lens focal length, image compression rate, γ correction amount, and image data quality There are cases where the relationship with the impact on In addition, if a long time has passed since the image data to be searched is captured, the user may have forgotten the imaging conditions at the time of imaging. Therefore, it is preferable that the classification items and the classification conditions for classifying easily represent the stored impressions.

  FIG. 12 is a diagram showing the relationship between the shooting conditions, feature point information, and items that are actually specified as classification items. Vertical items are shooting conditions and feature point information, and horizontal items are classification items. For example, the classification item (experience temperature) is determined based on the date, time, temperature, humidity, and position information if necessary. When the image to be retrieved is photographed, if there is an impression that it is cold, the image data that feels cold, for example, the temperature is 10 ° C. or less is retrieved. If there is an impression that it is warm, the date, temperature, and humidity are referred to, and image data having a temperature higher than the average temperature of the average year from late autumn to early spring is searched. From early summer to early fall, it's hard to have a psychologically warm impression, so you can exclude it from search.

  FIG. 13 is a tree diagram illustrating an example of the menu configuration of the classification items and the classification conditions when the user can easily express the classification items and the classification conditions based on FIG.

  As described above, the method for classifying image data using a plurality of related photographing conditions and feature point information by designating impressions on image data that the user wants to search as classification items and classification conditions has been described.

  Although the invention made by the present inventor has been described based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention.

  In the above embodiment, the storage means and comparison information storage means of the present invention correspond to the storage medium 24. The feature point information extraction unit of the present invention corresponds to the feature point information processing unit 17. The classification item input means and the classification condition input means of the present invention correspond to the operation key switch 31. The image data classification means, reading means, saving means, and index saving means of the present invention are realized by the CPU 21 in the overall control unit 20 reading and executing a predetermined program. The display means of the present invention corresponds to the liquid crystal monitor 30. The photographing condition extracting means of the present invention is performed simultaneously with the feature point information extraction in the feature point information processing unit 17.

  Furthermore, the imaging apparatus of the present invention further includes a printing unit that prints the image data classified by the image data classification unit and / or a transmission unit that transmits the image data classified by the image data classification unit to the outside. Also good.

  According to the present invention, an enormous number of image data stored in a large-capacity storage medium is automatically classified according to classification items without displaying each piece of image data on a display device or the like. Therefore, the present invention is useful as an image search program realized by a digital camera, a video camera with a still image shooting function, a device having a still image shooting function such as a mobile phone with a still image shooting function, and a PC.

Block diagram showing the configuration of the imaging device Flowchart from when the release switch in the imaging device is pressed until image data is stored Flow chart until selecting classification execution mode and finishing classification Diagram showing guidance for specifying classification items The figure which shows the example which displayed the list of classified image data Diagram showing the relationship between image data and indexes Diagram showing the relationship between image data and classified image data Diagram showing an example of image data to be searched The figure which shows the example of the screen which inputs the conditions which classify the image data used as search object Diagram showing an example of image data to be searched The figure which shows the area where color distribution is extracted Diagram showing the relationship between abstract category items, shooting conditions, and feature point information Diagram showing an example of the configuration of classification items

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Image pick-up lens 12 Aperture 13 Image pick-up element 14 A / D converter 15 Image processing part 16 Compression processing part 17 Feature point information processing part 18 Memory 19 Drive part 20 Overall control part 21 CPU
22 Aperture control unit 23 Lens control unit 24 Storage medium 25 GPS
26 RTC
27 Thermometer 28 Hygrometer 29 Microphone 30 LCD Monitor 31 Operation Key Switch 32 Release Switch

Claims (12)

In an imaging device that obtains image data by converting the output of an imaging device that converts incident light into an electrical signal into a digital value,
Storage means for storing the image data in a storage medium;
Feature point information extracting means for extracting feature point information included in the image data;
Classification item input means for inputting a classification item for classifying the feature point information;
Classification condition input means for inputting a classification condition according to the classification item;
Image data classification means for classifying the image data stored in the storage medium according to the classification condition;
Reading means for reading out the classified image data from the storage medium;
Display means for displaying the read image data;
An imaging apparatus comprising: A photographing condition extracting means for extracting photographing conditions when photographing the image data;
The storage means stores the shooting conditions and the feature point information in the storage medium in association with the image data,
The image data classification means classifies the image data according to the shooting conditions and / or the feature point information attached to the image data.
The imaging apparatus according to claim 1. Printing means for printing the image data classified by the image data classification means and / or
Transmitting means for transmitting the image data classified by the image data classifying means to the outside;
The imaging device according to claim 1, further comprising: The imaging apparatus according to claim 1, further comprising a storage unit that stores the image data classified by the image data classification unit in a storage medium. Index storage means for storing an index associated with the image data classified by the image data classification means in a storage medium;
The imaging apparatus according to claim 1, further comprising: The image data classification means performs repeated narrowing classification to further classify the image data classified by the image data classification means.
The imaging apparatus according to claim 1, wherein: The shooting conditions include date, time, exposure time, lens F value, ISO sensitivity, exposure correction amount, strobe emission amount, lens focal length, zoom magnification, image size, image compression ratio, image top-to-bottom direction, γ Including at least one of the correction amounts,
The imaging apparatus according to claim 1, wherein: The shooting conditions include at least one of a location at the time of shooting, temperature, humidity, and loudness.
The imaging apparatus according to claim 1, wherein: The feature point information is the luminance distribution and / or the color distribution for each of the entire image data or each area obtained by dividing the image data.
The imaging apparatus according to claim 1, wherein: The feature point information is combination information of clothing colors and / or patterns for specifying a person included in the image data.
The imaging apparatus according to claim 1, wherein: The feature point information is information including a face shape and / or coordinate position for specifying a human face included in the image data.
The imaging apparatus according to claim 1, wherein: Computer
As the image data classification means for classifying the image data according to the shooting conditions and / or the feature point information attached to the image data,
A program to make it work.

Images