JP4867428B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program suitable for a digital camera or the like having a scene program shooting function in which shooting conditions are specified in advance.

  Conventionally, photograph and photographed image data has been an important information source for controlling camera photographing processing and image signal processing. For this reason, camera makers that have a long history design camera control units so that optimal shooting conditions can be set for each shooting scene based on a huge database of photos taken by various photographers and shooting frequency data. Subject brightness and subject distance frequency data are built into the camera and a shooting scene estimation function is provided, or the most frequent subject distance is estimated from subject brightness, and the optimal focus position, exposure conditions, In some cases, the camera was designed to perform shooting processing such as balance.

  However, instead of cameras using silver halide films (hereinafter referred to as “silver halide cameras”), digital cameras capable of recording and storing image information in the form of electronic data have become mainstream. Unlike a photograph taken with a salt camera, many factors such as camera user characteristics, shooting purpose, shooting target, shooting conditions, and usage frequency are different.

  In addition, the preferences and desirable image quality of digital camera users, who are consumers, change with the passage of time, so even if there is historical data of photos from the silver salt camera era, shooting with digital cameras and new camera users It is necessary to keep collecting and updating information according to the tastes of people.

  On the other hand, with the spread of personal computers, mobile phones with camera functions, the Internet, etc., it is technically possible to upload personal captured images and information from cameras and collect them in a specific server device. It was.

  However, there are problems that need to be solved, such as personal information and information related to privacy. When photographing information including personal information is collected, it is necessary to strictly manage the system and leakage prevention measures.

  In addition, if only the images owned by the camera maker or the images transferred from the user are used for the database, the number of samples is small, the reliability of the statistical data decreases, or the camera users have a new trend. This will cause a problem that it cannot be covered.

  As a means to efficiently search many of these collected images, add "features" of the image content, "bookmarks" pointing to specific scenes, etc., and make these directly symmetric to search for content There is a way to change it.

  Various metadata description standards that define such metadata description methods are also used for general-purpose applications such as Dublin Core and MPEG7 (ISO / IEC 15938, Multimedia Content Description Interface), and content such as news and TV broadcasting such as SMPTE and EBU. Various standardizations such as P / Meta, TV-Anytime, XMLNews, and NewsML are in progress.

In the patent document, for example, shooting information such as subject distance at the time of shooting, lens focal length, strobe light source information and film processing information, and information necessary to correct an underexposed image is recorded on a removable recording medium. The technology of a “camera that records information in a device” is described. (Patent Document 1)
MPEG7 is different from the standard for compressing and encoding moving image data such as MPEG1, 2, and 4, and is a standard for metadata notation for multimedia content. Content title, policy related information, content content There are provided description methods ranging from a description tool close to a human recognition level to a multi-level feature description tool automatically obtained by signal processing, such as the color, shape, pattern, and movement of a visual object of video content.

Thus, unlike other metadata description standards for specific uses such as SMPTE, EBU P / Meta, and TV-Anytime, MPEG7 does not stipulate applications, handling forms, and content forms. Can be used for various purposes and content.
Japanese Patent Laid-Open No. 02-306228

  However, when a general user takes a picture with a digital camera, it is very complicated to use the MPEG7 technique to describe feature data one by one on a personal computer or to perform feature extraction. It's not realistic.

  Many common digital cameras can record shooting information such as shooting date / time, image size, and exposure conditions together with image data in an exchangeable image file (Exif) header. It was few, and it was not effective to search by image contents from a large number of images.

  The present invention has been made in view of the above circumstances, and its purpose is to effectively use the contents of image data obtained by photographing, such as searching for a desired image from the contents of the photographed image. It is an object to provide an imaging apparatus, a method, and a program capable of performing the above.

  According to the first aspect of the present invention, there is provided a setting means for setting photographing conditions, a photographing means for performing photographing according to the photographing conditions set by the setting means, and a predetermined viewpoint from among image information obtained by photographing with the photographing means. Based on the feature extraction means for extracting feature information indicating the feature of the image, and storage means for storing the shooting conditions set by the setting means and the feature information obtained by the feature extraction means in association with each other as shooting history information A plurality of shooting history information stored in the storage means, and using the counting results, a shooting condition for newly shooting is determined and set by the setting means, and the shooting means And a photographing control means for performing photographing according to the above.

  According to a second aspect of the present invention, in the first aspect of the invention, a correlation between various shooting conditions and various feature information is extracted by aggregating a plurality of shooting history information stored in the storage unit. A correlation extracting unit that stores correlation information; and the imaging control unit acquires feature information corresponding to an imaging target to be newly captured, and sets the imaging condition having a correlation with the acquired feature information. It is specified based on the stored correlation information, and the specified shooting condition is set by the setting unit, and shooting by the shooting unit is performed.

According to a third aspect of the present invention, there is provided a setting means for setting shooting conditions, a shooting means for executing shooting according to the shooting conditions set by the setting means, and a predetermined viewpoint from among image information obtained by shooting with the shooting means. Based on the feature extraction means for extracting the feature information indicating the feature of the image, the shooting condition corresponding to the image information obtained by the shooting means, and the feature obtained by the feature extraction means corresponding to the image information Various shooting conditions obtained by counting the shooting history information from a plurality of imaging devices by the transmission means for transmitting shooting history information associated with the information to the counting external device, and the counting external device And receiving means for receiving correlation information indicating the correlation between various feature information from the external device for tabulation and feature information corresponding to a subject to be newly photographed, and obtaining the acquired feature A shooting control unit that specifies shooting conditions having a high correlation with the information on the basis of the received correlation information, sets the specified shooting conditions by the setting unit, and performs shooting by the shooting unit. It is characterized by that.

The invention described in claim 4 further includes input means for inputting designation information indicating a predetermined viewpoint given from the outside of the apparatus, and the feature extraction means is based on the designation information input by the input means. It is characterized by extracting feature information indicating image features.

According to a fifth aspect of the present invention, the input means also inputs shooting condition information in addition to the designation information, and the setting means sets shooting conditions based on the information input from the input means. It is characterized by.

According to a sixth aspect of the present invention, the transmission means further outputs classification information for classifying the imaging device, and the input means provides designation information or photographing corresponding to the classification information given from outside the device. It is characterized by inputting condition information.

According to the seventh aspect of the present invention, there is provided a setting step for setting shooting conditions, a shooting step for performing shooting according to the shooting conditions set in the setting step, and a predetermined viewpoint from image information obtained by shooting in the shooting step. A feature extracting step for extracting feature information indicating the feature of the image based on the image, and a storing step for storing the shooting conditions set in the setting step and the feature information obtained in the feature extracting step in association with each other as shooting history information And a plurality of shooting history information stored in the storage step, and using the counting result, a shooting condition for newly shooting is determined and set by the setting step, and the shooting step And a photographing control step for performing photographing according to the above.

The invention described in claim 8 includes a setting step for setting shooting conditions, a shooting step for performing shooting according to the shooting conditions set in the setting step, and a predetermined viewpoint from among image information obtained by shooting in the shooting step. A feature extraction step for extracting feature information indicating the feature of the image based on the image, the shooting condition corresponding to the image information obtained in the shooting step, and the feature obtained in the feature extraction step corresponding to the image information Various shooting conditions obtained by collecting shooting history information from a plurality of imaging devices by the sending step for sending shooting history information associated with information to the counting external device, and the counting external device And receiving the correlation information indicating the correlation between the feature information and the various feature information from the external device for counting, and acquiring the feature information corresponding to the imaging target to be newly captured, and the acquired feature It was identified based on the high imaging condition correlated with broadcast the correlation information thus received, causes set by the specified photographing condition said setting step, and a photographing control step of performing photographing by the photographing step It is characterized by.

The invention according to claim 9 is a program executed by a computer built in the imaging apparatus, a setting step for setting shooting conditions, a shooting step for executing shooting according to the shooting conditions set in the setting step, Obtained in the feature extraction step of extracting feature information indicating the feature of the image based on a predetermined viewpoint from the image information obtained by photographing in the photographing step, the photographing condition set in the setting step, and the feature extraction step A storage step for associating and storing characteristic information with shooting history information, and a plurality of shooting history information stored in the storage step, and shooting conditions for newly taking a picture using the counting result And a shooting control step for performing shooting in the shooting step. Characterized in that to execute the computer.

The invention according to claim 10 is a program executed by a computer built in the imaging apparatus, a setting step for setting shooting conditions, a shooting step for performing shooting according to the shooting conditions set in the setting step, A feature extraction step for extracting feature information indicating the feature of the image based on a predetermined viewpoint from the image information obtained by photographing in the photographing step; and the photographing condition corresponding to the image information obtained in the photographing step; A transmission step of transmitting shooting history information in association with the feature information obtained in the feature extraction step corresponding to the image information to an external device for counting, and a plurality of imaging devices by the external device for counting Correlation information indicating the correlation between various shooting conditions and various feature information obtained by counting shooting history information is obtained from the external device for counting. The reception step for receiving and the feature information corresponding to the subject to be newly photographed are acquired, and the imaging condition having a high correlation with the acquired feature information is specified based on the received correlation information. The photographing condition is set by the setting step, and the computer is caused to execute a photographing control step for photographing by the photographing means .

According to the present invention, it is possible not only to effectively use the content of image data obtained by photographing, such as searching for a desired image from the content of the photographed image, but also from the image obtained by photographing. By collecting the feature extraction information and acquiring the feature information, the feature extraction information can be effectively used for setting the shooting conditions at the time of new shooting.

  Hereinafter, an embodiment in which the present invention is applied to a compact digital camera will be described with reference to the drawings.

  FIG. 1 mainly shows an external configuration of a digital camera 10 according to the present invention. FIG. 1 (A) is a front view, FIG. 1 (B) is a left side view, and FIG. 1 (C) is a rear view. .

  A photographing optical system light receiving window 11 is disposed at the front and upper left end of the digital camera 10, a strobe light emitting unit 12 is disposed at substantially the upper center of the front, and a grip unit 13 is disposed along the left end of the front.

  A power switch 14 and a shutter switch 15 are disposed on the upper surface and right end side of the digital camera 10.

  FIG. 1B shows the configuration of the photographing optical system inside the digital camera 10. The digital camera 10 employs a refractive optical system as a photographing system, and an optical image incident from the photographing optical system light receiving window 11 is reflected by a mirror (or prism) 16 at 45 ° and is enclosed by the housing of the digital camera 10. After the light beam is made parallel to the body, the light is imaged on the CCD 18 which is an image sensor after passing through the photographing lens group 17.

  The photographing lens group 17 employs an inner focus method and an inner zoom method, and the zoom angle and the focus position are changed by appropriately moving a predetermined lens along the photographing optical axis.

  Further, on the back of the digital camera 10, a mode switch 19 for switching between a photographing mode as a basic mode and a playback mode, a zoom switch 20, a ring-shaped cursor key 21, and an enter key 22 located at the center of the cursor key 21. Various operation keys such as a display switching key 23 and a menu key 24 are arranged, and a display unit 25 made of a color liquid crystal panel with a backlight is arranged.

  The display unit 25 functions as an electronic viewfinder for displaying the image formed by the CCD 18 in the photographing mode together with information on various other photographing conditions while displaying the selected image in the reproduction mode. It functions as a part.

  The housing bottom of the digital camera 10 is provided with a storage lid 26 for storing a memory card as a recording medium and a lithium ion rechargeable battery pack as a power source, and an external input / output terminal 27 by a USB connector, for example. Arranged.

  FIG. 2 illustrates an external configuration of another digital camera 10 ′ according to the present embodiment, which can replace the digital camera 10. FIG. 2A is a diagram showing a rear configuration from the left rear, and FIG. 2B is a diagram showing various medium storage configurations on the rear side.

  A digital camera 10 ′ having a photographing lens unit 31 and a microphone 32 on the front side includes a display unit 33 formed of a color liquid crystal panel with a backlight on the left side. The display unit 33 is capable of being stored with the screen facing the inner surface of the digital camera 10 ′ when not in use when the base part can be rotated within a predetermined angle range along two axes orthogonal to each other. It is possible to turn the screen in an arbitrary direction as shown in FIG.

  Similarly to the display unit 33 of the digital camera 10, the display unit 33 also functions as an electronic viewfinder in the shooting mode and as a monitor display unit in the playback mode.

  From the upper side of the digital camera 10 ′, a speaker 34, a zoom switch 35, a shooting switch 36, a ring-shaped cursor switch 37, a playback switch 38 positioned at the center of the cursor switch 37, and a power / mode switch 39 External input / output terminals 40 and 41 are provided.

  Among these, the switches 35 to 39 excluding the speaker 34 and the external input / output terminals 40 and 41 are formed on the outer surface of the opening / closing lid 42 that forms the housing rear side of the digital camera 10 ′. As shown in FIG. 2B, the lid 42 can be opened and closed with respect to the main body of the digital camera 10 ′. With the open / close lid 42 opened, the hard disk unit 43 and the memory card 44, which are both recording media, can be attached and detached at predetermined positions as shown in the figure.

  FIG. 3 shows a conceptual system configuration of the digital camera 10. In the figure, a key operation signal at an operation input unit 51 including the power switch 14, shutter switch 15, mode switch 19, zoom switch 20, cursor key 21, enter key 22, display switch key 23, and menu key 24. Is provided to the photographing / recording control unit 52, the photographing condition setting unit 53, and the feature extraction / description setting unit 54.

  The shooting / recording control unit 52 controls shooting conditions, operation timing, and the like for the shooting unit 55, the signal processing unit 56, the image data memory 57, and the feature extraction unit 58, and shooting and images obtained along with the shooting. Control data recording.

  The photographing unit 55 includes the photographing lens group 17 and the CCD 18, obtains digital value image data obtained by photographing, and outputs it to the signal processing unit 56.

  The signal processing unit 56 performs predetermined process processing or the like on the image data supplied from the imaging unit 55 and outputs the image data to the image data memory 57 that functions as a buffer memory.

  The image data held in the image data memory 57 is read to the feature extraction unit 58 and the image / moving image compression / encoding unit 59 as necessary.

  For example, if the image data is a still image, the image / moving image compression / encoding unit 59 performs data compression and encoding according to the JPEG standard, and if the image data is a moving image, the image data as the obtained encoded data. Are output to the multiplexing unit 60 and the feature extraction unit 58.

  The feature extraction / description setting unit 54 sets elements for performing feature extraction and a description method thereof according to the key operation signal from the operation input unit 51 and the shooting conditions given from the shooting condition setting unit 53, and extracts the features. To the unit 58 and the feature data encoding unit 61.

  Based on the feature information set by the feature extraction / description setting unit 54, the feature extraction unit 58 performs feature extraction processing on the image data subjected to data compression output from the image / moving image compression / encoding unit 59. To obtain “feature extraction A” as a processing result, while performing feature extraction processing on the image data before data compression output from the image data memory 57 to obtain “feature extraction B” as a processing result. The feature extraction process is executed according to the number of feature information set from the feature extraction / description setting unit 54, and the processing results are output to the feature data encoding unit 61 and the comparison circuit 62. .

  The feature data encoding unit 61 encodes the feature amount data extracted by the feature extraction unit 58 into feature description data, and outputs the feature description data to the multiplexing unit 60 and the update recording control unit 63.

The multiplexing unit 60 multiplexes and associates the image data provided from the image / video compression / encoding unit 59 and the feature description data provided from the feature data encoding unit 61, and outputs them to the memory control unit 64. To do.

  The memory control unit 64 records the image data and the feature description data in the image data memory 65 that is a recording medium of the digital camera 10 in association with the maximum capacity.

The image data and feature description data stored in the image data memory 65 are read by an external input / output unit 66 to an external device of the digital camera 10, for example, a personal computer (not shown) if necessary.

  The update recording control unit 63 uses the feature description data from the feature data encoding unit 61 to update and set the contents of the statistical storage unit 67 that already stores the shooting scene and shooting conditions based on the feature description data.

  The stored contents of the statistical storage unit 67 are read out as appropriate to the comparison circuit 62 and the photographing condition setting unit 53.

  The comparison circuit 62 compares the feature amount data extracted from the image data (obtained by photographing) given from the feature extraction unit 58 with the feature amount data stored in the statistical storage unit 67, and the comparison result is described above. The image is output to the shooting condition setting unit 53.

  The contents stored in the image data memory 65 and the statistics storage section 67 are appropriately rewritten according to the contents input from the outside by the external input / output section 66.

  However, the contents of the rewritten image data memory 65 are taken out by the memory control unit 64 that has received an instruction from the reproduction control unit 68, separated into image data and feature description data by the demultiplexing unit 69, and image data Are sent to the image / moving image decompression / decoding unit 70 and the feature description data are sent to the feature data combination unit 71, respectively.

  The image / moving image decompressing / decoding unit 70 decompresses and decodes the image data that has been compressed and encoded in the same procedure as the data compression in the image / moving image compression / encoding unit 59. And output to the reproduction control unit 68.

  The reproduction control unit 68 temporarily stores the sent image data in the image data memory 57, reads it, and outputs it to the display control unit 72.

  Under the control of the reproduction control unit 68, the display control unit 72 creates display data from the image data read from the image data memory 57, and drives the display unit 73 to display the image. The display unit 73 includes the display unit 25 and its drive circuit.

Next, FIG. 4 illustrates a specific circuit configuration of the digital camera 10 together with connections to external devices. This figure is slightly different from the conceptual system configuration of FIG.

  In the drawing, a part of the photographing lens group 17 is driven by a lens driver 80 to move a position in the optical axis, and a photographing light image is formed on the CCD 18. The image signal obtained by the CCD 18 is sent to the video signal processing unit 81, digitized, and sent to the image memory 82 and the control circuit 83 through various process processes.

  Also, during voice recording and moving image shooting, a voice signal input from the microphone 84 is amplified by the amplifier 85, digitized by the A / D conversion circuit 86, and held in the voice memory 87.

  The control circuit 83 reads out the image data held in the image memory 82 and the audio data held in the audio memory 87, performs predetermined data compression processing by compression encoding / decompression decoding 88, and then attaches / detaches the memory card or the like. It is recorded on a free recording medium 89.

  On the other hand, the image memory 82 also sends the image data read from the image memory 82 and the sound data read from the sound memory 87 to the feature extraction / recognition processing unit 90.

  The feature extraction / recognition processing unit 90 performs feature extraction of image data and audio data based on the content stored in the scene / shooting condition statistical data memory 91, and compresses / decompresses the extracted data file. The data is sent to the decryption 88 and recorded on the recording medium 89 together with the original image data and audio data.

  The contents of the scene / photographing condition statistical data memory 91 are appropriately updated by the control circuit 83.

  The control circuit 83 is based on the operation program fixedly stored in the program memory 92, and performs the recording / reproducing operation of the image and sound while writing / reading necessary data to / from the data memory 93 as appropriate. In addition to the imaging control unit 94, the display control unit 95, the input circuit 96, the external input / output interface 97, the hard disk interface (HDD / IF) 98, and the D / A conversion circuit 99. Connected.

  The photographing control unit 94 is based on the distance measurement data output by the photometry / ranging unit 101 from the sensor output obtained by the distance measuring sensor 100 and the control signal from the control circuit 83, and the lens driver 80, the CCD 18, and the video signal. The photographing operation in the processing unit 81 is controlled.

  The display control unit 95 drives the display unit 25 based on the display data supplied from the control circuit 83, thereby causing the display unit 25 to function as an electronic viewfinder during photographing and as a display monitor during reproduction.

  The input circuit 96 receives an operation signal from the operation input unit 104 including various key switches such as the power switch and the shutter switch and sends the operation signal to the control circuit 83.

The external input / output interface 97 transmits and receives data according to various standards between the external input / output terminal 27 and the control circuit 83.

  The hard disk interface 98 mediates between the control circuit 83 when the hard disk unit 43 is mounted as well as a memory card as a recording medium, like the digital camera 10 'shown in FIG. The mounted hard disk unit 43 functions as an enlargement of the capacity of the recording medium 89.

  The D / A conversion circuit 99 converts the digital L audio data supplied from the control circuit 83 into analog during reproduction of the audio data. The output audio signal is appropriately amplified by the amplifier 102 and then the back side of the digital camera 10. A loudspeaker is emitted by the loudspeaker 103 arranged in the position.

For example, a personal computer (PC) 110 as an external device is line-connected to the external input / output terminal 27 as shown in the figure.

  The personal computer 110 automatically installs the driver software for the digital camera 10 in advance, and automatically connects image / audio data and feature description data recorded on the recording medium 89 when connected to the digital camera 10. In addition, the contents stored in the statistical data memory 91 of the above scene / shooting conditions can be appropriately updated and set.

  That is, the personal computer 110 is connected to, for example, a server device 112 operated by the manufacturer of the digital camera 10 via the Internet 111 via a gateway (GW), and upgrade firmware for the digital camera 10 or scene / photographing. Various data such as statistical data of conditions are downloaded, and the digital camera 10 is updated and set.

  In the server device 112, image data and feature extraction data sent from an unspecified number of personal computers registered in advance as user members via the Internet 111 are stored in an image database (DB) 113, and the model of the digital camera, etc. Statistical data based on various classifications is calculated and stored, and the contents are distributed to a personal computer owned by the user member.

  Next, the operation of the above embodiment will be described.

  5 to 7 show general processing contents after the power is turned on in the digital camera 10. Initially, it is first determined from the operation position of the mode changeover switch 19 whether or not the basic mode is the photographing mode (step A01).

  In the case of the shooting mode, after selecting the shooting mode (step A02), whether or not the auto shooting mode or the scene identification mode is set next is determined according to the contents of the memory setting at the time when the power is turned off last time. Judgment is made (step A03).

  When it is determined that the auto shooting mode or the scene identification mode is set, the photometry process and the process of returning to the zoom position at the time when the power was previously turned off are immediately executed (steps A04 and A05). The subject brightness obtained by the photometric process and the lens focal length by the zoom process are detected (step A06).

  Next, the feature amount of the image is extracted from the image data obtained at that time (step A07). Detailed contents of the feature amount extraction processing will be described later.

  A shooting scene is automatically selected based on the extracted feature value and the shooting history statistical data stored in the scene / shooting condition statistical data memory 91 (step A08).

  If it is determined in step A03 that the auto shooting mode or the scene identification mode is not set, then a scene-specific shooting mode for selecting one of a plurality of preset scene programs is set. It is determined whether or not there is (step A09).

  Here, when it is determined that the scene-specific shooting mode is not set, processing in other modes is performed (step A26). However, since it is not related to the contents according to the present invention, the description thereof will be given here. Shall be omitted.

  On the other hand, if it is determined in step A09 that the scene-specific shooting mode is set, a process for selecting a shooting scene is executed in accordance with a user operation (step A10), and the selected shooting scene is selected. The photometric process and the zoom process are executed again according to the contents (steps A11 and A12), and the subject brightness obtained by the photometric process and the lens focal length by the zoom process are detected (step A13).

  After the processing of step A08 or A13, the shooting condition data corresponding to the shooting scene selected at that time is read from the scene / shooting condition statistical data memory 91 (step A14).

  In accordance with the read shooting condition data, an AF (automatic focus) range is set, and AF processing within the range including selection of the focus area is executed (step A15).

  Next, shooting conditions such as exposure conditions, white balance (WB), and light emission conditions of the strobe light emitting unit 12 are set according to the selected shooting scene (step A16).

  Except for the flash emission conditions in this manner, the current image data obtained through the CCD 18 under the set shooting conditions is displayed through on the display unit 25 together with the shooting condition information (step A17), and shooting by the shutter switch 15 is performed. It waits for an instruction operation or another key switch operation (step A18).

  Here, when a key switch other than the shutter switch 15 is operated, key processing corresponding to the operated key switch is performed (step A24), and display processing corresponding to the key processing result is performed. (Step A25), the process returns to the processing position corresponding to the operated key.

If it is determined in step A18 that the shutter switch 15 has been operated, a shooting process is executed in synchronization with the light emission processing of the strobe in accordance with the set condition corresponding to the operation (step A19). The image data is compressed and encoded by the compression encoding / decompression decoding 88 and recorded on the recording medium 89 of the digital camera 10 (step A20).
.

  At the same time, the feature extraction / recognition processing unit 90 extracts the feature data from the image data before compression, and associates it with the original image data together with the feature data of the shooting conditions. Specifically, the same file name apart from the identifier Is recorded on the recording medium 89 (step A21).

  Thereafter, the image data recorded by shooting and information such as shooting conditions are reviewed and displayed on the display unit 25 for a predetermined time, for example, 2 seconds (step A22).

  Finally, history data such as shooting scene and subject distance, lens focal length, exposure condition, white balance setting value, strobe emission condition, and other feature quantities, shooting conditions, and the like are additionally stored in the scene / shooting condition statistical data memory 91. Then, the totalization processing is again performed for each shooting mode such as still images, moving images, and voice memos (step A23), and the processing from step A18 is performed to prepare for the next shooting, assuming that a series of processing related to shooting has been completed. Return.

  If it is determined in step A01 that the basic mode is not the shooting mode when the power is turned on, it is then determined whether or not the communication mode is for communication with the personal computer 110 as an external device (step A27). ).

  If it is determined that the communication mode is not selected, processing in another mode is performed (step A26). However, the description is omitted here because it is not related to the contents of the present invention. .

  If it is determined in step A28 that the communication mode is selected, connection processing including synchronization probability with the part weapon function personal computer 110 or the like is performed (step A28), and then authentication processing is performed (step A28). A29).

  In this authentication process, the personal computer 110 determines the authenticity based on the contents registered in advance with data such as the model name and serial number of the digital camera 10.

  When transmission / reception of data is permitted through the above authentication process, the image data newly recorded on the recording medium 89 and its feature data are read out and set in advance on the personal computer 110 side via the external input / output terminal 27. Is transmitted to the current album area (step A30).

  Next, it is determined whether or not the personal computer 110 is currently connected to the server device 112 via the Internet 111 (step A31).

  At this time, if it is determined that the connection with the server device 112 is not established, the setting process is executed to terminate the connection with the personal computer 110 (step A38), and the process returns to step A01. .

  If it is determined in step A31 that the personal computer 110 is currently connected to the server device 112, it is then determined whether or not it is in a state of permitting collection of shooting histories according to prior user settings. (Step A32).

  Only when it is determined that the state is permitted here, the above-described photographing information and photographing history information are encrypted and transmitted to the server device 112 via the personal computer 110 (step A33).

  Thereafter, the update program data of the control software for the digital camera 10 sent from the server device 112 is received (step A34), and update setting processing of the program memory 92 is performed according to the received contents (step A35).

  Next, an update of the statistical data stored in the statistical data memory 91 of the scene / shooting condition is received from the server device 112 (step A36), and update setting processing of the statistical data memory 91 is performed according to the received content (step A37). .

  As described above, the setting process is executed to end the connection with the personal computer 110 (step A38), and the process returns to the process from step A01.

  Next, the operation in the setting mode for making various settings in the digital camera 10 will be described with reference to the flowcharts of FIGS. 8 and 9 and the operation screen examples of FIGS.

  The setting mode is selected by operating the menu key 24 and the cursor key 21 up or down, and it is determined to confirm whether or not the setting mode is set at the beginning of the processing (step B01). If it is not set, processing in other modes is performed (step B02), but since it is not related to the contents according to the present invention, the description thereof is omitted here.

  If it is confirmed in step B01 that the setting mode is set, it is then determined whether or not it is the feature extraction setting mode (step B03).

  In this case as well, for setting modes other than feature extraction, for example, when setting the image size and image quality, the processing is performed in each mode (step B04). The description will be omitted.

  If it is determined that the setting mode is the feature extraction setting mode, display processing in the setting mode such as display of the setting state is performed (step B05).

  FIG. 10A illustrates a setting screen for “feature extraction / identification” on the display unit 25, and shows a state in which the item name “AF on extracted subject” located at the head is also displayed. Show.

  From such a display state, by operating the up or down key of the menu key 24 and the enter key 22, "selection of photographing method by feature extraction", "setting of characteristic data recording", "selection of characteristic data", " It is sequentially determined whether the selection is “selection of feature extraction method”, “selection of description format of feature data”, or “selection of subject to be identified” (steps B06 to B11).

  If the selected item is “selection of shooting method by feature extraction”, this is determined in step B06, and whether or not the feature is extracted according to the operation, and the type of shooting processing by feature extraction is selected ( Step B12).

  FIG. 10B illustrates various sub-items in the item. Here, “(feature extraction) OFF (not extracted)” “subject feature recording” “subject identification / classification” “photographing scene” “Automatically select”, “AF on extracted subject”, “Track extracted subject”,...

  If the selected item is “feature data recording setting”, this is determined in step B07, and whether or not characteristic data is added and recorded according to the operation is set (step B13). .

  If the selected item is “selection of feature data”, this is determined in step B08, and the feature item or feature quantity to be extracted is selected in accordance with the operation (step B14). A feature extraction method is selected on the basis of the feature item and the feature amount thus obtained (step B15).

  If the selected item is “selection of feature extraction method”, this is determined in step B09, and an auto or custom setting feature extraction method is selected according to the operation (step B16).

  FIG. 11A shows a state in which this “feature extraction method” is selected, and by selecting this item, as shown in FIG. This example illustrates that various detailed custom settings such as extraction, contour extraction, texture extraction, motion extraction, and arrangement extraction can be performed.

  If the selected item is “selection of description format of feature data”, this is determined in step B10, and then it is selected whether or not to set for each destination (step B17).

  Here, when it is selected that the setting is made for each destination, the feature data and the feature description format to be sent for each destination are selected according to the operation (step B18).

  On the other hand, if it is selected in step B17 that no setting is made for each destination, only one type of feature data and feature description format is selected according to the operation (step B19).

  After the process of step B18 or B19, it is selected whether or not upload transmission of shooting history information is permitted (step B20).

  Only when it is selected that permission is to be given, the feature data to be uploaded and transmitted to the server device 112, the shooting conditions, the feature description format, and the like are selected according to the operation (step B21).

  FIG. 12A shows a state in which the permission of “shooting history information (upload) transmission” is selected. By selecting this item, an upload destination as shown in FIG. Server URL, feature description format, presence / absence of encryption and specific contents to be uploaded (“image data”, “shooting date / time”, “shooting location”, “shooting condition”, “color distribution”, “contour shape”, “texture”, “motion” And so on) individually.

  If the selected item is “select subject to be identified”, this is determined in step B11, and the type or identification method of the subject to be identified is selected based on the feature data in accordance with the operation (step S11). B22).

  If it is determined that the subject to be identified is not selected in step B11, other key processing is performed (step B23).

  After executing the setting process based on the selection results in steps B06 to B11, the display contents corresponding to the setting result are updated (step B24), and the process returns to the process before the setting mode. To do.

  Next, a connection configuration between the digital camera 10 and the personal computer 110 and the server device 112 via the Internet 111 will be described.

  FIG. 13 illustrates an example of a functional configuration in the case where the server device 112 side outside the digital camera 10 has a shooting history information aggregation function and an image search function of shooting scenes and shooting conditions.

  In this figure, there are a camera control program P11 and a PC transfer program P12 as application programs that operate on the firmware P10 on the digital camera 10 side, and image data recorded on the recording medium 89 and feature descriptions associated therewith. Data and contents of statistical data such as program data and shooting conditions stored in the program memory 92 and the data memory 93 are transmitted to and received from the personal computer 110.

  The personal computer 110 connected to the digital camera 10 by, for example, USB is a part of driver software dedicated to the WWW (World Wide Web) browser P111 operating on the OS (Operating System) P110, which is basic software, and the digital camera 10. The camera transfer software P112 exists at least as a necessary application program.

  The functional configuration of the server device 112 connected to the personal computer 110 via the Internet 111 is as illustrated.

  That is, in this server device 112, a CGI (Common Gateway Interface) program P201 for authenticating a member who owns the digital camera 10, a CGI program P202 for registering image data and feature description data, etc., a search for image data, etc. The CGI program P203 that performs the above is mainly connected to server software P205 that performs overall control of the server device 112 via a document database D204 that stores HTML documents and XML documents.

  The CGI program P201 refers to the stored contents of the member registration database D207 via the authentication software P206 to perform processing such as member authentication, new registration, and deletion.

  The CGI program P202 performs registration and update to the image registration software P208, the image database D210 that stores image data via the description language encoder C209, and the feature description database D211 that stores feature description data.

  The contents stored in the image database D210 and the feature description database D211 are read into the CGI program P203 by the search software P212 and the description language decoder C213.

  On the other hand, the tabulation software P214 for shooting scenes and shooting conditions directly accesses the feature description database D211, reads the feature description data, and obtains the feature description data via the description language decoder C213, thereby performing the tabulation processing. Therefore, the totaled result is updated and stored in the distribution database D215 as statistical data of scenes and shooting conditions.

  The distribution database D215 also stores update data of the camera program. The stored contents are read by the server software P205 and sent to the digital camera 10 via the personal computer 110 via the Internet 111. Is done.

  14 and 15 illustrate the uploading of image data and feature description data from the digital camera 10 to the server device 112 and the downloading of statistical data from the server device 112 to the digital camera 10 in the functional configuration shown in FIG. This shows the processing contents.

  Initially, the digital camera 10 executes communication connection processing with the personal computer 110 to establish a connection state (steps C01 and D01).

  At this time, the digital camera 10 and the personal computer 110 obtain the other party's specifications and capacity information from each other (steps C02 and D02).

  After that, the digital camera 10 responds to the transmission request for the device authentication data issued by the personal computer 110, transmits the device authentication data to the distance measuring sensor 100, and receives the personal computer 110 (steps D03, C03), the digital camera 10 It is then determined whether or not image data is to be transmitted (step C04).

  Here, it is assumed that the user arbitrarily selects image data and feature description data on the digital camera 10 side, and is selected only when there is at least one image data and feature description data selected by the user. The image data, shooting conditions, and feature description data thus transmitted are transmitted to the personal computer 110 (step C05).

  In response to this, the personal computer 110 stores and sets the received image data in a predetermined album folder (step D04), and then determines whether or not the automatic update mode for performing automatic update from the server device 112 is set to ON. Judgment is made (step D05).

  Here, only when it is determined that the setting is on, a connection request is transmitted to the server device 112 via the Internet 111 (step D06).

  The server device 112 always accepts a connection request from the personal computer 110 registered as a member via the Internet 111. When the connection request is received, the server device 112 determines this (step E01) and sends the request to the personal computer 110. Request device authentication data (step E02).

  The personal computer 110 that has received the request for device authentication data from the server device 112 (step D07) transmits the device authentication data received from the digital camera 10 in step D03 to the server device 112 (step D08).

  The server device 112 executes authentication processing by verifying the contents of the member registration data stored in the member registration database D207 by the authentication software P206 based on the device authentication data sent from the personal computer 110 (step E03). From the authentication result, it is determined whether or not the personal computer 110 that has sent the connection request is used by a legitimate user member (step E04).

  If it is determined from the authentication result that the personal computer 110 is not used by a legitimate user member, a response indicating that the connection is not possible is made (step E06), and a request from the personal computer 110 is made thereafter. On the other hand, if it is determined to be used by a legitimate user member, a connection permission response is returned to the personal computer 110 (step E05).

  The personal computer 110 determines whether or not the authentication on the server device 112 side is successful based on the presence or absence of this connection permission response (step D09).

  If it is determined that the authentication has not been successful, the personal computer 110 determines whether to end the communication connection with the server device 112 as it is (step D17), and disconnects the connection with the server device 112 when it is determined to end. (Step D18), and the series of processes is completed once.

  Further, when it is determined in step D09 that the authentication on the server device 112 side is successful, the personal computer 110 requests index information such as update information from the server device 112, and responds to the request from the server device 112. The index information sent is received (steps D10 and E07).

  The personal computer 110 receiving the index information from the server device 112 next requests the digital camera 10 to transmit shooting history information (step D11).

  In response to the shooting history information transmission request from the personal computer 110, the digital camera 10 determines in advance whether or not transmission of the shooting history is permitted by the user's setting (step C06), and determines that it is permitted. Only in this case, the shooting history information set to be transmitted is transmitted (step C07).

  Receiving this, the personal computer 110 transfers the shooting history information of the digital camera 10 to the server device 112 (step D12).

  The server device 112 determines whether or not the shooting history information of the digital camera 10 has been received via the personal computer 110 (step E08). Only when it is determined that there is shooting history information sent thereafter. Is once registered and held in the distribution database D215 (step E09), and the shooting conditions, feature data, and the like are tabulated for each shooting mode by the tabulation software P214, and the contents of the distribution database D215 are updated and stored by the tabulation result (step E10). ).

  The personal computer 110 determines whether there is an update program for the digital camera 10 after the transfer processing of the shooting history information in the step D12 based on the index information received from the server device 112 (step D13). Only when it is determined that there is an update program, a download request for the update program is transmitted to the server device 112 (step D14).

  The server device 112 determines whether or not there is a request for an update program from the personal computer 110 (step E11). If it is determined that there is a request, the server program 112 reads the update program from the distribution database D215 and uses the server software P205 to execute the personal computer. It transmits to 110 side (step E12).

  Thereafter, the server device 112 reads out statistical data such as shooting scenes and shooting conditions stored in the distribution database D215, regardless of whether or not there is a request from the personal computer 110, and transmits the statistical data to the personal computer 110 (step E13). As described above, assuming that the connection with the series of personal computers 110 is completed, a process of temporarily disconnecting the connection with the personal computer 110 is executed, and then there is a wait for a connection request from another personal computer.

  After transmitting the download request in step D14, the personal computer 110 transfers the update program sent from the server device 112 in response to the download request to the digital camera 10 (step D15). The received statistical data such as the photographing conditions is also transferred to the digital camera 10 (step D16).

  After the transfer process to the digital camera 10 or when it is determined in step D13 that there is no update program for the digital camera 10, communication is performed depending on whether communication connection with the server device 112 related to the digital camera 10 is still necessary. It is determined whether or not to end the connection (step D17). If it is determined that the connection is necessary, the process returns to step D04. On the other hand, if it is determined that the connection is not necessary, the server device 112 and the digital camera 10 After the mutual confirmation, a process for disconnecting the connection is executed (step D18), and a series of processes are completed to prepare for the next process.

  The digital camera 10 determines whether or not an update program has been received from the personal computer 110 thereafter, regardless of whether or not there is a process of transmitting shooting history information in step C07 (step C08).

  If it is determined that there is an update program, an update program sent from the personal computer 110 is received (step C09), and statistical data such as shooting conditions is received (step C10). The contents of the program memory 92 and the data memory 93 are updated based on the above (step C11).

  Regardless of whether or not the processing of these steps D09 to D11 is executed, it is further determined whether or not to end the communication connection depending on whether or not the communication connection with the server device 112 via the personal computer 110 is necessary ( Step C12) When it is determined that it is necessary, the process returns to the process from Step C04. On the other hand, when it is determined that it is not necessary, a process for cutting off the connection is performed after mutual confirmation with the personal computer 110. (Step C13), a series of processing is completed to prepare for the next processing.

  Further, the functional configuration in FIG. 13 and the processing contents in FIG. 14 and FIG. 15 show the case where the server apparatus 112 performs the shooting history information summation function and the image search function on the shooting scene and shooting conditions. However, the functional configuration in the case of performing aggregation, image search, or the like inside the digital camera 10 is as shown in FIG. 16, for example.

  That is, FIG. 16 illustrates a functional configuration in the case where a totaling function and a search function are provided on the digital camera 10 side.

  In the digital camera 10, there are PC transfer software P21 and search aggregation software P22 as application programs that operate on the firmware P10, and metadata and the like sent from the personal computer 110 are received by the PC transfer software P21 and then described. After being decoded by the language decoder C23, it is passed to the search aggregation software P22.

  The search aggregation software P22 manages the statistical data D25, the metadata memory D26, and the image data memory D27, and performs search of image data, aggregation of feature description data, and the like. The hardware shown in FIG. In this circuit configuration, statistical data D25 is stored in the scene / shooting condition statistical data memory 91, and metadata D26 and metadata D26 are stored in the feature extraction / recognition processing unit 90.

  The data sent from the personal computer 110 is received by the PC transfer software P21, decoded by the image data decoder C24, and held and stored as image data D27.

  The personal computer 110 acts as an intermediary between the digital camera 10 and the server device 112, and a WWW browser P121 and camera transfer software P122 operate on the OSP 110.

  In the server device 112 connected to the personal computer 110 via the Internet 111, the server software P231 manages the feature description database D232 for storing metadata such as MPEG7 and the image database D233, and from the feature description database D232 together. If necessary, the index data D234 is read and transmitted to the digital camera 10 via the personal computer 110.

  Next, a flow of feature extraction processing for extracting feature amounts from image data and recording processing of the extracted feature data will be described with reference to FIG.

  Before shooting, a shooting condition setting operation directly related to shooting (referred to as “setting operation (1)” in the figure) and other setting item setting operations (referred to as “setting operation (2)” in the figure). ).

  Shooting processing S02 is performed in a state where the shooting condition S01 is specified by an operation for instructing shooting, specifically, operation of the shutter switch 15 in the shooting mode, and shooting information such as data of the shot image S03 and shooting conditions. S04 is obtained.

  FIG. 18A illustrates the contents of shooting information obtained by shooting, and here, the information conforms to the standards of Exif (Exchange image file format) and DPOF (Digital Print Order Format).

  Image compression / encoding S05 and feature extraction S06 are performed on the image data S03 obtained by photographing, and the encoded image data and the thumbnail image obtained by greatly reducing the number of pixels are the main parts of the image data file S11. In addition to being recorded as an element, the above-described photographing information is also added to the image data file S11 as part of the header information.

  Also, an extraction item S08 for performing feature extraction, an on / off setting for feature data setting S09, and a description format S10 for feature data are selected from the setting item S07 by the previous setting operation (2) not directly related to the shooting. The

  Among these, the content of the extracted item S08 is adjusted by the shooting information S04, and the extracted item S08 and the feature data addition on / off setting S09 are related to the feature extraction S06.

  In the feature extraction S06, the details will be described later, but the extraction item S08 and feature data addition on / off setting S09 are applied to the image data S03 before compression / encoding and the image data after compression / encoding. A plurality of feature amounts are extracted and processed, and extracts A, B,... That are processing results are collectively used as feature extraction data S12, and are used for feature data description encoding processing S13, while a shooting scene or the like is used. This is referred to as S15 or the like.

  In the feature data description coding process S13, the feature data S12 is coded after being coded using one specified from a plurality of predetermined description formats A, B,... As a feature description code S14. Additional setting is made to the image data file S11.

  FIG. 18B illustrates the contents of the feature description code S14, and here shows a case where the MPEG7 (ISO / IEC 15938) standard is used as the description format.

  In the image data file S11, the feature description code S14 is added as part of the extended header information.

  According to the method (description format) selected by the user operation, the feature data is, for example, the feature data as header information or extension data of an image file such as DCF or Exif, a metadata description file according to the above-mentioned MPEG7 standard, or SMPTE Standards and languages for describing various metadata such as Metadata standard of EBU, P / Meta standard by EBU, TV-Anytime standard, SMIL standard by W3C, XML syntax by Dublin Core and RDF, or general-purpose description languages such as XML It can be arbitrarily selected and recorded / output from a conforming description file or a description data file obtained by converting these into binary data.

  For example, in Dublin Core (1985), which has been used as a metadata description method for bibliographic information for the longest time, title, description, publisher, creator, copyright (rights) ), Date, type, type, format, etc., and 15 basic attributes (Dublin Core Metadata Element Set: DCMES) are defined, and the notation is, for example, RDF according to XML It is described using a schema of (Resource Description Framework).

  Even in various other metadata description standards, descriptors and description schemes differ depending on the application-specific standards such as broadcast video, bitstream video, and network distribution. be able to.

  In the MPEG7 Visual of the metadata description standard MPEG7 for multimedia, a descriptor (Descriptor) capable of describing various characteristics such as images, video data, and audio, a description format (Description Scheme), and the like are also defined in detail.

For example, the following descriptors are prepared to represent features such as colors, textures, and contour formats, and may be described according to these. Ie
Color space: descriptor that specifies the color space,
Color quantization:
A descriptor that specifies how the color space is quantized,
Dominant color:
A descriptor representing a representative color in an arbitrarily shaped region,
Scalable color:
A descriptor representing color features as a histogram in the HSV color space;
Color layout:
A descriptor representing the color distribution in terms of DCT coefficients;
Color structure:
A descriptor representing the degree of bias in the color distribution,
Homogeneous texture:
A descriptor that represents the features of the texture by image intensity and energy distribution in the frequency space,
Texture browsing:
A descriptor that expresses the characteristics of the texture with three elements of uniformity, granularity, and directionality.
Edge histogram:
A descriptor representing the distribution of edge features as a histogram,
Region shape:
A descriptor representing the shape features of the region,
Contour shape:
A descriptor that represents the shape feature of the closed contour of the region. FIG. 19 and FIG. 20 show examples of description of feature data extracted from an image, and various feature data can be obtained by dividing a region from an entire image by a contour line and gradually subdividing it. I know you can.

  FIG. 19A shows original image data, FIG. 19B shows feature data for the entire image, and FIGS. 19C, 19D, 19E, and 19F show primary regions. The segmented image and its description data, FIGS. 19G to 19K and FIGS. 19L to 19P show the secondary region segmented image and its description data.

Similarly, FIG. 19 (Q) shows the original image data, FIG. 19 (R) is the feature data for the entire image, FIGS. 19 (S), (T) and FIGS. 19 (U), (V) are 1 The next segmented image and its description data are shown. ,
21A, FIG. 21B, and FIG. 22 show examples of description of extracted feature data. FIG. 21A shows MPEG7, FIG. 21B shows XML, and FIG. No. 22 uses each description format of SMIL.

  FIG. 23 illustrates a hierarchical structure of image data files recorded on the recording medium 89 (or the hard disk unit 43) of the digital camera 10, for example, as described above.

  In the figure, under the folder “Root” F000 as the removable disk medium of the digital camera 10, a folder “DCIM” F001, a folder “ALBUM” F002, a folder “MISC” F003, and a file “INDEX.PVM” are shown. "F004" exists.

  The folder “DCIM” F001 is a folder for storing image files, and has folders “100ABCD” F005, “101ABCD” F006,.

  The folder “ALBUM” F002 is a folder for storing a playback playlist and the like, and has a file “ALBUM.PVM” F007 and the like below.

  The folder “MISC” F003 is a folder for storing the DPOF file, and below that, the file “AUTOPRINT.MRK” F008, the file “UNICODE.MRK” F009, the file “AUTOXFER.MRK” F010, and the file “AUTOPLAY.MRK” F011,...

  The “INDEX.PVM” F004 is a meta information description file of the heading of the entire medium described in, for example, MPV, MPEG7, XML, or the like.

  In the folder “100ABCD” F005, the file “CIMG0001.JPG” F012, the file “CIMG0002.JPG” F013,... Are described in, for example, MPV, MPEG7, XML, etc. A file “LIST.PVM” F014, which is a file of image feature description data, is arranged.

The file “ALBUM.PVM” F007 is also a file such as playlist description data described in MPV, MPEG7, XML, etc. The file “AUTOPRINT.MRK” F008 is an automatic print file, and the file “UNICODE.MRK”. F009 is a Unicode character string description file, the file “AUTOXFER.MRK” F010 is an automatic transmission file, and the file “AUTOPLAY.MRK” F011 is an automatic reproduction file, both of which are defined by the DPOF standard.

  FIG. 24 shows an example of the structure of a JPEG compressed data file conforming to the Exif standard, and corresponds to the file “CIMG0001.JPG” F012, the file “CIMG0002.JPG” F013, etc. in FIG.

  FIG. 24A shows JPEG data files corresponding to the above-mentioned “CIMG0001.JPG” F012 and “CIMG0002.JPG” F013 obtained by compressing one still image, and the first information which is attached to the Exif standard. The structure of the application marker segment “APP1” is shown in FIG.

  Further, FIG. 24C shows the structure of an Exif IFD (1st IFD) that is an information tag in “APP1”.

Next, detailed processing contents of the feature extraction processing will be described with reference to FIG.
FIG. 25 shows the subroutine of step A07 in FIG. At the beginning of the processing, first, an image file to be processed is fetched (step F01), and then setting data for performing feature extraction processing is read (step F02).

  It is determined whether or not the feature amount to be extracted is automatically set (step F03), and if it is automatically set, after shooting information of the image is read (step F04), the shooting mode or shooting is performed. Based on the condition, the type of feature quantity to be extracted is selected (step F05).

  If it is determined in step F03 that automatic setting has not been performed, the type of feature value to be extracted, which is manually set by the user, is selected instead (step F06).

  After the process of step F05 or F06, the contents of the preprocessing and feature extraction means are selected according to the type of feature to be extracted (step F07), and the preprocessing in the first stage according to the selected preprocessing contents, Two stages of pre-processing are sequentially executed (steps F08 and F09).

  As described above, the preprocessing is determined by the content of the feature amount to be extracted. Examples of the first stage preprocessing include image enhancement processing, sharpening processing, and noise removal processing. Examples of the two-stage preprocessing include binarization processing, normalization processing, rotation processing, coordinate conversion processing, thinning processing, and expansion / contraction processing.

Hereinafter, some of the preprocessing will be described.
(Sharpening, contour extraction)
Image sharpening processing and contour extraction processing can be performed using the processing of the primary differential filter or the secondary differential filter, and by such filter processing, image sharpening and contour extraction are performed in the preprocessing of feature extraction. Can be used for edge extraction.

  When the contour shape, outline pattern, area, etc. of the subject image are used as feature data, the brightness of the image data to be extracted is subjected to contour enhancement and edge detection filter calculation for each pixel. In many cases, feature data and the like are extracted after the image is converted into an extracted image.

In general, an image f (i, j) in which the edge of a portion where gradation changes is blurred
Δx _f = f (i + 1, j) −f (i−1, j),
Δy _f = f (i, j + 1) −f (i, j−1),
g (i, j) = √ ((Δx _f ) ² + (Δy _f ) ² ), or
g (i, j) = | Δx _f | + | Δy _f |
If a first derivative or gradient is obtained by the above-described calculation, a gradient portion or contour where the gradation changes can be extracted.
Or

Or
▽ ² f (i, j + 1)
= F (i + 1, j) + f (i-1, j)
+ F (i, j + 1) + f (i, j-1) -4f (i, j)
By performing secondary differentiation (Laplacian) processing for further differentiation by subtracting the result from the original image data, it is possible to synthesize an image that emphasizes the high-frequency component of the edge portion, resulting in a blurred edge portion Can be emphasized.

  In order to perform edge and contour enhancement processing by software processing, first-order differential spatial filter processing such as “Prewitt filter”, “Sobel filter”, “Kirsch filter”, “Roberts filter”, or second-order differentiation such as “Laplacian filter”, etc. These spatial filter operators can be used.

(Binarization, averaging, luminance conversion)
By means of luminance histogram conversion processing, halftone enhancement, binarization processing, extraction of a predetermined luminance region, and the like can be performed. When the distribution pattern of image data (luminance values and color difference values) of the subject image is used as feature data, the luminance distribution (histogram) of the image data is obtained, and luminance conversion processing is performed to enhance or compress intermediate tones. Or binarized with a predetermined threshold value to extract only a predetermined luminance region. Furthermore, it is possible to simplify the pattern and compress the information amount by averaging the luminance value and the color difference value for each of a plurality of adjacent pixels.

(Expansion, shrinkage, thinning, line drawing)
In order to discriminate the shape and characteristics of the subject from the contour shape, it is desirable to first simplify the shape and compress the amount of information, or to perform discrimination processing based on a predetermined line figure.

  For example, “1” pixels (one adjacent eight neighbors) for one layer of pixels in a binarized image outline or a boundary region between binarized “1” pixels and “0” pixels When the so-called “expansion” process is performed to make the outer side thicker, the small holes and grooves at the boundary portion such as the contour are removed.

  On the contrary, since the protrusions and isolated points of the boundary portion can be removed by the “contraction” process of thinning by one layer, the combination of the above expansion process and the contraction process can be performed multiple times. Removal and shape simplification can be performed.

  Further, a rough shape of a skeleton can be obtained by a so-called “thinning” process in which a center line having a line width of 1 is extracted from a binarized image.

  Further, the binarized contour pattern is traced along the boundary from the outside and the inside thereof, the boundary pixel is set to “1”, and the remaining pixels are set to “0” by “boundary line tracking” processing. The image can be converted into an image with only the boundary line of the outline.

  In addition, it is possible to use a “near broken line” process or the like that connects only selected pixels for every several pixels of the contour shape with broken lines.

After the above-described two-stage preprocessing is completed, the actual feature extraction processing is executed by the feature extraction / recognition processing unit 90 (step F10).
FIG. 26 partially shows a subroutine listing specific processing procedures of the feature extraction processing, and sequentially executes the first to xth feature extraction processing.

  That is, in step F10, color regions are extracted as the first feature extraction process (step G01), and then representative colors, color distributions, RGB histograms, DCT coefficients,... Based on the extracted color regions. Is created (step G02).

  At the same time, the contour shape is extracted as the second feature extraction process (step G03), and then feature description data such as a shape evaluation value, Fourier descriptor, power spectrum,... Is generated based on the extracted contour shape. (Step G04).

  In addition, texture feature extraction is performed as a third feature extraction process (step G05), and feature description data such as a luminance histogram, Fourier spectrum, co-occurrence matrix, autocorrelation function,... Is created (step G06).

  At the same time, position / placement feature extraction is performed as a fourth feature extraction process (step G07), and feature description data such as position coordinates, DCT coefficients,... Are then created based on the extracted position / placement. (Step G08).

  Hereinafter, after performing some predetermined feature extraction processes, finally, motion is extracted as the x-th feature extraction process (step G09), and then a motion vector, an intensity distribution based on the extracted motion amount, Feature description data such as... Is created (step G10).

  After all the predetermined feature extraction processes are completed, the process returns to the process of FIG. 25 to determine whether or not to perform the recognition process based on the previously set contents (step F11).

  If it is determined that recognition processing is to be performed, template data to be recognized is further read out from the scene / photographing condition statistical data memory 91 (step F12), template matching by the read template data, or the degree of correlation or similarity. A recognition process such as calculation of degree, similarity distance, and difference sum is executed (step F13).

  Thereafter, the extracted feature data and the recognition processing result obtained in step F13 following the determination that the recognition processing is performed in step F11 are output to the compression encoding / decompression decoding 88 (step F14). The feature extraction process at A07 ends.

Next, some specific processing contents of the feature extraction processing will be described in detail below.
FIG. 27 shows the contents of the outline extraction process in step G03 of FIG. 26 and the feature description data creation in step G04.
Initially, the feature extraction / recognition processing unit 90 reads the image data of the subject (step H01), and divides the read image data into a plurality of regions based on the luminance gradation value or the color difference signal (step H02).

  FIG. 28A shows an example of the image data of the subject read in step H01. When such an image is read, it is divided into a plurality of areas in step H02. For example, the image shown in FIG. 28B and the image shown in FIG. 28C are obtained.

  One of the plurality of regions thus divided is selected as a seed subject (step H03), the image of the selected region is binarized (step H04), and the edge is detected (step H05).

  FIG. 28D shows the case where the image shown in FIG. 28C is selected from the two image data shown in FIGS. 28B and 28C and binarized, and the edge is detected. The result is shown in FIG.

  After approximating the contour line obtained in this way with equilateral polygons or polygonal lines of a plurality of N sides (step H06), a Fourier descriptor such as a P (Phase) type descriptor of the contour line is obtained (step H07). ) A power spectrum is obtained (step H08).

  FIG. 28 (F) shows a result obtained by approximating a polygonal line to the edge detection result shown in FIG. 28 (E). By obtaining a Fourier descriptor of the contour line, the spectral normal value with respect to the spatial frequency is obtained. The obtained power spectrum is shown in FIG.

  N components are extracted from the low frequency components of the power spectrum and compressed into contour feature data (step H09). The obtained contour feature data is used for later feature description data creation. It is temporarily stored in the data memory 93 (step H10).

  Next, the feature amount is sequentially compared with the template data stored in the scene / photographing condition statistical data memory 91 to obtain the similarity (step H11).

  FIG. 29 exemplifies the contents of the template data stored in advance in the scene / photographing condition statistical data memory 91 to be compared with the contour feature data.

  After determining the similarity for each template using the contour line feature data of the subject, it is determined that the type corresponds to the template data with the highest similarity value, and the type of feature data Is described (step H12), and the feature description data is output (step H13), thus completing the subroutine processing of FIG.

FIG. 30 shows the contents of the representative color extraction processing during the feature description data creation processing executed in step G02 following the color region extraction processing in step G01 of FIG.
Initially, the feature extraction / recognition processing unit 90 reads the image data of the subject (step I01), and divides the read image data into a plurality of contour regions based on the luminance gradation value or the color difference signal (step I02). .

  With respect to the plurality of divided contour regions, 1 to n subject regions are selected in the order of increasing area or the order close to the focus frame at the center of the image (step I03), and the histogram distribution Pr for each RGB for each divided region ( i), Pg (i), Pb (i) are obtained (step I04).

Next, the obtained histogram distributions Pr (i), Pg (i), and Pb (i) are normalized so that the sum total is “1.0” as a whole (step I05), and then each histogram Pr (i ), Pg (i), and Pb (i), the mode values μ _r , μ _g , and μ _b are obtained (step I06). That is,
μ _r = Max (Pr (i)),
μ _g = Max (Pg (i)),
μ _b = Max (Pb (i)).
It is.

Next, an average μ of the RGB histograms Pr (i), Pg (i), and Pb (i) is obtained (step I07). That is,
μ _r = Σi · Pr (i),
μ _g = Σi · Pg (i),
μ _b = Σi · Pb (i).

Then, the variance σ ² of the RGB histograms Pr (i), Pg (i), and Pb (i) is obtained using the obtained average (step I08). That is,
σ _r ² = Σ (i−μ _r ) ² · Pr (i),
σ _g ² = Σ (i−μ _g ) ² · Pg (i),
σ _b ² = Σ (i−μ _b ) ² · Pb (i).

  The mode value or average value of the RGB histogram of each region thus obtained is used as a representative color, and the representative color and RGB variance are described and output as color characteristic data (step I09). The subroutine processing is terminated.

FIG. 31 shows the contents of the color distribution extraction process during the feature description data creation process executed in step G02 following the color region extraction process in step G01 of FIG. is there.
At the beginning, the feature extraction / recognition processing unit 90 reads a captured image or a preset region portion in the captured image (step J01), and reads the read image data in a block of m vertical pixels × n horizontal pixels. A plurality of block regions are divided into units (step J02).

  Next, the average color is calculated for each block (step J03), and the average color calculated for each block is converted into DFT (Discrete Fourier Transform) or DCT (Discrete transform) in the primary color RGB space or luminance color difference system YCbCr space. Each coefficient is obtained by performing orthogonal transformation by Cosine Trasform (discrete cosine transformation) to convert to a spatial frequency component axis (step J04).

  The low-frequency component coefficient closer to the DC component as a result of the orthogonal transformation is scanned by zigzag scanning to make it one-dimensional (step J05), and each one-dimensionalized coefficient is directly quantized and digitally encoded as a sequence of numbers. Obtain (step J06).

  By describing and outputting the digitally encoded number sequence as color distribution feature data (step J07), the subroutine processing of FIG. 31 is completed.

  Next, although not shown in the figure, the operation in the case of extracting the hue from the image data, the skin color portion, and the human face will be briefly described.

  As an example of extracting a region of a predetermined color, when extraction of a human skin color region is considered, spectral reflectance characteristics of human skin, or an RGB value or HSB (Hue: hue of a representative color of a predetermined region of a captured image, It is possible to extract a skin color area, a human skin area, or a face area based on the Saturation value, the value of brightness, and the brightness value.

  For example, when distribution data of skin color hue (Hue) and saturation (Saturation) is taken, most of the skin color image data has a wide range of saturation, but the hue circle has a range of about 6 ° to 38 °. It is known to be distributed within.

  Therefore, if this point is used, a region of a subject with a specific color can be extracted as appropriate, such as recognizing a range of hues of approximately 6 ° to 38 ° as the skin color of a human face from the HSB value.

  Similarly, a specific recognition process may be required for each specific subject or subject of interest. For example, in order to extract an eye area of a face, a mask pattern of a pupil area is created and used as a reference pattern. A search can be performed using template matching or the like, and an image region with an eye or a face can be specified from the input image.

In particular, as a condition for recognizing the eye, for example, when the length of the eye = b / a (a: the length of the eye in the vertical direction, b: the length of the eye in the horizontal direction), α1 ≦ b / a ≦ α2 ( α1 and α2: constants) or the area of the eye is S1≈π × a × b, the area of the black eye part S2 = πr ² (r: the diameter of the black eye part), and the ratio S2 / S1 of the black eye The region of the subject image that satisfies the condition of β1 ≦ S2 / S1 ≦ β2 (β1, β2: constant) can be identified as the “eye region”.

  Furthermore, as condition data for recognizing a human face, for example, the length h1 from the eyebrow to the nose and the length h2 from the nose to the chin are substantially equal, or the right eye width W1 and the binocular A region of the subject image that satisfies a condition such that the length W2 between them and the width W3 of the left eye are both substantially equal can be identified as a “face region”.

  Next, a motion feature extraction process performed on the subject image constituting the motion JPEG file (AVI file) or MPEG (-1, 2, 4) file will be described with reference to FIG.

FIG. 32 shows the contents of the motion extraction process in step G09 of FIG. 26 and the feature description data creation process executed in step G10.
Initially, the feature extraction / recognition processing unit 90 reads a plurality of temporally continuous images (step K01), and divides each read image data into, for example, a block unit of vertical m pixels × horizontal n pixels. Block area (step K02).

  The luminance value or the color difference value of each block of each image is template-matched with the image positioned immediately before in time, and the moving direction and moving amount are extracted (step K03).

  After quantizing the moving direction and moving amount for each divided block to obtain the motion vector of each block (step K04), the motion vector (moving direction and moving amount) for each image and each block is converted into the motion vector. By describing and outputting as feature data (step K0507), the subroutine processing of FIG. 32 is completed.

  Although not shown in the drawings, the factors of feature amount comparison, similar figure search, correlation, similarity of template matching, absolute value difference sum, and distance will also be described.

(Comparison of features and search for similar figures)
A pattern matching method such as template matching or a block matching method in motion vector detection can be used to determine or search the similarity between the stored template image and the subject image.

  From the feature data of the input image f [i, j] in the feature extraction region by template matching, for example, the position of the image that matches the reference image or feature data t [k, l] recorded in (m × n). To detect. The center or end point of the reference image is placed so that it overlaps a certain point (i, j) of the input image and the point (i, j) is sequentially scanned in raster, and the similarity of the image data in the overlapping part is calculated sequentially. Thus, the point (i, j) at the position with the highest degree of similarity can be obtained as the position where there is a similar subject.

(Correlation degree (correlation function))
The degree of correlation between the input image f [i, j] and the template image value reference image t [k, l] can be calculated by the Pearson correlation coefficient (product moment correlation coefficient) R of the receiver, etc. The position where the value of the correlation function R is the largest is obtained as the position where there is a similar subject to be searched.

(Similarity of template matching)
When searching for an image t [k, l] having an image size (m × n) from images f [i, j] such as template matching, the similarity can be calculated by the following equation: The point (i, j) at the scanning position where the similarity R (i, j) is the largest is obtained as the position of the similar subject.

Similar to the correlation function R described above, this may be normalized and used, for example, by subtracting the average value.
(Absolute difference sum and distance)
Since the amount of calculation increases because the similarity R (i, j) is a multiplication, when the average value can be subtracted or normalization can be omitted, such as a binarized image, the calculation is changed to the similarity calculation. Thus, D (i, j) representing the degree of difference (“distance”) can be obtained from the sum of differences between images as in the following equation, and this can be used as an evaluation function.

In this case, since calculation can be performed only by addition and subtraction, the operation can be speeded up, and the point (i, j) with the smallest distance D (i, j) represents the matching position.

  Next, texture analysis using a luminance histogram will be described with reference to FIG.

FIG. 33 shows the contents of the texture feature extraction process in step G05 of FIG. 26 and the feature description data creation process executed in step G06.
Initially, the feature extraction / recognition processing unit 90 reads the image data of the subject (step L01), and obtains the luminance histogram P (i) from the read image data (step L02).

Next, after normalizing the histogram so that the total sum is 1.0 (step L03), the average value μ and variance σ ^{2 of} the luminance histogram are obtained (steps L04 and L05). That is,
μ = Σi · P (i),
σ ² = Σ (i−μ) ² · P (i),
Further, the degree of distortion S (Skewness) and the degree of kurtosis K (Kurtosis) are obtained (steps L06 and L07). That is.

S = (Σ (i−μ) ³ · P (i)) / σ ³
K = (Σ (i−μ) ⁴ · P (i)) / σ ⁴
The distortion degree S thus obtained is feature extraction data f1, the kurtosis K is the same data f2, and both are described and output as texture feature data (step L08), thus completing the subroutine processing of FIG.

In addition to the above-described texture feature extraction method, for example, a texture analysis method from a Fourier spectrum may be used.
In this case, Fourier spectrum obtained by discrete two-dimensional Fourier transform
F (u, v) = ΣΣf (x, y) W1 ^xu W2 ^yv
(W1 = exp (−j2π / M),
W2 = exp (−j2π / N). )
Is the power spectrum of F (u, v)
P (u, v) = | F (u, v) | ²
Is converted into P (r, θ) in the polar coordinate format, and the energy p (r) of the donut-shaped region centered at the origin and the energy q ( θ)

  From the statistics such as the peak, average, and variance of the histogram distribution, the texture characteristics such as the texture “mesh” size by the P (r) peak and the texture pixel directionality by the q (θ) peak, for example. The feature data may be extracted by extracting the quantity and output.

Furthermore, texture features may be extracted using a luminance co-occurrence matrix.
That is, in the image f (x, y), the luminance of the pixel f (x2, y2) in the positional relationship (d, θ) at the distance d and the angle θ from the pixel f (x1, y1) is represented by d (1, 2, ……), θ (0 degrees, 45 degrees, 90 degrees, 135 degrees, etc.)
(However, d = max (| x1 = x2 |, | y1-y2 |))
The brightness of two points in the positional relationship (d, θ)
f (x1, y1) = i,
f (x2, y2) = j
The frequency (histogram) that is paired with each other is calculated to obtain a luminance co-occurrence matrix P (i, j; d, θ), and the normalized co-occurrence matrix
p (i, j) = P (i, j) / Σ _i Σ _j (i, j)
From the angular second moment that represents the overall uniformity
f1 = ΣΣ (p (i, j)) ² ,
Contrast representing local changes (Comtrast)
f2 = Σ (n ² p (i, j))
Correlation about direction
f3 = ΣΣ (p (i, j) p (i, j) −μ ² ) / σ ²
(Where μ = Σp (i) = 1 / g (g: number of gradations),
σ = √ (Σ (p (i) −μ) / (g−1)),
Sum of squares representing local homogeneity
f4 = ΣΣ ((i−μ) ² p (i, j))
Inverse difference moment representing a measure of complexity
f5 = ΣΣ (p (i, j) / (1+ (i−j)))
Etc. can be obtained and used as texture feature data.

Or you may extract the feature-value of a texture with a high-order local autocorrelation function.
Specifically, the RGB component of each pixel is converted to a YIQ component to obtain a Y image, an image as a value, and a Q image, and a plurality of mask patterns 1 to (n × n) pixels according to the set characteristics. 25 reference points are overlapped and multiplied by the luminance value f (i, j) of each pixel (logical product) to obtain the following feature vectors C1 to C25.

C1 = Σ _i Σ _j (f (i, j)),
C2 = Σ _i Σ _j (f (i, j) f (i + 1, j)),
...
C5 = Σ _i Σ _j (f (i, j) f (i−1, j + 1)),
C6 = Σ _i Σ _j (f (i, j) f (i−1, j) f (i + 1, j)),
...
C25 = Σ _i Σ _j (f (i, j) f (i−1, j + 1) f (i + 1, j + 1)),
Add (sum of products) the above C1 to C25
Ci = Ci / Σj (Cj) (i, j = 1, 2,..., 25)
The feature vectors C1 to C25 obtained by local autocorrelation obtained for each YIQ image, or their statistics may be described and output as texture feature data.

  34, the statistical data obtained by compiling the photographing history information from the digital cameras 10 owned by many users as described above is downloaded from the server device 112 and stored in the statistical data memory 91 of scenes and photographing conditions in the digital camera 10. The options for selecting a shooting scene based on the contents stored in the statistical data memory 91 of the scene / shooting condition are shown together with feature data extracted after shooting and a template image that is compared at the time of extraction. Is.

  As shown in the figure, the scene-specific shooting modes include large categories such as “person”, “children”, “landscape”, “night view and person”, “character”, etc., for example, “one person” and “multiple” in the large classification “person”. There are small classifications such as “two-shot”, etc., and when the user selects a scene-specific shooting mode, various shooting conditions corresponding to the shooting mode are automatically set.

  For example, in the “Class” category, the color emphasis is set to the skin color in common, and the shutter speed is set relatively fast during exposure, resulting in a shallow depth of field. In this case, the area around the subject will be moderately blurred and the person will be raised. On the other hand, in the horizontal direction, finer settings are performed, such as setting the positions to the left and right of the center of the screen, and the points are reflected in the extracted feature data.

  FIG. 35 and FIG. 36 exemplify shooting control items in the shooting mode for each scene based on the contents stored in the above-described scene / shooting condition statistical data memory 91. Only the contents of the major classification of the scene-by-scene shooting mode shown in FIG. 34 above, for example, if the major classification is “person”, the serial number “SP n” (“SP” is the scene This is shown together with a template image corresponding to another program) and sub-classification, and the shooting control items that are easy to understand for the user at the bottom of the screen are explained together with precautions as necessary.

  The user of the digital camera 10 can automatically update and set the contents of such a scene-specific shooting mode by communicating with the server device 112 only by connecting the digital camera 10 to the personal computer 110 or the like.

  As described above, since the feature extraction of the captured image is performed and stored in association with the original image data, a desired image is retrieved from the content of the captured image. Not only can the contents be used effectively, but also using an external information processing device such as a personal computer to analyze the correlation between shooting conditions and shooting results and the correlation between shooting conditions and shooting scenes, etc. When outputting externally, it is not always necessary to output the captured image itself. For example, even when the load of external output is heavy or when a private captured image is not desired to be output externally, it can be handled without any problem. .

  In addition, for example, each digital camera 10 can provide information effective for aggregation and analysis to a server device that can collect information from a plurality of digital cameras 10 via a wide area network.

  In addition, anyone can effectively use the shooting parameters set by many people when shooting various scenes, and the shooting conditions necessary to obtain the desired shooting results are not dependent on their own experience. The other person's photographing result can be determined as a reference.

  In addition, when downloading statistical data from the server device 112, since only necessary information is downloaded from the server device 112 with reference to model information of the digital camera 10, it is owned without wasteful communication. Only various types of information suitable for the digital camera 10 can be automatically selected and acquired and used.

  In addition, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention.

Further, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, at least one of the problems described in the column of the problem to be solved by the invention can be solved, and described in the column of the effect of the invention. In a case where at least one of the obtained effects can be obtained, a configuration in which this configuration requirement is deleted can be extracted as an invention.

1 is a diagram mainly showing an external configuration of a digital camera according to a first embodiment of the present invention. The figure which shows the other external appearance structure which concerns on the same embodiment. FIG. 2 is a block diagram showing a conceptual system configuration according to the embodiment. The block diagram which shows the specific circuit structure and connection with an external apparatus which concern on the embodiment. The flowchart which shows the general processing content after the power activation which concerns on the embodiment. The flowchart which shows the general processing content after the power activation which concerns on the embodiment. The flowchart which shows the general processing content after the power activation which concerns on the embodiment. The flowchart which shows the processing content of the feature extraction under the setting mode which concerns on the embodiment. The flowchart which shows the processing content of the feature extraction under the setting mode which concerns on the embodiment. The figure which illustrates the setting screen of the feature extraction which concerns on the embodiment. The figure which illustrates the setting screen of the feature extraction which concerns on the embodiment. The figure which illustrates the setting screen of the feature extraction which concerns on the embodiment. The block diagram which illustrates about a function structure in case a server apparatus has the total and image search function which concern on the embodiment. 7 is a flowchart showing processing contents among a digital camera, a personal computer, and a server device when performing aggregation and image search according to the embodiment. 7 is a flowchart showing processing contents among a digital camera, a personal computer, and a server device when performing aggregation and image search according to the embodiment. The block diagram which illustrates about a function structure in case a digital camera has the total and image search function which concern on the embodiment. The figure explaining the flow of the feature extraction process which extracts the feature-value from the image data which concerns on the embodiment, and the recording process of the extracted feature data. The figure which illustrates the content of the imaging | photography information obtained by FIG. 17 and the characteristic description code based on the embodiment. The figure which shows the example of description of the feature data extracted from the image which concerns on the embodiment. The figure which shows the example of description of the feature data extracted from the image which concerns on the embodiment. The figure which shows the example of description of the extracted feature data based on the embodiment. The figure which shows the example of description of the extracted feature data based on the embodiment. The figure which illustrates the hierarchical structure of the image data file recorded on the recording medium based on the embodiment. The figure which shows the structural example of the compressed data file which concerns on the embodiment. The flowchart which shows the processing content of the subroutine of the feature-value extraction of FIG. 5 which concerns on the embodiment. The flowchart which enumerates the specific process sequence of the subroutine of the characteristic extraction process of FIG. 25 based on the embodiment, and shows the processing content. The flowchart which shows the processing content of the subroutine of the contour extraction process of FIG. 26 which concerns on the embodiment. The figure which illustrates the content of the recognition process by the outline extraction which concerns on the same embodiment, and the extracted outline. The figure which illustrates the content of the recognition process by the outline extraction which concerns on the same embodiment, and the extracted outline. The flowchart which shows the processing content of the subroutine of the representative color extraction process of FIG. 26 which concerns on the same embodiment. The flowchart which shows the processing content of the subroutine of the color distribution extraction process of FIG. 26 which concerns on the embodiment. The flowchart which shows the processing content of the subroutine of the movement extraction process of FIG. 26 which concerns on the same embodiment. The flowchart which shows the processing content of the subroutine of the texture analysis by the brightness | luminance histogram of FIG. 26 which concerns on the embodiment. The figure which illustrates the feature-value and template image for every imaging | photography mode according to the embodiment concerning the same embodiment. The figure which illustrates the content of the display screen and imaging | photography control item for every imaging | photography mode according to the embodiment concerning the same embodiment. The figure which illustrates the content of the display screen and imaging | photography control item for every imaging | photography mode according to the embodiment concerning the same embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,10 '... Digital camera, 11 ... Shooting optical system light receiving window, 12 ... Strobe light emitting part, 13 ... Grip part, 14 ... Power switch, 15 ... Shutter switch, 16 ... Mirror, 17 ... Shooting lens group, 18 ... CCD , 19 ... Mode switch, 20 ... Zoom switch, 21 ... Cursor key, 22 ... Enter key, 23 ... Display switch key, 24 ... Menu key, 25 ... Display section, 26 ... Storage lid, 27 ... External input / output terminal, DESCRIPTION OF SYMBOLS 31 ... Shooting lens part, 32 ... Microphone, 33 ... Display part, 34 ... Speaker, 35 ... Zoom switch, 36 ... Shooting switch, 37 ... Cursor switch, 38 ... Playback switch, 39 ... Power supply / mode switch, 40, 41 ... External input / output terminal, 42 ... Open / close lid, 43 ... Hard disk unit, 44 ... Memory card, 51 ... Operation input 52 ... Shooting / recording control unit, 53 ... Shooting condition setting unit, 54 ... Feature extraction / description setting unit, 55 ... Shooting unit, 56 ... Signal processing unit, 57 ... Image data memory, 58 ... Feature extraction unit, 59 ... Image / moving image compression / encoding unit, 60 ... multiplexing unit, 61 ... feature data encoding unit, 62 ... comparison circuit, 63 ... update recording control unit, 64 ... memory control unit, 65 ... image data memory, 66 ... External input / output unit, 67 ... statistic storage unit, 68 ... reproduction control unit, 69 ..., 70 ... image / moving image expansion / decoding unit, 71 ... feature data combination unit, 72 ... display control unit, 73 ... display unit , 80 ... Lens driver, 81 ... Video signal processing unit, 82 ... Image memory, 83 ... Control circuit, 84 ... Microphone, 85 ... Amplifier, 86 ... A / D conversion circuit, 87 ... Audio memory, 88 ... Compression encoding / Decompression decoding, 89 ... recording medium, 90 ... special Extraction / recognition processing unit, 91 ... Statistical data memory of scene / shooting conditions, 92 ... Program memory, 93 ... Data memory, 94 ... Shooting control unit, 95 ... Display control unit, 96 ... Input circuit, 97 ... External input / output interface 98 ... Hard disk interface (HDD / IF), 99 ... D / A conversion circuit, 100 ... Distance measuring sensor, 101 ... Photometry / ranging unit, 102 ... Amplifier, 103 ... Speaker, 104 ... Operation input unit, 110 ... Personal Computer (PC), 111 ... Internet, 112 ... Server device, 113 ... Image database (DB), P10 ... Firmware, P11 ... Camera control program, P12 ... PC transfer program, P21 ... PC transfer software, P22 ... Search and aggregation software, C23: Description language decoder, C24: Image data decoder, D25 ... Statistical data, D26 ... Metadata, D27 ... Image data, P110 ... OS, P111 ... WWW browser, P112 ... Camera transfer software, P201-P203 ... CGI program, D204 ... Document database, P205 ... Server software, P206 ... Authentication software , D207 ... Member registration database, P208 ... Image registration software, C209 ... Description language encoder, D210 ... Image database, D211 ... Feature description database, P212 ... Search software, C213 ... Description language decoder, P214 ... Aggregation software, D215 ... Distribution database P231: server software, D232: feature description database, D233: image database, D234: index data.

Claims (10)

Setting means for setting shooting conditions;
Photographing means for performing photographing in accordance with photographing conditions set by the setting means;
Feature extraction means for extracting feature information indicating the characteristics of the image based on a predetermined viewpoint from image information obtained by photographing with the photographing means;
Storage means for storing the shooting conditions set by the setting means and the feature information obtained by the feature extraction means in association with each other as shooting history information;
A plurality of shooting history information stored in the storage means is totaled, and shooting conditions for newly taking a picture are determined by using the totaled result and set by the setting means, and shooting by the shooting means is performed. An imaging apparatus comprising: an imaging control means for performing A total of a plurality of shooting history information stored in the storage means is added to extract a correlation between various shooting conditions and various feature information, and further includes a correlation extraction means for storing as correlation information,
The shooting control means acquires feature information corresponding to a shooting target to be newly shot, specifies a shooting condition having a correlation with the acquired feature information based on the stored correlation information, and specifies the specified shooting condition 2. The image pickup apparatus according to claim 1, wherein the image pickup device is set by the setting means and the image pickup means is used for shooting. Setting means for setting shooting conditions;
Photographing means for performing photographing in accordance with photographing conditions set by the setting means;
Feature extraction means for extracting feature information indicating the characteristics of the image based on a predetermined viewpoint from image information obtained by photographing with the photographing means;
Transmission for transmitting photographing history information in which the photographing conditions corresponding to the image information obtained by the photographing means and the feature information obtained by the feature extracting means corresponding to the image information are associated to the external device for aggregation Means,
Correlation information indicating a correlation between various shooting conditions and various feature information obtained by totaling shooting history information from a plurality of imaging devices by the external device for aggregation is received from the external device for aggregation. Receiving means for
The feature information corresponding to the subject to be newly photographed is acquired, the shooting condition having a high correlation with the acquired feature information is specified based on the received correlation information, and the specified shooting condition is set in the setting An image pickup apparatus, further comprising: a shooting control unit configured to set by the shooting unit and perform shooting by the shooting unit. It further comprises input means for inputting designation information indicating a predetermined viewpoint given from the outside of the apparatus,
4. The imaging apparatus according to claim 3, wherein the feature extraction unit extracts feature information indicating the feature of the image based on the designation information input by the input unit. The input means also inputs shooting condition information in addition to the designation information,
The imaging apparatus according to claim 4 , wherein the setting unit sets an imaging condition based on information input from the input unit. The transmission means outputs together the classification information for classifying the imaging device,
6. The image pickup apparatus according to claim 4 , wherein the input unit inputs designation information or photographing condition information corresponding to the classification information given from the outside of the apparatus. A setting process for setting shooting conditions;
A shooting process for performing shooting according to the shooting conditions set in this setting process;
A feature extraction step of extracting feature information indicating features of the image based on a predetermined viewpoint from the image information obtained by photographing in the photographing step;
A storage step of associating and storing the shooting conditions set in the setting step and the feature information obtained in the feature extraction step as shooting history information;
A plurality of shooting history information stored in this storage step is totaled, and the shooting conditions for newly taking a picture are determined by using the totaled result and set by the setting step, and shooting by the shooting step is performed. An imaging method comprising: an imaging control step for performing A setting process for setting shooting conditions;
A shooting process for performing shooting according to the shooting conditions set in this setting process;
A feature extraction step of extracting feature information indicating features of the image based on a predetermined viewpoint from the image information obtained by photographing in the photographing step;
Transmission for transmitting photographing history information in which the photographing conditions corresponding to the image information obtained in the photographing step and the feature information obtained in the feature extraction step corresponding to the image information are associated to the external device for aggregation Process,
Correlation information indicating a correlation between various shooting conditions and various feature information obtained by totaling shooting history information from a plurality of imaging devices by the external device for aggregation is received from the external device for aggregation. A receiving process,
The feature information corresponding to the subject to be newly photographed is acquired, the shooting condition having a high correlation with the acquired feature information is specified based on the received correlation information, and the specified shooting condition is set in the setting An imaging method comprising: an imaging control step of performing imaging in accordance with the imaging step, and setting in accordance with a process. A program executed by a computer incorporated in an imaging apparatus,
A setting step for setting shooting conditions;
A shooting step for performing shooting according to the shooting conditions set in this setting step;
A feature extraction step for extracting feature information indicating the feature of the image based on a predetermined viewpoint from the image information obtained by photographing in the photographing step;
A storage step of storing the shooting conditions set in the setting step and the feature information obtained in the feature extraction step in association with each other as shooting history information;
A plurality of shooting history information stored in the storage step is totaled, and the shooting conditions for newly shooting are determined by using the totaled result and set in the setting step, and shooting in the shooting step is performed. A program for causing a computer to execute a photographing control step for performing A program executed by a computer incorporated in an imaging apparatus,
A setting step for setting shooting conditions;
A shooting step for performing shooting according to the shooting conditions set in this setting step;
A feature extraction step for extracting feature information indicating the feature of the image based on a predetermined viewpoint from the image information obtained by photographing in the photographing step;
Transmission for transmitting photographing history information in which the photographing conditions corresponding to the image information obtained in the photographing step and the feature information obtained in the feature extraction step corresponding to the image information are associated to the external device for aggregation Steps,
Correlation information indicating a correlation between various shooting conditions and various feature information obtained by totaling shooting history information from a plurality of imaging devices by the external device for aggregation is received from the external device for aggregation. Receiving step to
The feature information corresponding to the subject to be newly photographed is acquired, the shooting condition having a high correlation with the acquired feature information is specified based on the received correlation information, and the specified shooting condition is set in the setting A program that causes a computer to execute a shooting control step of setting by steps and performing shooting by the shooting means .

Images