JP2008182544A - Image storage device, and image storage method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To appropriately and conveniently handle imaged image data. <P>SOLUTION: For the imaged image data, its image evaluation is performed to generate evaluation data. The evaluation data is considered as a value obtained by evaluating images by such evaluation items that indicates degrees of satisfaction of a user to imaged images. The evaluation data is considered as the data indicating, for example, blurring, camera shake, light and darkness, tones, definition, an angle gap, fogging of an asymmetrical object and furthermore, expression of an object person, etc. indicating presence/absence and degrees of a state that the user generally feels dissatisfaction to photographs, etc. Then, imaged image data with low evaluation is deleted, replaced with other imaged image data with similar content to be stored, imaged image data with high evaluation is selected and output when display output, etc. is performed. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image storage apparatus and an image storage method that perform storage processing on captured image data, and more particularly, to an apparatus and method for performing evaluation on acquired captured image data and storing captured image data together with evaluation data.

JP 2005-128092 A JP 2006-33123 A JP 2006-222490 A JP 2003-110999 A JP 2005-303511 A

Patent Documents 1 to 3 disclose various techniques related to camera shake prevention.
Patent Documents 4 and 5 disclose techniques of an image (photo) sharing system and an image (photo) sharing service.

However, with conventional cameras (video cameras and still cameras), no countermeasures have been taken with respect to the image that has been captured.
In general, in image data captured by a user with a camera, various images that are normally failed are generated. For example, the image may be blurred, affected by camera shake, the brightness may not be appropriate due to insufficient light or backlight, an asymmetric subject such as a finger or a strap may be captured, or the angle may deviate from the horizontal. And so on. In addition, when a person is imaged, the person who becomes the subject may pinch his eyes at the shutter timing or the expression may deteriorate, which may not be a preferable captured image.

  As described above, various situations occur in actual imaging, and an object of the present invention is to make it possible to handle captured image data appropriately and conveniently thereafter.

  An image storage apparatus according to the present invention includes an acquisition unit that acquires captured image data, an image evaluation unit that performs image evaluation on a predetermined evaluation item, and generates evaluation data, and a storage processing unit that performs storage processing of the captured image data. When the captured image data to be evaluated is acquired by the acquisition unit, the image evaluation unit generates evaluation data for the captured image data, and the captured image data and the evaluation data are stored in association with each other. As described above, the storage processing unit includes a control unit that executes a storage process.

The acquisition unit includes an imaging unit that captures an image, and acquires captured image data obtained by imaging with the imaging unit. The imaging unit is configured using a CCD sensor or a CMOS sensor as an imaging element.
The acquisition unit includes a communication unit that communicates with an external device, and receives and acquires captured image data through communication with the external device at the communication unit.
The acquisition unit includes a reproduction unit that reproduces information from a portable recording medium, and reproduces the portable recording medium by the reproduction unit to obtain captured image data.
The image evaluation unit performs image evaluation as to whether to adopt or save captured image data to be evaluated.
The image evaluation means performs image evaluation at a stage or a score.
The image evaluation means includes the evaluation item for blurring, blurring, brightness / shade, hue, sharpness, angle shift, or all or part of the asymmetrical subject fog for the captured image data to be evaluated. Image evaluation.
The image evaluation unit determines whether or not the captured image data to be evaluated is a person as a subject, and if the person is a subject, the captured image data to be evaluated. The image evaluation is performed using all or part of the eye opening / closing, facial expression, facial orientation, or eyeball color as the evaluation items.
The storage processing unit includes a recording unit that records information on a recording medium, and records the captured image data and the evaluation data in association with each other as the storage process.
The storage processing unit includes a communication unit that communicates with an external device, and transmits the captured image data and the evaluation data to the external device in association with each other as the storage processing.

The control means further performs a deletion process for deleting the captured image data stored by the storage processing means based on the evaluation data stored correspondingly.
Further, the control unit further performs a replacement process for replacing the captured image data stored by the storage processing unit with another captured image data based on the corresponding stored evaluation data.
Further, the image processing apparatus further includes image output means for performing image output processing, and the control means further includes, based on the evaluation data stored correspondingly to the captured image data stored by the storage processing means, the image. Processing to be output by the output means is performed.
The image output means includes a display unit, and as the output process, the display unit performs display output on the captured image data.
The image output unit includes a communication unit that communicates with an external device. As the output process, the communication unit transmits and outputs captured image data to the external device.
Further, the image output means has a recording unit for recording information on a portable recording medium, and the picked-up image data is recorded on the portable recording medium by the recording unit as the output process.

  The image storage method of the present invention includes a step of acquiring captured image data to be evaluated, a step of performing image evaluation on the captured image data using predetermined evaluation items, and generating evaluation data, and a captured image data And a step of executing a storing process so that the evaluation data is stored in association with each other.

  That is, in the present invention described above, evaluation data is generated by performing image evaluation on captured image data acquired by, for example, imaging or inputting image data captured by an external imaging device. Assume that the evaluation data is, for example, a value evaluated by an evaluation item indicating the degree of user satisfaction with the captured image. For example, data indicating the presence / absence or degree of a state in which a user generally feels dissatisfied with a photograph, such as blurring, blurring, light / dark, hue, sharpness, angle shift, asymmetric subject fogging, and even a facial expression of a subject. To do. Such evaluation data is stored in association with the captured image data.

According to the present invention, evaluation data serving as a reference for user satisfaction is generated for acquired captured image data, and the captured image data is stored together with the evaluation data. Thereby, there is an effect that the stored captured image data can be handled appropriately and conveniently.
For example, it is possible to delete captured image data with low evaluation, replace it with other captured image data with similar contents, save it, and select and output captured image data with high evaluation when performing display output etc. . Thereby, the satisfaction and convenience for the user can be improved.

Hereinafter, embodiments of the image storage device and the image storage method of the present invention will be described. Here, an imaging device will be described as an example of the image storage device. Note that the image storage device of the present invention can be realized by a device other than the imaging device, which will be described as a modified example and an extended example.
The description will be given in the following order.
[1. Appearance example of imaging device]
[2. Configuration example of imaging apparatus]
[3. Image evaluation]
[4. Image saving process]
[5. Image display processing]
[6. Image deletion process]
[7. Image replacement process I]
[8. Image replacement process II]
[9. Effect of embodiment, modification and extension example]

[1. Appearance example of imaging device]

Although various forms are assumed as the imaging device 1 of the embodiment, examples of their appearance are illustrated in FIGS. 1 (a), (b), (c), and (d).
FIG. 1A shows an imaging apparatus 1 that is a glasses-type display camera. For example, the imaging apparatus 1 has a mounting unit having a frame structure that makes a half turn from both sides of the head to the back of the head, and is mounted on the user by being placed on both ear shells as illustrated.
In the imaging apparatus 1, the imaging lens 3 a is arranged facing forward so that imaging is performed with the user's visual field direction as the subject direction when the user is wearing the imaging apparatus 1.
Further, in the wearing state as shown in the figure, a pair of display panels 2a and 2a for the left eye and the right eye are arranged immediately in front of both eyes of the user, that is, at a position where a lens in normal glasses is located. Has been. For example, a liquid crystal panel is used for the display panel unit 2a. By controlling the transmittance, a through state as shown in the figure, that is, a transparent or translucent state can be obtained. Since the display panel unit 2a is in the through state, even if the user always wears it like glasses, there is no problem in normal life.
In addition to a pair of display panel units 2a corresponding to both eyes, a configuration in which one display panel unit 2a is provided corresponding to one eye is also conceivable.

  FIG. 1B shows the image pickup apparatus 1 that the user similarly wears on the head, but does not include the display panel unit 2a as shown in FIG. For example, it is mounted on the user's head by a mounting unit that is hung on the ear shell. In this state, the imaging lens 3a is arranged facing forward so that the user's field of view is taken as the subject direction.

In FIGS. 1 (a) and 1 (b), the imaging device 1 mounted on the user's head by the spectacle-type or head-mounted mounting unit has been described. However, the structure for the user to mount the imaging device 1 is as follows. Think variously. For example, a headphone type, a neckband type, an ear hook type, or any other mounting unit may be worn by the user. Further, for example, a configuration may be adopted in which the user wears the device by attaching the clip to a normal pair of glasses, a visor, headphones, or the like. Moreover, it does not necessarily have to be worn on the user's head.
In addition, although the imaging direction is the user's field of view, the configuration in which the imaging lens 3a is attached so as to capture the user's rear, side, upper, and foot directions when worn, and the imaging direction is the same or different. A configuration in which a plurality of imaging systems is provided is also conceivable.
Furthermore, an imaging direction variable mechanism that can change the subject direction manually or automatically may be provided for one or a plurality of imaging lenses 3a.

FIG. 1C shows an imaging apparatus 1 having a form generally known as a digital still camera.
FIG. 1D shows an imaging apparatus 1 having a form generally known as a video camera.
As shown in FIGS. 1C and 1D, the imaging apparatus 1 that the user has and uses is also an aspect of the embodiment of the present invention. Although only the imaging lens 3a is shown in the figure, a display device such as a panel display unit for an imaging monitor and a viewfinder is also provided.

Needless to say, forms other than those shown in FIGS. 1A, 1B, 1C, and 1D are also conceivable as imaging apparatuses that perform still image capturing and moving image capturing. For example, a device such as a mobile phone, a PDA (Personal Digital Assistant), or a portable personal computer that has a function as an image pickup device can be assumed as the image pickup device 1 of the present embodiment.
In these various forms, for example, a microphone that collects external sound may be provided, and an audio signal to be recorded together with image data may be obtained at the time of imaging. Further, a speaker unit and an earphone unit that perform audio output may be formed.
It is also conceivable that a light emitting unit that performs illumination in the direction of the subject is provided in the vicinity of the imaging lens 3a using, for example, an LED (Light Emitting Diode) or a flash light emitting unit for capturing a still image.

[2. Configuration example of imaging apparatus]

A configuration example of the imaging apparatus 1 will be described with reference to FIG.
The system controller 10 includes, for example, a microcomputer including a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a nonvolatile memory unit, and an interface unit, and controls the entire imaging device 1. The control unit The system controller 10 performs various arithmetic processes and exchanges control signals with each unit via the bus 13 based on a program held in an internal ROM or the like, and causes each unit to execute a required operation.

The imaging unit 3 includes an imaging optical system, an imaging element unit, and an imaging signal processing unit.
In the imaging optical system in the imaging unit 3, the imaging system 3a shown in FIG. 1, a lens system including an aperture, a zoom lens, a focus lens, and the like, and a focusing operation and a zooming operation are performed on the lens system. Drive system and the like.
In addition, the imaging device section in the imaging section 3 is provided with a solid-state imaging device array that detects imaging light obtained by the imaging optical system and generates an imaging signal by performing photoelectric conversion. The solid-state imaging device array is, for example, a CCD (Charge Coupled Device) sensor array or a CMOS (Complementary Metal Oxide Semiconductor) sensor array.
The imaging signal processing unit in the imaging unit 3 includes a sample hold / AGC (Automatic Gain Control) circuit that performs gain adjustment and waveform shaping on a signal obtained by the solid-state imaging device, and a video A / D converter, and serves as digital data. Captured image data is obtained. Also, white balance processing, luminance processing, color signal processing, blur correction processing, and the like are performed on the captured image data.

Imaging is performed by the imaging unit 3 including these imaging optical system, imaging element unit, and imaging signal processing unit, and image data obtained by imaging is obtained.
Image data obtained by the imaging operation of the imaging unit 3 is processed by the imaging control unit 6.
The imaging control unit 6 performs processing for converting the captured image data into a predetermined image data format under the control of the system controller 10, and converts the converted captured image data into an image evaluation unit 12, a storage unit 14, communication The process which supplies to the part 15 and the display control part 7 is performed.
The imaging control unit 6 controls on / off of the imaging operation in the imaging unit 3, drive control of the zoom lens and focus lens of the imaging optical system, and control of sensitivity and frame rate of the imaging element unit based on an instruction from the system controller 10. Then, parameter control of each process of the imaging signal processing unit, execution process setting, and the like are performed.

The image evaluation unit 12 performs image evaluation on the captured image data acquired by the imaging apparatus 1, that is, captured image data obtained by imaging by the imaging unit 3. The image evaluation is performed along predetermined evaluation items based on, for example, image analysis or various imaging situations. Then, evaluation data is generated as an evaluation result.
An example of image evaluation will be described later.
Note that captured image data that can be acquired by the imaging device 1 may be captured image data captured by an external imaging device. For example, captured image data captured by an external imaging device may be transmitted and received by the communication unit 15.
In addition, captured image data captured by an external imaging device is recorded on a portable recording medium, and the portable recording medium is reproduced by the storage unit 14 (when the storage unit 14 has a reproduction function for the portable recording medium). You may get it.
For these captured image data, the image evaluation unit 12 may perform image evaluation to generate evaluation data.

A display unit 2 and a display control unit 7 are provided as a configuration for displaying to the user in the imaging apparatus 1.
The display unit 2 is provided with a display panel unit 2a using the above-described liquid crystal panel and the like, and a display driving unit that drives the display panel unit 2a.
The display drive unit includes a pixel drive circuit for displaying the image data supplied from the imaging control unit 6 on the display panel unit 2a that is a liquid crystal display, for example. The pixel driving circuit applies a driving signal based on the video signal to each pixel arranged in a matrix in the display panel unit 2a at a predetermined horizontal / vertical driving timing, thereby causing display to be performed.

The display control unit 7 drives the pixel driving circuit in the display unit 2 based on the control of the system controller 10, and causes the display panel unit 2a to execute a predetermined display.
That is, the display panel unit displays a display as an imaging monitor in the imaging unit 3, a reproduction display of captured image data captured in the storage unit 14, a display of data received by the communication unit 15, a display of various characters, and the like. Run in 2a.
For these displays, for example, brightness level adjustment, color correction, contrast adjustment, sharpness (outline emphasis) adjustment, and the like can be performed. Also, generation of an enlarged image obtained by enlarging a part of the image data, generation of a reduced image, soft focus, mosaic, luminance inversion, highlight display (highlight display) of a part of the image, change in atmosphere of the whole color, etc. Image effect processing, separation and synthesis of images for divided display of captured images, generation of character images and image images, processing of combining the generated images with captured images, and the like can also be performed.
Further, the display control unit 7 controls the display driving unit based on an instruction from the system controller 10 and controls the transmittance of each pixel of the display panel unit 2a so as to be in a through state (transparent or translucent state). You can also.

The storage unit 14 is used for storing various data. For example, it is used for storing captured image data.
The storage unit 14 may be configured by a solid-state memory such as a RAM or a flash memory, or may be configured by an HDD (Hard Disk Drive), for example.
Further, instead of a built-in recording medium, a portable recording medium, for example, a recording / reproducing drive corresponding to a recording medium such as a memory card incorporating a solid-state memory, an optical disk, a magneto-optical disk, or a hologram memory may be used.
Of course, both a built-in type memory such as a solid-state memory and an HDD, and a recording / reproducing drive for a portable recording medium may be mounted.
The storage unit 14 records and stores captured image data based on the control of the system controller 10. In particular, the captured image data is recorded in association with the evaluation data generated by the image evaluation unit 12.
Further, based on the control of the system controller 10, the recorded data is read and supplied to the system controller 10, the display control unit 7, and the like.

The communication unit 15 transmits / receives data to / from an external device. The communication unit 15 may be configured to perform network communication via a short-range wireless communication with a network access point using a method such as a wireless LAN or Bluetooth, or between external devices having a corresponding communication function. It may be one that performs direct wireless communication.
Further, it is not necessarily wireless communication, and communication with an external device may be performed through a wired connection with the external device.

The imaging device 1 is provided with a date / time counting unit 16 and a position detection unit 17.
The date / time counting unit 16 counts the date / time (year / month / day / hour / minute / second) as a so-called clock unit, and outputs the current date / time information to the system controller 10. For example, when the imaging unit 3 performs imaging, the system controller 10 acquires information on the imaging date and time of the captured image data from the date and time counting unit 16.
The position detector 17 is, for example, a GPS receiver. The GPS receiver receives radio waves from a GPS (Global Positioning System) satellite, and outputs latitude / longitude information as the current position to the system controller 10.
The position detection unit 17 may use a location information service provided by WiFi (Wireless Fidelity) or a mobile phone company, or may be used in combination with GPS.

In the imaging apparatus 1, an operation input unit 11 is provided for user operation.
For example, the operation input unit 11 may include a controller such as a key and a dial to detect a user operation as a key operation or the like, or may detect a user's conscious behavior. .
In the case of providing an operator, for example, power on / off operation, imaging system operation (for example, shutter operation, zoom operation, signal processing instruction operation), display related operation (for example, selection of display contents and display adjustment operation) It is sufficient that an operator used for the above is formed.
For example, when an imaging apparatus as shown in FIGS. 1C and 1D is considered, it is preferable to perform various operations using an operator.
On the other hand, in the case of an imaging device mounted on the head as shown in FIGS. 1A and 1B, an operator may be provided. However, it is preferable to adopt a configuration in which a user's behavior is detected and regarded as an operation input. Also become.

In the case of a configuration for detecting user behavior, it is conceivable to provide an acceleration sensor, an angular velocity sensor, a vibration sensor, a pressure sensor, and the like.
For example, when the user taps the imaging device 1 from the side, the acceleration is detected by an acceleration sensor, a vibration sensor, or the like. For example, when the lateral acceleration exceeds a certain value, the system controller 10 recognizes the user operation. To be able to. Also, if it is possible to detect whether the user has struck the side part (the part corresponding to the eyeglass vine) from the right side or the side part from the left side by using an acceleration sensor or an angular velocity sensor, these can be used as predetermined operations. It can also be determined.
Further, it is possible to detect that the user turns his head or shakes his / her head with an acceleration sensor or an angular velocity sensor, and the system controller 10 can recognize this as a user operation.
Further, pressure sensors are arranged on the left and right side portions (portion corresponding to the eyeglasses) of the imaging apparatus 1, and when the user presses the right side with a finger, the zoom operation in the telephoto direction is performed. When pressed with, zoom operation in the wide-angle direction can be performed.

Furthermore, a configuration as a biosensor may be provided so that user biometric information is detected and recognized as an operation input. Biological information is pulse rate, heart rate, electrocardiogram information, electromyography, respiratory information (eg, respiratory rate, depth, ventilation), sweating, GSR (skin electrical response), blood pressure, blood oxygen saturation level Skin surface temperature, brain waves (for example, information on α wave, β wave, θ wave, and δ wave), blood flow change, eye condition, and the like.
The system controller 10 recognizes the detection information of the biosensor as a user operation input. For example, the user's conscious behavior may be an eye movement (change in line-of-sight direction or blinking), but if it is detected that the user has blinked three times, it is determined as a specific operation input. Furthermore, it is possible to detect that the user has attached or removed the imaging device 1 or that a specific user has attached by detecting the biological information, and the system controller 10 can turn on / off the power in response to the detection. You may make it perform off.
The biological sensor may be arranged inside the wearing frame of the eyeglass-type imaging device 1, for example, so as to be able to detect the various information by contacting the user's temporal region or the back of the head. It may be attached to a predetermined part of the body as a separate body from the one mounting frame portion.
In order to detect the state of the eye, the state of movement, the pupil, and the like, it is conceivable to use a camera that images the user's eyes.

The operation input unit 20 supplies information obtained as the operation element, the acceleration sensor, the angular velocity sensor, the vibration sensor, the pressure sensor, the biological sensor, and the like to the system controller 10, and the system controller 10 uses the information to operate the user. Is detected.
In addition, as a structure for detecting a user's behavior, other sensors, such as a sensor which detects a user's voice and a sensor which detects the motion of a lip, may be provided.

  The configuration of the imaging device 1 has been described above, but this is only an example. Addition and deletion of various components are naturally conceivable depending on the actual operation example and function.

FIG. 3 illustrates an operation mode of the imaging apparatus 1 in relation to an external device.
FIG. 3A shows a case where the imaging apparatus 1 operates alone. In this case, the imaging apparatus 1 stores captured image data acquired by, for example, an imaging operation in the storage unit 14. At this time, the evaluation data generated by the image evaluation unit 12 is also associated and stored.
The captured image data stored in the storage unit 14 can be displayed and output based on the evaluation data. Further, it can be deleted based on the evaluation data, or replaced with other captured image data.

FIG. 3B is an example in which the imaging apparatus 1 communicates with the external device 70 through the communication unit 15. In this case, the captured image data and the evaluation data can be transmitted to the external device 70 and stored in the image storage unit 71 in the external device 70 as the captured image data storage process. That is, it is a case where the imaging device 1 stores the captured image data and the evaluation data in the external device instead of the internal storage unit 14 (or in addition to the storage unit 14).
In addition, when the communication unit 15 communicates with the external device 70, it is assumed that captured image data is acquired from the external device 70. For example, when the external device 70 is an imaging device, or when the external device 70 stores captured image data captured by another imaging device, the captured image data is transmitted from the external device 70 and received by the communication unit 15. To do. As described above, the image evaluation unit 12 generates evaluation data for the captured image data received by the communication unit 15 and stores the captured image data and the evaluation data in association with each other in the storage unit 14 or the captured image data and the external device. An operation example in which evaluation data is transmitted in association with each other and stored in the image storage unit 71 can be considered.
Communication with the external device 70 is assumed not only wireless communication but also wired communication.

FIG. 3C illustrates an example in which the communication unit 15 of the imaging apparatus 1 has a communication access function via the network 60 such as the Internet, and communicates with the external device 70 connected via the network 60. An example of the operation can be considered as in the case of FIG.
In addition to the Internet, various types of networks 60 such as a mobile phone communication network, a PHS communication network, an ad hoc network, and a LAN can be considered.
Further, the external device 70 in FIGS. 3B and 3C can be assumed to be another imaging device 1 having a configuration as shown in FIG. 2, a data server, a personal computer, a video player, a video camera having a communication function, Various devices such as a digital still camera, a mobile phone having an imaging function, and a PDA can be assumed.

[3. Image evaluation]

An example of captured image data evaluation processing in the image evaluation unit 12 will be described.
The image evaluation unit 12 generates evaluation data by performing, for example, image analysis on the captured image data to be evaluated. Assume that the evaluation data is, for example, a value evaluated by an evaluation item indicating the degree of user satisfaction with the captured image. For example, image blur and blur, image contrast, color tone, sharpness, image angle deviation, and asymmetric object fog are evaluated. Also, when the subject is a person (possibly in the case of animals, etc.), analysis of the face part makes it possible to open and close eyes, facial expressions, face orientations, eyeball colors (such as red eyes during so-called strobe imaging) ) Etc.
The evaluation data based on these evaluation items is data indicating the presence / absence and degree of image content / state in which the user generally feels dissatisfied with a photograph or the like. In other words, the evaluation data is data indicating that the user feels that the image has been captured well or that the image has failed to be captured.
Note that the evaluation data may be generated based on the subjective criteria of the user. For example, the user inputs and sets an item as a subjective reference, and the image evaluation unit 12 generates evaluation data according to the setting.
Furthermore, when the above-described biosensor is provided, the user's emotion (satisfaction with the image) is estimated from the brain waves detected when the captured image data is presented to the user, and as a result, the user's subjectivity is met. Evaluation data may be generated.

4, 5 and 6 show examples of various captured images.
FIG. 4A shows an example of an image in a case where imaging is appropriately performed, that is, satisfactory for the user.
FIG. 4B is an image in which blurring has occurred.
FIG. 4C shows an example of a case in which the illuminance is insufficient and the entire image is dark.
FIG. 4D is an example of an image in which an asymmetric subject such as a user's finger is put on the imaging lens 3a.
Although not illustrated here, there may be a case where an image out of focus and blurred, an image with low definition, an image with an inappropriate hue, or the like is captured.

  FIG. 5A shows an image picked up at a correct angle, whereas FIG. 5B shows an example of an image in which an angle shift occurs. For example, in the head-mounted imaging device 1 as shown in FIGS. 1A and 1B, when the user tilts his / her neck at the time of imaging (shutter timing), the angle as shown in FIG. 5B Deviation occurs. Of course, even in the imaging apparatus 1 as shown in FIGS. 1C and 1D, an angle shift occurs depending on how the user holds the image.

  FIG. 6 illustrates the state of the captured facial expression when a person is a subject. FIG. 6A shows a case where a smiling image can be captured, but FIG. 6B shows an image with a strange expression when FIG. Shows the case. When taking a still image with a person as a subject, it is often the case that a good facial expression cannot be taken due to the shutter timing and instantaneous facial expression change. It may also be so-called red eyes or look sideways. For example, when photographing an infant's face, it is difficult to capture a desirable facial expression.

The image evaluation unit 12 performs image analysis of the captured image data as an image state as in these examples, and generates evaluation data.
For example, blurring, blurring, and sharpness of an image can be determined by detecting a contour line or a focus state in the image. In addition, when an angular velocity sensor, a vibration sensor, or the like is mounted, it is possible to use it for blur determination by storing the detection values of those sensors at the shutter timing.
The brightness of the image can be determined by calculating the luminance information of the entire pixel. As for the hue, the color trend of the entire screen is analyzed.
An image in which a finger, a strap, or the like is placed on the imaging lens 3a can be determined by whether or not there is a portion where extreme defocusing occurs in a certain range on the screen.
The angle shift can be determined by analyzing the verticality and horizontality of a building, natural object, person, etc. as a subject.
For the determination of the human face, the face portion is extracted from the image and analyzed. Whether the eye is opened or closed can be determined by the presence or absence of a pupil portion. The facial expression is estimated from, for example, a comparison with various types of facial expression patterns or the shape of eyes and mouth. The orientation of the face can be estimated from the positional relationship on the image such as the eyes, nose, mouth, and ears, the outline of the face, and the like.

The image evaluation unit 12 performs these processes to generate evaluation data.
The evaluation data is, for example, data indicating OK / NG, such as whether the image is a good captured image or whether the captured image is to be stored. Or it is good also as data which show evaluation in three steps or more. For example, it is an example in which the value of the evaluation data is 10 levels, 10 is determined to be the best, 1 is determined to be the worst. The evaluation data value may be a score.
The evaluation data may generate one evaluation data value as a comprehensive evaluation value based on the evaluation result of each evaluation item as described above for one captured image data, or the evaluation data for each evaluation item. A value may be generated.

[4. Image saving process]

FIG. 7 shows image storage processing when captured image data is acquired when an imaging operation is performed in the imaging unit 3 or the like. FIG. 7 shows a control process of the system controller 10.
When captured image data to be evaluated (saved) is acquired, the system controller 10 advances the process from step F101 to F102. The case where the captured image data is acquired is a case where the imaging operation is executed by the imaging unit 3.
3B and 3C, when the captured image data (image data captured by another imaging device) transmitted from the external device 70 by the communication unit 15 is received, or the portable recording When captured image data is reproduced from a portable recording medium in a configuration equipped with a recording / reproducing drive for the medium, it may be determined in step F101 that the captured image data to be evaluated is acquired.

For example, when the process proceeds to step F102 in accordance with the execution of the imaging operation in the imaging unit 3, the system controller 10 transfers the captured image data from the imaging control unit 6 to the image evaluation unit 12, for example, and causes the image evaluation unit 12 to perform image evaluation processing. Is executed. The image evaluation unit 12 evaluates captured image data along various evaluation items as described above. Then, evaluation data is generated as a result of the evaluation process.
In step F103, the system controller 10 takes in the evaluation data generated by the image evaluation unit 12. Also, image management information for the captured image data to be stored is generated. The image management information includes, for example, a file name of captured image data, an image type (moving image / still image distinction, etc.), a data size, a compression method, and identification information of a device that has performed imaging. It is also preferable to include date / time information (year / month / day / hour / minute / second at the time of imaging) counted by the date / time counting unit 16 and latitude / longitude information as an imaging location detected by the position detection unit 17. If the imaging direction can be detected, information on the imaging direction is also added. Further, for example, when a map database is stored in the storage unit 14 (or when the communication unit 15 can access a map database of an external device), the imaging location and subject name (location name, building name) that can be searched from latitude / longitude Etc.) may be included.

  In step F <b> 104, the system controller 10 causes the storage unit 14 to execute processing for storing captured image data. In this case, the evaluation data and the image management information are transferred to the storage unit 14 and recorded together with the captured image data.

When the captured image data to be stored is acquired, the above-described storage process is performed, so that the captured image data is stored in, for example, the storage unit 14 in the format shown in FIG.
For example, captured image data VD1, VD2,... VDn as still images, for example, are stored as entries # 1, # 2... #N, respectively, and image management information C1 corresponding to these image data, respectively. , C2... Cn are recorded, and evaluation data E1, E2.
That is, evaluation data is stored in association with each captured image data. As a result, the stored captured image data can be handled based on the evaluation data.

Although the example of saving in the storage unit 14 has been described here, as the saving process in step F104, a process of transmitting and outputting the image management information and the evaluation data from the communication unit 15 together with the captured image data may be performed. . That is, in the form as shown in FIGS. 3B and 3C, it may be considered that the image storage unit 71 of the external device 70 stores the image in the form as shown in FIG.

[5. Image display processing]

Next, image display processing as an example of handling based on evaluation data will be described.
FIG. 9 shows processing of the system controller 10 for displaying the stored captured image data on the display unit 3 when the captured image data is stored in the storage unit 14 as shown in FIG. Yes.

In step F201, the system controller 10 checks whether a display trigger has occurred. The display trigger generation is a case where a display output of a stored image is obtained by a user operation, for example. Alternatively, a case where a request for display output of a stored image is generated by some application program running on the system controller 10 corresponds to this.
When a display trigger occurs, the system controller 10 performs a process of sequentially displaying, for example, stored captured image data VD1, VD2,... VDn, and for this purpose, the process proceeds to step F202.

In step F202, evaluation data E (E1, E2... En) for each captured image data VD (VD1, VD2,... VDn) stored in the storage unit 14 as shown in FIG. It is determined whether or not the captured image data VD is an image to be displayed based on the evaluation data E, and the captured image data VD to be displayed is extracted. That is, from the value of the evaluation data E, only the captured image data VD that is evaluated highly is extracted.
There are various extraction methods. For example, when the evaluation data E is information indicating binary values of OK / NG, only the captured image data VD having a value indicating that the evaluation data E indicates “OK” may be extracted.
Further, when the evaluation data E indicates the evaluation result with, for example, 10 levels of 1 to 10, for example, the evaluation data E having a value of 5 or more is extracted, and the predetermined value is selected. As a result, the captured image data VD having a value of the evaluation data E equal to or higher than the selection value may be extracted. In this case, the selection value may be changed within a range of 1 to 10 by a user setting or the like. For example, when the user wants to carefully select and view only good photos, the selection value is set to a high value such as 8. In that case, the system controller 10 extracts a value of the evaluation data E of 8 or more in step F202. If the user wants to view as much captured image data VD as possible even if the user is not good, the selection value is changed to a low value such as 3. In that case, the system controller 10 extracts the evaluation data E having a value of 3 or more in step F202.
Further, when the evaluation data E is generated and stored individually for each evaluation item, the extraction may be performed using the evaluation data E for each evaluation item instead of the comprehensive evaluation value. For example, the evaluation items used in the extraction process can be specified in advance by the user or can be selected at the time of display. When the user wants to see a captured image without blurring or blurring, the evaluation data regarding the blurring or blurring is designated as an evaluation item used for the extraction process. In this case, the system controller 10 may extract the captured image data VD having a good value with reference to the evaluation data E regarding blurring and blurring in step F202.

After extracting the captured image data VD to be displayed based on the evaluation data E, the system controller 10 causes the display unit 2 to display the captured image data VD extracted in step F203.
For example, among the captured image data VD1, VD2, VD3, VD4, DV5... VDn stored as shown in FIG. 8, the values of the evaluation data E3 and E4 for the captured image data VD3 and VD4 are low, and these are extracted. If the captured image data VD1, VD2, DV5... VDn are extracted in step F202, display processing of these captured image data VD1, VD2, DV5. For example, these are sequentially displayed according to the user's operation, automatically displayed sequentially by the slideshow method, or displayed as a list with reduced images.

[6. Image deletion process]

Next, image deletion processing as an example of handling based on evaluation data will be described.
FIG. 10 illustrates a process of the system controller 10 that deletes what can be determined to be unnecessary from the stored captured image data when the captured image data is stored in the storage unit 14 as illustrated in FIG. Show.

In step F301, the system controller 10 checks the generation of the deletion trigger. The generation of a deletion trigger is a case where, for example, a user operation instructs to delete an image that can be determined to be low in user satisfaction among stored images. Alternatively, this also applies to a case where an unnecessary image deletion process request is generated by some application program running on the system controller 10.
If a deletion trigger has occurred, the system controller 10 advances the process to step F302.

In step F302, the evaluation data E (E1, E2... En) for each captured image data VD (VD1, VD2... VDn) stored in the storage unit 14 as shown in FIG. It is determined whether or not the captured image data VD is an image to be deleted based on the evaluation data E, and the captured image data VD to be deleted is extracted. That is, from the value of the evaluation data E, only the captured image data VD that is evaluated as being low evaluation is extracted.
Various extraction methods are conceivable as in the case of the above display processing.
For example, when the evaluation data E is information indicating binary values of OK / NG, only the captured image data VD having a value indicating that the evaluation data E indicates “NG” may be extracted.
Further, when the evaluation data E indicates the evaluation result with, for example, 10 levels of 1 to 10, for example, the evaluation data E is extracted by selecting a predetermined value, for example, when the evaluation data E has a value of 4 or less. As a result, the captured image data VD having a value of the evaluation data E equal to or higher than the selection value may be extracted. In this case, the selection value may be changed within a range of 1 to 10 by a user setting or the like. For example, when the user wants to carefully select and leave only images with very good evaluation, the selection value is changed to a high value such as 8. In that case, the system controller 10 extracts those whose evaluation data E value is 8 or less as a deletion target in step F302. Also, if the user wants to delete only those with a considerably low evaluation, the selection value is changed to a low value such as 3. In this case, the system controller 10 extracts the evaluation data E having a value of 3 or less in step F302.
Further, when the evaluation data E is generated and stored individually for each evaluation item, the extraction may be performed using the evaluation data E for each evaluation item instead of the comprehensive evaluation value. For example, the user can select an evaluation item used for the extraction process. When the user wants to delete a captured image in which an asymmetric subject is captured, the evaluation data regarding the presence of the asymmetric subject (presence / absence or captured range) is designated as an evaluation item used for the extraction process. . In that case, the system controller 10 may refer to the evaluation data E relating to the asymmetric subject in step F302 and extract the captured image data VD having a low value as a deletion target.

After extracting the captured image data VD to be deleted based on the evaluation data E, the system controller 10 causes the storage unit 14 to delete the captured image data VD extracted in step F303.
For example, among the captured image data VD1, VD2, VD3, VD4, DV5... VDn stored as shown in FIG. 8, the values of the evaluation data E3 and E4 for the captured image data VD3 and VD4 are low, and these are deleted. When extracted as a target, the captured image data VD3 and VD4 are deleted, and the storage contents of the storage unit 14 are as shown in FIG. The number of stored captured image data VD is nx obtained by subtracting the number of deleted images x.

When deletion is performed in step F303, it is preferable that the captured image data to be deleted is displayed on the display unit 2, the user is asked for permission to delete, and the deletion is executed after the user performs a permission operation. It is.
Further, considering that the deletion is executed with the permission of the user, it may be considered that the system controller 10 periodically performs the processes of steps F302 and F303 even if there is no deletion trigger by a user operation or the like. .

[7. Image replacement process I]

Next, image replacement processing as an example of handling based on evaluation data will be described.
The image replacement process is a process of replacing the captured image data VD having a low evaluation with other captured image data based on the evaluation data E, and storing the replaced image data. Here, an example in which replacement is performed with other captured image data stored in the image storage unit 71 of the external device 70 in the form shown in FIGS. 3B and 3C will be described.
FIG. 12 shows the case where captured image data is stored in the storage unit 14 as shown in FIG. 8 described above. The process of the system controller 10 for replacing with image data is shown. The processing of the external device 70 is also shown.

In step F401, the system controller 10 checks for the occurrence of a replacement trigger. The occurrence of a replacement trigger is, for example, a case where there is an instruction to replace an image that can be determined to be low in user satisfaction among other stored images by a user operation. Alternatively, this also applies to a case where a request for replacement processing for a low evaluation image is generated by some application program running on the system controller 10.
If a replacement trigger has occurred, the system controller 10 advances the process to step F402.

  In step F402, the evaluation data E (E1, E2... En) for each captured image data VD (VD1, VD2... VDn) stored in the storage unit 14 as shown in FIG. The captured image data VD to be replaced is extracted based on the evaluation data E for the captured image data VD. That is, from the value of the evaluation data E, the captured image data VD that is evaluated as being low evaluation is extracted. Various extraction methods are conceivable as in the case of the image deletion process.

After extracting the replacement target captured image data VD based on the evaluation data E, the system controller 10 executes replacement processing for each extracted captured image data VD in step F403 and subsequent steps.
First, in step F403, the system controller 10 generates image request data for requesting replacement image data for one captured image data VD to be replaced. This image request data includes date information and position information included in the image management information C for the captured image data VD. Furthermore, you may make it include information, such as an imaging direction and the name of an imaging point. The captured image data VD itself may also be included.

After generating the image request data, the system controller 10 transmits the image request data from the communication unit 15 to the external device 70 in step F404.
In steps F405 and F406, the transmission from the external device is waited.

On the external device 70 side, when the image request data from the imaging device 1 is received, the process proceeds from step F451 to F452, and an image search is performed on the captured image data stored in the image storage unit 71.
For example, when the external device 70 is an imaging device, captured image data captured by the external device 70 is stored in the image storage unit 71. Further, when the external device 70 is a server device such as an image sharing service system, captured image data taken by a large number of imaging devices is stored.
The external device 70 searches for captured image data that matches the image request data from the imaging device 1 among the captured image data stored in the image storage unit 71. That is, an image suitable for replacement is searched.
Since the image request data from the imaging apparatus 1 includes position information and date / time information at the time of imaging, the external device 70 searches for images having the same or similar position information and date / time information. For example, it is searched whether or not there is captured image data taken at the same place at almost the same date and time. If it does not exist, the search is performed by expanding the date and time range and the position range.
If there is an image in which the position information and the date / time information at the time of image capturing and the image capturing directions match or are close to each other, the image is used as a replacement image.
Regarding position information, not only latitude and longitude, but also information on the imaging direction may be added to the search condition, or name information such as a place name and a building name may be added to the search condition.
The date and time information may be searched not only based on the date and time being close but also on the condition that the time of the season or one day is common.
Further, when the image request data includes the captured image data itself to be replaced, the external device 70 compares the image contents themselves when an image having the same or similar position information or date / time information is searched. The similarity of the image contents is judged, and if they are similar, a replacement image may be used. The determination of the similar image can be based on whether the contour line (image edge) is close.
When the captured image data to which the evaluation data E is added is stored in the image storage unit 71 of the external device 70, the value of the evaluation data E is high among the captured image data extracted based on the position and date / time conditions. You may make it select a thing.

If an image suitable for replacement exists in the image storage unit 71, the processing of the external device 70 proceeds from step F453 to F454, and the captured image data searched for replacement is transmitted to the imaging device 1. .
On the other hand, if there is no image suitable for replacement in the image storage unit 71, the process proceeds from step F453 to F455, and a notification indicating that there is no replacement image is transmitted to the imaging apparatus 1.

When the replacement captured image data is transmitted from the external device 70 and received by the communication unit 15, the system controller 10 of the imaging apparatus 1 advances the process from step F 405 to F 408, and one captured image to be replaced A process of replacing the data VD with the received captured image data is performed. That is, the storage unit 14 rewrites the captured image data VD.
On the other hand, when the notification that the captured image data for replacement does not exist is transmitted from the external device 70 and received by the communication unit 15, the process proceeds from step F406 to F407, and the replacement is not possible. The image data VD is not rewritten.

In step F409, it is determined whether or not the processing in steps F403 to F408 has been completed for all the captured image data VD extracted as the replacement target in step F402. If there remains a replacement target captured image data VD that has not yet been processed, the process proceeds to step F403, and the processes of steps F403 to F408 are similarly performed on the captured image data VD.
When the above processing is completed for all the captured image data VD to be replaced, the processing in FIG.

For example, among the captured image data VD1, VD2, VD3, VD4, DV5... VDn stored as shown in FIG. 8, the values of the evaluation data E3 and E4 for the captured image data VD3 and VD4 are low, and these are replaced. When extracted as a target, the processes of steps F403 to F408 are performed on the captured image data VD3 and VD4, respectively.
Assuming that the replacement captured image data is transmitted from the external device 70 for each of the captured image data VD3 and VD4, the storage contents of the storage unit 14 are as shown in FIG. That is, each of the captured image data VD3 and VD4 is rewritten to the transmitted captured image data VD3new and VD4new. In response to this, the image management information C3 and C4 are also rewritten to image management information C3new and C4new corresponding to the new captured image data VD3new and VD4new.
The evaluation data is updated to information dp indicating that the image is a replaced image. Note that the evaluation data E may be generated by the image evaluation unit 12 for the transmitted captured image data VD3new and VD4new, and the new evaluation data E3new and E4new may be recorded. Alternatively, when the captured image data to which the evaluation data E is added is stored in the image storage unit 71 of the external device 70 and the evaluation data is transmitted together with the captured image data, the evaluation data is recorded. You may do it.

In the processing example of FIG. 12, image data for replacement is selected on the external device 70 side. However, selection processing of captured image data to be replaced by the imaging apparatus 1 can also be performed. For example, a plurality of captured image data extracted according to the position and date / time conditions are transmitted from the external device 70 to the imaging device 1, and the captured image data VD to be replaced is transmitted on the imaging device 1 side. The captured image data to be replaced can be selected based on the similarity of the contour lines.

[8. Image replacement process II]

Next, another example of image replacement processing as an example of handling based on evaluation data will be described. This image replacement process is a process of replacing the captured image data VD having a low evaluation with another captured image data based on the evaluation data E, and here, the imaging apparatus 1 internally performs another imaging. An example of performing replacement with image data will be described.

In this case, the imaging unit 3 is configured to capture a plurality of images by the high-speed shutter function when the user performs a shutter operation.
For example, if the captured image data VD1 is captured at the shutter operation timing as the user's still image capturing operation, several captured images are also captured at intervals of several milliseconds to several hundred milliseconds.
For example, FIG. 15 shows an example of saving the captured image data VD in the storage unit 14. However, the image management information C1, C2,... And the evaluation data E1, E2... Are recorded in correspondence with the captured image data VD1, VD2,. The image captured in step 1 is recorded as sub image data in association with each other.
For example, the sub image data VD1s1, VD1s2, and VD1s3 are stored for the captured image data VD1. Similarly, when the captured image data VD2 is captured and recorded, the sub image data VD2s1, VD2s2, and VD2s3 captured by the high-speed shutter function at that time are also recorded.
For example, on the premise that a plurality of pieces of sub-image data that are substantially simultaneous as described above are stored, the following image replacement process can be performed.

  14, when captured image data is stored in the storage unit 14 as illustrated in FIG. 15, captured image data having a low evaluation among the stored captured image data VD1, VD2,. The process of the system controller 10 for replacing with other captured image data (sub-image data) is shown.

In step F501, the system controller 10 checks the occurrence of a replacement trigger in the same manner as in step F401 in FIG.
If a replacement trigger has occurred, the system controller 10 advances the process to step F502, and extracts captured image data VD to be replaced. That is, the evaluation data E1, E2... En for each captured image data VD1, VD2,... VDn stored in the storage unit 14 as shown in FIG. The captured image data VD having a low value is extracted as a replacement target. Various extraction methods are conceivable as in the case of the image deletion process.

After extracting the replacement target captured image data VD based on the evaluation data E, the system controller 10 executes a replacement process for each extracted captured image data VD in step F503 and subsequent steps.
First, in step F503, the image evaluation unit 12 generates evaluation data for each sub image data corresponding to one captured image data VD to be replaced.
For example, when the captured image data VD3 is to be replaced, evaluation data is generated for each of the sub image data VD3s1, VD3s2, and VD3s3.
Next, in step F504, it is determined whether there is an evaluation data value suitable as a stored image among the sub image data.
If it does not exist, the replacement process is disabled in step F506.
If it exists, a replacement process for rewriting the image data VD to sub-image data is performed in step F505.

For example, if the sub image data VD3s1, VD3s2, and VD3s3 for the captured image data VD3 to be replaced are all evaluation data values that are the same as or lower than the captured image data VD3, the image is suitable for storage. In step F506, the replacement is impossible.
On the other hand, if sub image data VD3s1, VD3s2, and VD3s3 have a higher evaluation data value than captured image data VD3, a replacement process is performed in step F505. If a plurality of sub image data VD3s1, VD3s2, and VD3s3 has a higher evaluation data value than the captured image data VD3, the highest sub image data may be used as a replacement image.
However, even if the sub image data VD3s1, VD3s2, and VD3s3 have an evaluation data value higher than the captured image data VD3, the evaluation data value is a value within the range to be replaced in step F502. If so, it may be determined that there is no sub-image data for replacement.

In step F507, it is determined whether or not the processing in steps F503 to F506 has been completed for all the captured image data VD extracted as the replacement target in step F502. If captured image data VD to be replaced that has not yet been processed remains, the process returns to step F503, and the processes of steps F503 to F506 are similarly performed on the captured image data VD.
When the above processing is completed for all the captured image data VD to be replaced, the process proceeds from step F507 to F508, the sub-image data not used in the replacement processing in step F505 is deleted, and the processing of FIG. To do.

For example, among the captured image data VD1, VD2, VD3, VD4, DV5... VDn stored as shown in FIG. 15, the values of the evaluation data E3 and E4 for the captured image data VD3 and VD4 are low, and these are replaced. When extracted as a target, the processes of steps F503 to F506 are performed on the captured image data VD3 and VD4, respectively.
If sub image data suitable for replacement exists for each of the captured image data VD3 and VD4, the storage content of the storage unit 14 is as shown in FIG. That is, the captured image data VD3 is rewritten to the sub image data VD3s1 having a higher evaluation data value, and the captured image data VD4 is rewritten to the sub image data VD4s3 having a higher evaluation data value. As the evaluation data E, the evaluation data for the sub image data VD3s1 and VD4s3 generated in step F503 is recorded as E3new and E4new.
Also, the sub image data and the original captured image data VD3 and VD4 that are the replacement targets are deleted by the processing in step F508.

Note that the deletion process in step F508 may not be executed, and the sub image data may be stored as it is.
In the processing of FIG. 14, the example in which the captured image data VD having a low evaluation data value is the replacement target has been described. However, all the captured image data VD1, VD2,... VDn stored as illustrated in FIG. You may perform the process of step F503-F506 as object. Then, all the captured image data VD1, VD2,... VDn compare the sub image data and the evaluation data value, respectively, and leave the image data having the best evaluation data value as the main captured image data VD. Can do.

[9. Effect of embodiment, modification and extension example]

According to the above embodiment, evaluation data serving as a reference for user satisfaction is generated for captured image data acquired by imaging or the like, and the captured image data is stored together with the evaluation data. Thereby, the stored captured image data can be handled appropriately and conveniently.
Evaluation items include blurring and blurring of the image, lightness and darkness, hue and sharpness of the image, angle deviation of the image, fogging of the asymmetrical subject, opening and closing of eyes, facial expression, face orientation, eyeball color, etc. In general, the evaluation can be performed in accordance with the degree of satisfaction / dissatisfaction that the user feels about the photographic image.

  Then, according to the image display processing as shown in FIG. 8, only the captured image data with a high evaluation is selectively displayed based on the evaluation data representing the satisfaction / dissatisfaction level of the user. A highly convenient display operation can be realized.

Further, according to the image deletion processing as shown in FIG. 10, for example, it is possible to delete captured image data having a low evaluation as unnecessary. Particularly when used by a user who frequently performs imaging operations, or when assuming an imaging device 1 that is mounted as shown in FIGS. A large number of captured image data VD are stored in the storage unit 14, but the image deemed inappropriate by the image deletion processing of FIG. 10 is deleted, and the arrangement of stored images proceeds, which is suitable for the user. .
For example, when undesired images such as out-of-sensitivity, shutter speed, camera shake, vibration, etc. are taken, such as out-of-focus images, blurred images, dark and invisible images, images with poor facial expressions, etc. Is automatically deleted, and as a result, an image album satisfying the user can be created.

According to the image replacement processing I as shown in FIG. 12, a captured image with a low evaluation can be replaced with an image transmitted from an external device 70 such as another imaging device or a server device of an image sharing site on a network. . An image transmitted from the external device 70 is an image suitable for replacement by being an image that approximates an imaging location, date and time, an imaging direction, or an image that approximates the imaging content. For this reason, the user of the imaging device 1 can obtain a better image, for example, for an image (scene) that he / she thinks has failed to photograph.
For example, people often take pictures at theme parks and landscapes such as trips in Japan and abroad, so there are often taken images at the same place, at the same angle, and at almost the same time. Are many and useful.

According to the image replacement processing II as shown in FIG. 14, an image with a low evaluation can be replaced with a preferable preferable image from a series of images taken by the user, and the satisfaction of the user can be improved.
For example, when a person is a subject, it is difficult to capture a satisfactory image due to a change in facial expression, but a good expression can be achieved by storing several sub-image data captured with the high-speed shutter function. Opportunities to capture the moment will increase. For example, by performing replacement based on evaluation data relating to facial expressions, a captured image with high satisfaction can be obtained.
In addition, when capturing multiple captured image data almost simultaneously using the high-speed shutter function, the brightness level may be different in terms of signal processing or color tone at each capture timing of each captured image data (sub-image data). Or the sensitivity may be changed. Further, the focus lens may be moved slightly during the period of taking in the series of image data. Then, the captured image data VD and the sub image data corresponding to the captured image data VD are different in brightness, hue, brightness, focus, and the like. For this reason, a plurality of images having different evaluation data values for these evaluation items will be stored, and in the replacement process, the optimum one such as brightness, hue, brightness, focus, etc. will be selected. Become.

Note that the present invention is not limited to the above-described embodiment, and various modifications and expansion examples are conceivable as configuration examples and processing examples of the imaging apparatus 1.
In addition to realizing the image storage device of the present invention as the imaging device 1, various devices are assumed.
FIG. 17 shows a system controller 10 as a control means, an image acquisition unit 20, an image evaluation unit 12, a storage processing unit 22, and an image output unit 21 as components of the image storage apparatus of the present invention. Yes.
For example, the image storage device of the present invention can be realized as various devices including such components.

The image acquisition unit 20 is a part that generates evaluation data in the image evaluation unit 12 and acquires captured image data to be stored, and corresponds to the imaging unit 3 and the imaging control unit 6 in the above embodiment. In addition to the part as such an imaging function, for example, the communication unit 15 that receives captured image data to be subjected to evaluation processing and storage processing from the external device 70 can also be the image acquisition unit 20. Of course, the captured image data from the external device 70 may be received by cable connection. Furthermore, a playback drive that performs playback on a portable recording medium such as an optical disk or a memory card and reads captured image data recorded on the portable recording medium can also be the image acquisition unit 20.
The captured image data acquired by the image acquisition unit 20 may be still image data or moving image data.

The storage processing unit 22 is a part that performs processing for storing captured image data and evaluation data in association with each other, and corresponds to the storage unit 14 of the above embodiment.
However, when the captured image data and the evaluation data are transmitted to the external device 70 and the external device 70 performs storage, the communication unit 15 that performs the transmission corresponds to the storage processing unit 22. Of course, either wireless transmission or wired transmission may be used.

In the case of the above-described embodiment, the image output unit 21 corresponds to the display unit 2 that displays and outputs stored captured image data. An output unit that outputs an image signal for display on an external monitor device can also be the image output unit 21.
The communication unit 15 that transmits the stored captured image data to the external device 70 by wire or wirelessly can also be the image output unit 21.
The image output unit 21 can also be a recording drive that records the stored captured image data on a portable recording medium and allows the portable recording medium to be taken out of the apparatus.

  For example, as the image acquisition unit 20, the image output unit 21, and the storage processing unit 22, very various forms are assumed. Considering them, the present invention can be realized not only as an imaging apparatus such as a still camera or a video camera, but also as a monitor display apparatus, a communication apparatus, an image data storage dedicated apparatus or the like without an imaging function.

It is explanatory drawing of the example of an external appearance of the imaging device of embodiment of this invention. It is a block diagram of the imaging device of an embodiment. It is explanatory drawing of the usage condition of the imaging device of embodiment. It is explanatory drawing of the various examples of a captured image. It is explanatory drawing of the various examples of a captured image. It is explanatory drawing of the various examples of a captured image. It is a flowchart of the image preservation | save process of embodiment. It is explanatory drawing of the preservation | save state of the captured image data of embodiment. It is a flowchart of the image display process of an embodiment. It is a flowchart of the image deletion process of an embodiment. It is explanatory drawing of the preservation | save state of the captured image data after the deletion process of embodiment. It is a flowchart of image substitution processing I of an embodiment. It is explanatory drawing of the preservation | save state of the captured image data after the replacement process of embodiment. It is a flowchart of image substitution processing II of an embodiment. It is explanatory drawing of the preservation | save state of the captured image data before the replacement process of embodiment. It is explanatory drawing of the preservation | save state of the captured image data after the replacement process of embodiment. It is explanatory drawing of the component of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Imaging device, 2 Display part, 2a Display panel part, 3 Imaging part, 6 Imaging control part, 7 Display control part, 10 System controller, 11 Operation input part, 12 Image evaluation part, 14 Storage part, 15 Communication part, 16 Date and time counting unit, 17 position detection unit, 20 image acquisition unit, 21 image output unit, 22 storage processing unit

Claims (18)

Acquisition means for acquiring captured image data;
Image evaluation means for performing image evaluation on a predetermined evaluation item and generating evaluation data;
Storage processing means for storing captured image data;
When captured image data to be evaluated is acquired by the acquisition unit, the image evaluation unit generates evaluation data for the captured image data, and the captured image data and the evaluation data are stored in association with each other. Control means for causing the storage processing means to execute storage processing;
An image storage device comprising:   The image storage device according to claim 1, wherein the acquisition unit includes an imaging unit that captures an image, and acquires captured image data obtained by imaging with the imaging unit.   The image storage device according to claim 2, wherein the imaging unit is configured using a CCD sensor or a CMOS sensor as an imaging element.   The image storage device according to claim 1, wherein the acquisition unit includes a communication unit that communicates with an external device, and receives and acquires captured image data through communication with the external device at the communication unit. .   The said acquisition means has a reproducing part which reproduces | regenerates information from a portable recording medium, and reproduces | regenerates the said portable recording medium by the said reproducing part, and acquires picked-up image data, It is characterized by the above-mentioned. Image storage device.   The image storage device according to claim 1, wherein the image evaluation unit performs an image evaluation as to whether to adopt or store captured image data to be evaluated.   The image storage device according to claim 1, wherein the image evaluation unit performs image evaluation in a stage or a score.   The image evaluation means is the evaluation item including, for the captured image data to be evaluated, the degree of blur, the degree of blurring, the degree of brightness, the hue, the hue, the sharpness, the angle shift, or the fog of an asymmetric subject. The image storage apparatus according to claim 1, wherein image evaluation is performed as:   The image evaluation unit determines whether or not the captured image data to be evaluated is a person being a subject, and if the person is a subject, The image storage apparatus according to claim 1, wherein image evaluation is performed using all or a part of eye opening / closing, facial expression, facial orientation, or eyeball color as the evaluation items.   The storage processing unit includes a recording unit that records information on a recording medium, and records the captured image data and evaluation data in the recording medium in association with each other as the storage processing. 2. The image storage device according to 1.   The image according to claim 1, wherein the storage processing unit includes a communication unit that communicates with an external device, and the captured image data and the evaluation data are transmitted to the external device in association with each other as the storage processing. Storage device.   2. The control unit according to claim 1, wherein the control unit further performs a deletion process for deleting the captured image data stored by the storage processing unit based on the evaluation data stored correspondingly. Image storage device.   The control means further performs a replacement process for replacing the captured image data stored by the storage processing means with other captured image data based on the corresponding stored evaluation data. The image storage device according to claim 1. Furthermore, an image output means for performing image output processing is provided,
The control means further performs processing for causing the image output means to output the captured image data saved by the saving processing means based on the correspondingly stored evaluation data. Item 2. The image storage device according to Item 1.   The image storage device according to claim 14, wherein the image output unit includes a display unit, and performs display output of the captured image data in the display unit as the output process.   The said image output means has a communication part which communicates with an external apparatus, and performs the transmission output to the external apparatus about captured image data by the said communication part as the said output process, The said output part is characterized by the above-mentioned. Image storage device.   15. The image output means includes a recording unit that records information on a portable recording medium, and the output processing records captured image data on the portable recording medium as the output process. The image storage device described in 1. Obtaining captured image data to be evaluated;
Performing image evaluation on a predetermined evaluation item for the captured image data and generating evaluation data;
Executing a storage process so that the captured image data and the evaluation data are stored in association with each other;
An image storage method comprising:

Images