JP5041050B2 - Imaging apparatus and image processing program - Google Patents

Description

  The present invention relates to an imaging device that captures a subject image and generates an image, and an image processing program for realizing image processing on an image to be processed by a computer.

  In an image generated by an imaging device such as an electronic camera, for the purpose of searching or editing, a technique for extracting feature quantities such as motion information and shape information of a subject included in the image is considered (for example, Patent Document 1) below.

Japanese Patent Application Laid-Open No. 5-46765

  An image generated by an imaging device such as an electronic camera may include geometric distortion caused by a photographing optical system such as a photographing lens. Such geometric distortion includes distortion due to the presence of undesirable aberrations in the photographing optical system and distortion intentionally added by the user for the purpose of video effects. Examples of the latter include, for example, an image generated by photographing using a fisheye lens. In any case, feature values extracted from images that contain geometric distortion are different in nature from feature values extracted from images that do not contain geometric distortion, and are therefore inappropriate for image search and editing. There are many cases.

  It is an object of the present invention to extract an appropriate feature amount from an image and accurately search and edit the image.

The imaging device of the present invention is an imaging device including an imaging unit that captures an image of a subject via a photographing lens and generates an image, and an acquisition unit that acquires lens information of the photographing lens, and shape information for each subject From a shape storage unit that stores the image in advance, an image correction unit that corrects geometric distortion of the image generated by the imaging unit based on the lens information, and the image that has been corrected by the image correction unit, Based on a shape extraction unit that extracts shape information of a subject included in the image, the shape information extracted by the shape extraction unit from the corrected image, and the shape information stored in the shape storage unit A specifying unit that specifies the type of the subject included in the image generated by the imaging unit, the image generated by the imaging unit, and the type of the subject specified by the specifying unit A recording unit for associating recorded, on the basis of the type of the subject, Ru and an image extraction unit configured to extract an image having a predetermined feature from the recorded the image on the recording unit.

Note that the image correction unit may correct the geometric distortion by performing aberration correction on an image having a lower pixel number than an image obtained by still image shooting by the imaging unit.
Further, the image generated by the imaging unit and the type of the subject are recorded in the recording unit in association with each other, or the image corrected by the image correcting unit is used instead of the image generated by the imaging unit. And a selection unit for selecting whether to record in association with the subject type.

An image processing program of the present invention is an image processing program for executing image processing on an image to be processed by a computer, and includes an image to be processed and lens information of a photographing lens used at the time of generating the image. An acquisition procedure to be acquired, an image correction procedure for correcting geometric distortion of the image to be processed based on the lens information, and an image to be processed from the image to be processed after the correction by the image correction procedure. Based on a shape extraction procedure for extracting shape information of included subjects, the shape information extracted by the shape extraction procedure from the corrected image to be processed , and shape information for each subject stored in advance a specifying step for specifying the type of the subject included in the image of the processing target, the image of the processing target, the object scene that is specified by the specifying steps Comprising of a recording procedure for recording in association with the type, based on the type of the object, and an image extracting procedure of extracting an image having a predetermined feature from the image of the processing object recorded by the recording procedure.

  According to the imaging apparatus and the image processing program of the present invention, it is possible to extract an appropriate feature amount from an image and perform accurate search and editing of the image.

It is a functional block diagram of electronic camera 1 of a 1st embodiment. It is a flowchart which shows operation | movement of the electronic camera 1 of 1st Embodiment. It is a figure explaining a distortion aberration. It is a flowchart which shows operation | movement of the electronic camera 1 of 2nd Embodiment.

<First Embodiment>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. In the first embodiment, an electronic camera will be described as an example.

  FIG. 1 is a functional block diagram of an electronic camera 1 according to the first embodiment of the present invention. As shown in FIG. 1, the electronic camera 1 includes a replaceable photographing lens 11, an image sensor 12, a lens driving circuit 13, a control circuit 14, an operation member 15, an image sensor driving circuit 16, a signal processing circuit 17, and a data processing circuit. 18, compression / expansion processing circuit 19, monitor 20, display control circuit 21, photometry circuit 22, shake detection sensor 23, and recording medium 24.

  The photographic lens 11 is composed of a plurality of lens groups constituting a photographic optical system, and forms a subject image on the imaging surface of the imaging element 12. The photographic lens 11 includes a focus lens (not shown), and the lens drive circuit 13 drives the focus lens to advance and retract in the optical axis direction, so that the focus of the photographic lens 11 is adjusted. The photographing lens 11 includes a zoom lens (not shown), and zoom adjustment (change of focal length) of the photographing lens 11 is performed by the lens driving circuit 13 driving the zoom lens forward and backward in the optical axis direction. The lens driving circuit 13 generates a lens driving signal in accordance with a lens driving command output from the control circuit 14, and moves each lens by driving a lens driving mechanism (not shown) using the generated lens driving signal. Let

  In addition, the photographing lens 11 is configured to be able to communicate with the control circuit 14 and transmits a focal length, a zoom magnification, an aperture value (not shown), aberration information described later, and the like to the control circuit 14 at the time of photographing. can do.

  The image sensor 12 can capture a moving image as well as a still image. The image sensor 12 is configured by, for example, a CCD image sensor or a CMOS image sensor.

  The image sensor drive circuit 16 generates a drive signal at a predetermined timing in response to a command output from the control circuit 14 and supplies the generated drive signal to the image sensor 12. The image sensor 12 controls charge accumulation (imaging) and reading of accumulated charges by the supplied drive signal. The control circuit 14 obtains information on the brightness of the object scene using the photometry data of the subject obtained by the photometry circuit 22, and based on the information on the brightness, the charge accumulation time of the image sensor 12, the aperture of the photographing lens 11, and The amplification degree of the image signal output from the image sensor 12 is determined. The information about the brightness of the object scene may be obtained from a signal output from the image sensor 12. In this case, the image sensor 12 controls the function of the photometry circuit 22.

  The image signal read from the image sensor 12 is input to the signal processing circuit 17. The signal processing circuit 17 performs signal processing such as amplification, DC reproduction, A / D conversion, white balance, and gamma conversion on the input signal in accordance with a command from the control circuit 14, and the signal-processed data is used as image data. The data is output to the data processing circuit 18.

  The data processing circuit 18 is necessary for outputting the image data output from the signal processing circuit 17 to the compression / decompression processing circuit 19 and displaying the reproduced image on the monitor 20 in response to a command from the control circuit 14. A resolution conversion (pixel number conversion) process is performed on the image data, and the image data after the resolution conversion process is output to the display control circuit 21. Note that the data processing circuit 18 performs resolution (number of pixel) conversion processing on input image data when performing electronic zoom processing, and outputs it to the compression / decompression processing circuit 19 and the display control circuit 21, respectively. To do.

  The data processing circuit 18 receives lens setting information (focal length, zoom magnification, etc.) at the time of shooting and aberration information of the shooting lens 11 from the control circuit 14, and aberrations for the image data input from the signal processing circuit 17. Correction is performed (details will be described later). Further, the data processing circuit 18 has a storage circuit for storing dictionary data in which the relationship between the type of subject and various feature amounts is recorded, and the type of subject is determined based on the feature amount calculated for each image. Specify (details will be described later).

  The compression / decompression processing circuit 19 performs compression processing on the image data input from the data processing circuit 18 in a predetermined format in response to a command from the control circuit 14 and records the compressed image data on the recording medium 24. . When image data is recorded on the recording medium 24 at the time of shooting, a reproduced image corresponding to the image data to be recorded is displayed on the monitor 20. When the operation member 15 instructs to record the image data without compression, the compression / decompression processing circuit 19 records the image data on the recording medium 24 without performing the compression process. Also in this case, a reproduced image corresponding to the image data to be recorded is displayed on the monitor 20.

  Further, the electronic camera 1 is configured to be able to display a reproduction image based on the image data recorded on the recording medium 24 on the monitor 20 (reproduction mode). In this case, the compression / decompression processing circuit 19 reads the image data recorded on the recording medium 24 in response to a command from the control circuit 14, performs a decoding process on the read data, and then performs the decoding process. The image data is sent to the data processing circuit 18. During reproduction, the data processing circuit 18 performs resolution conversion processing on the decoded image data and outputs it to the display control circuit 21, whereby the reproduced image is displayed on the monitor 20. Note that when uncompressed image data recorded in the recording medium 24 is read, the decoding process, which is the reverse process of the compression process, is not performed. The compression / decompression processing circuit 19 is configured to be able to perform lossless compression (so-called lossless encoding).

  In response to a command from the control circuit 14, the display control circuit 21 performs predetermined signal processing on the image data input from the data processing circuit 18 and outputs it to the monitor 20. The display control circuit 21 further performs a process of superimposing overlay image data such as a shooting menu and a cursor on the image data. Thereby, the subject image on which the overlay image is superimposed is displayed on the monitor 20.

  The operation member 15 includes a release button, a REC button, and the like, and outputs an operation signal corresponding to the operation content to the control circuit 14. When a release operation signal based on the release button pressing operation is input from the operation member 15, the control circuit 14 sets a focus detection area set in advance in the imaging screen in the image signal read from the imaging device 12. A known contrast AF (autofocus) operation is performed using the corresponding signal.

  Specifically, the lens drive is performed so as to maximize the integrated value (so-called focus evaluation value) of the high frequency component for the image data corresponding to the focus detection area among the image data processed by the signal processing circuit 17. A command (focus adjustment signal) is sent to the lens driving circuit 13. The position of the focus lens that maximizes the focus evaluation value is a focus position that eliminates blurring of the edge of the subject image captured by the image sensor 12 and maximizes the contrast of the image (increases sharpness). In addition to the contrast AF operation, a phase difference AF operation by a known pupil division method may be performed.

  The operation member 15 further includes a zoom operation member. When a zoom operation signal based on the zoom operation is input from the operation member 15, the control circuit 14 generates the lens drive command described above, and causes the lens drive circuit 13 to drive the zoom lens forward and backward. Thereby, the subject image formed on the imaging surface of the image sensor 12 is enlarged or reduced, and the zoom is optically adjusted.

  Further, when a zoom operation signal based on the zoom operation is input from the operation member 15, the control circuit 14 outputs a command to the data processing circuit 18 and changes the conversion ratio of the resolution conversion process for the image data in accordance with the operation signal. Thereby, the image displayed on the monitor 20 is enlarged or reduced, and the zoom is electrically adjusted (electronic zoom). The resolution conversion ratio corresponds to the electronic zoom magnification. When the data processing circuit 18 changes the conversion ratio in the direction of increasing the electronic zoom magnification, a part of the reproduced image is enlarged and displayed on the monitor 20. In this case, the enlargement ratio is increased, but the display range of the reproduced image is narrowed. On the other hand, when the data processing circuit 18 changes the conversion ratio in the direction of decreasing the electronic zoom magnification, the enlargement ratio of the reproduced image displayed on the monitor 20 is lowered, but the display range of the reproduced image is widened. When the photographing mode by the image sensor 12 is set, the image data of the photographed image corresponding to the image displayed on the monitor 20 can be recorded on the recording medium 24.

  The shake detection sensor 23 includes, for example, an angular velocity sensor, a gyro sensor, and the like, is provided in the casing of the electronic camera 1, and detects the shake of the casing in two directions orthogonal to each other. When shake occurs in the housing based on the operation of the release button of the operation member 15, the shake detection sensor 23 detects this and outputs shake amount data in two orthogonal directions to the control circuit 14. The control circuit 14 includes a camera shake correction optical system built in the photographing lens 11 and a lens driving circuit 13 so that the movement of the subject image formed on the imaging surface of the imaging element 12 due to the shake of the housing is canceled. Drive through. Thus, so-called camera shake correction is realized.

  The recording medium 24 includes a memory card, a small hard disk, an optical disk such as a DVD, and the like. The recording medium 24 may be built in the electronic camera 1 or may be detachably mounted. Further, it may be provided outside the electronic camera 1. In that case, the recording medium 24 and the electronic camera 1 are electrically connected by wire or wirelessly.

  An outline of operations at the time of shooting in the electronic camera 1 having the above-described configuration will be described with reference to a flowchart shown in FIG. The flowchart of FIG. 2 is a process common to the three operation modes of “through image shooting operation”, “still image main shooting operation”, and “moving image shooting operation”. Note that through-image shooting is preliminary shooting performed as a pre-stage of still image main shooting. The still image main shooting is, for example, shooting performed in response to a release button full-press operation signal (shooting instruction) of the operation unit 15. The moving image shooting is shooting performed in response to an operation of the REC button of the operation unit 15, for example.

  In step S1, the control circuit 14 determines whether imaging has been instructed. If the control circuit 14 determines that imaging has been instructed via the operation member 15, the process proceeds to step S2.

  In step S <b> 2, the control circuit 14 captures a subject image by controlling each unit.

  In step S <b> 3, the control circuit 14 acquires lens setting information (focal length, zoom magnification, etc.) at the time of shooting and aberration information of the shooting lens 11 from the shooting lens 11. The aberration information is a design value of the photographic lens 11 or a test value obtained by testing characteristics of the photographic lens 11 in advance. Further, aberration information may be recorded in the electronic camera 1 for each type of photographing lens, and the type of photographing lens may be acquired as aberration information.

  In step S4, the control circuit 14 controls the data processing circuit 18 to correct image distortion (geometric distortion) based on the information acquired in step S3. The data processing circuit 18 temporarily records image data of an image before correcting distortion in an internal memory or the like.

  For example, as shown in FIG. 3A, when a mountain Ma exists in the lower part of the subject and the airplane Fa moves from the left to the right in the upper part, when distortion occurs, the mountain Ma is, for example, the mountain Mb (FIG. 3B). ) And the trajectory of the airplane Fa is deformed as, for example, the airplane Fb (FIG. 3B). Based on the information acquired in step S3, the data processing circuit 18 determines parameters for deforming the image in the vertical and horizontal directions so as to cancel out this distortion, and deforms the image data input from the signal processing circuit 17. Thus, distortion is corrected. The value of this parameter varies depending on the position in the image. When the distortion is corrected, the image data input from the signal processing circuit 17 shown in FIG. 3B is corrected to image data with the corrected distortion shown in FIG. 3A. It should be noted that during the through image shooting operation and the moving image shooting operation, the distortion aberration is corrected for each of a plurality of continuously generated images.

  In step S5, the control circuit 14 controls the data processing circuit 18 to calculate the motion vector of the subject included in the image whose distortion has been corrected in step S4. Note that this step is skipped during the still image main photographing operation.

  The data processing circuit 18 calculates a motion vector using the image data of a plurality of frames whose aberrations are corrected in step S4 during the through image capturing operation and the moving image capturing operation. The data processing circuit 18 calculates a motion vector for each subject in the screen. The motion vector can be calculated, for example, by performing block matching processing for each subject between temporally adjacent frames. In step S5, since the motion vector of the subject included in the image whose distortion has been corrected is calculated, an appropriate motion vector can be calculated even for an image including geometric distortion.

  In step S6, the control circuit 14 controls the data processing circuit 18 to extract the shape information of the subject included in the image whose distortion has been corrected in step S4.

  The data processing circuit 18 extracts shape information of a subject included in the image for each of the plurality of frames of image data whose aberration has been corrected in step S4. The data processing circuit 18 extracts shape information for each subject in the screen. The shape information can be extracted, for example, by extracting a closed (straight) line having a predetermined amount or more of edge components from the image data. In step S6, the shape information of the subject included in the image whose distortion has been corrected is extracted, so that appropriate shape information can be extracted even for an image including geometric distortion.

  In step S7, the control circuit 14 controls the data processing circuit 18 to specify the type of the subject based on the shape information of the subject extracted in step S6.

  The data processing circuit 18 stores dictionary data in which the relationship between the type of subject and various feature quantities is recorded. In the dictionary data, for example, shape information without distortion is stored as a feature quantity corresponding to the subject type. The data processing circuit 18 identifies the type of the subject by comparing the dictionary data with the shape information of the subject extracted in step S6.

  In step S <b> 8, the control circuit 14 records the image, the feature amount, and the subject type information in association with each other on the recording medium 24. The feature amount is the motion vector calculated in step S5 (except during the still image main photographing operation) and the shape information extracted in step S6. The control circuit 14 records the feature quantity in the recording medium 24 in association with the image data of the image before correcting the distortion aberration temporarily recorded in the internal memory or the like. The feature amount is recorded in the form of text data, for example.

  In recording, the image data of the image in which the distortion is corrected in step S4 may be recorded instead of the image before the distortion is corrected. Also, image data of both images before and after correcting distortion aberration may be recorded. Which image is to be recorded may be selectable by the user via the operation unit 15. In addition, for example, when it can be estimated that the user intentionally adds distortion aiming at the video effect by photographing using a fisheye lens or the like based on lens setting information at the time of photographing or aberration information of the photographing lens 11, the distortion may occur. It may be configured to automatically record image data of an image before correcting the aberration.

  Next, details of each of “through image shooting operation”, “still image main shooting operation”, and “moving image shooting operation” will be described.

<Through-image shooting operation>
When the electronic camera 1 performs a still image main photographing operation, when a half-press operation signal based on a half-press operation of the release button is input from the operation member 15 to the control circuit 14, the through image accompanied with the AF operation described above is obtained. Perform the shooting operation. The through image capturing operation is repeatedly performed at a predetermined frame rate. During the half-press operation of the release button, the above-described AF operation is performed in parallel with the through image capturing operation, and the captured through image is displayed on the monitor 20. Is displayed.

  The control circuit 14 sends an instruction to the image sensor drive circuit 16 to output a drive signal for executing a through image photographing operation. The image sensor 12 receives a drive signal for a through image capturing operation and continuously outputs accumulated charges at a high frame rate of 30 frames / second, for example. The exposure condition for through-image shooting is determined based on the photometric data from the photometric circuit 22. The signal processing circuit 17 performs the above-described signal processing on the input signal, adds the signals of the same color pixels (in the case of a single-plate color image sensor) located in the vicinity on the image sensor 12, and later-described still images The video signal is output to the data processing circuit 18 as a video signal having a lower resolution (lower number of pixels) than at the time of actual photographing.

  The data processing circuit 18 calculates the motion vector and specifies the type of subject as described above, and outputs the image data subjected to resolution conversion processing to the display control circuit 21. As a result, a through image is displayed on the monitor 20. As a result, the user can observe the state of the object scene to be photographed from now on the screen of the monitor 20. That is, in the through image shooting operation, the imaging operation and the display operation are performed in parallel.

<Still image shooting>
When the release button is fully pressed following the through image shooting operation, a full press operation signal is input to the control circuit 14. The electronic camera 1 starts a still image main photographing operation in response to the full-press operation signal.

  When the control circuit 14 detects a full-press operation signal from the operation member 15 (release button), it sends an instruction to the image sensor drive circuit 16 to output a drive signal for executing the main photographing operation. The image sensor 12 receives a drive signal for the still image main photographing operation, performs charge accumulation for main photographing based on the exposure calculation result, and outputs the accumulated charge. The exposure condition for the main photographing is determined based on the brightness information of the object scene obtained from the signal value of the image data of the latest through image. The signal processing circuit 17 performs the above-described signal processing on the input signal, and outputs the image data to the data processing circuit 18 as image data having a higher resolution (higher number of pixels) than when a through image is captured.

  The data processing circuit 18 specifies the type of subject as described above, and outputs the image data subjected to resolution conversion processing to the display control circuit 21. As a result, the still image actual captured image is displayed on the monitor 20. Further, the image data compressed by the compression / decompression processing circuit 19 is recorded on the recording medium 24 together with the specified subject type information and feature amount.

  If the operation member 15 is instructed to record on the recording medium 24 in an uncompressed state, the compression / decompression processing circuit 19 does not perform the compression process, and the recording on the recording medium 24 is performed. Is called. In addition, when performing a recording operation on the recording medium 24, the compression rate may be changed by operating the operation member 15.

  Furthermore, shooting distance information acquired in the above-described still image main shooting is recorded in the recording medium 24 in association with the still image data.

<Movie shooting operation>
When the electronic camera 1 performs a moving image shooting operation, when the operation signal of the REC button is input from the operation member 15 to the control circuit 14, the electronic camera 1 performs the moving image shooting operation with the AF operation described above. The moving image shooting operation is repeatedly performed at a predetermined frame rate. During the pressing operation of the REC button, the above-described AF operation is performed in parallel with the moving image shooting operation, and the captured moving image is displayed on the monitor 20. At the same time, the compressed image data is recorded on the recording medium 24.

  The control circuit 14 sends an instruction to the image sensor drive circuit 16 to output a drive signal for executing a moving image shooting operation. The image sensor 12 receives a drive signal for moving image shooting operation and continuously outputs accumulated charges at a high frame rate of, for example, 30 frames / second. The exposure conditions for moving image shooting are determined based on the photometric data from the photometric circuit 22.

  The data processing circuit 18 calculates the motion vector and specifies the type of subject as described above, and outputs image data subjected to resolution conversion processing according to the number of display pixels of the monitor 20 to the display control circuit 21. As a result, the captured moving image is displayed on the monitor 20.

  The data processing circuit 18 outputs the image data before the resolution conversion described above to the compression / decompression processing circuit 19. That is, in the moving image shooting operation, the imaging operation, the display operation, and the recording operation are performed in parallel.

  As described above, the data processing circuit 18 receives lens setting information (focal length, zoom magnification, etc.) at the time of shooting and distortion information of the shooting lens 11 from the control circuit 14, and corrects the distortion aberration. Then, a motion vector is calculated using the image data of a plurality of frames whose aberrations are corrected.

  Finally, processing for searching and editing images recorded on the recording medium 24 will be described.

<About video>
The user operates the operation member 15 to select feature quantities (motion information, shape information) recorded in association with each frame image from among a plurality of frames constituting the moving image recorded on the recording medium 24. The desired scene can be extracted based on the subject type information.

  The control circuit 14 displays a menu on the monitor 20 so that a predetermined feature amount and subject type information can be selected. The user uses the operation member 15 to select a desired subject type from among the subject types displayed on the monitor 20. For example, when “person A” is selected as the subject type and the movement direction and the movement amount are selected as the motion information (feature amount), the control circuit 14 reads the selected subject type and feature amount, and records the recording medium. The first frame of the scene in which the selected type of subject is moving in the designated moving direction and moving amount is extracted from the moving image image data recorded in 24 and displayed on the monitor 20. . When a plurality of scenes are extracted, the control circuit 14 may display a reduced image of the first frame of each scene on the monitor 20.

  When one of the scenes is selected by the user, the control circuit 14 sequentially reads out image data corresponding to the selected scene from the recording medium 24 and starts a reproduction operation to the monitor 20.

<About still images>
The user operates the operation member 15 to select a desired image from the still images recorded on the recording medium 24 based on the feature values (shape information and the type of subject) recorded in association with the still image. A still image can be extracted.

  For example, the control circuit 14 displays a menu on the monitor 20 so as to select predetermined subject type information. The user uses the operation member 15 to select a desired subject type from among the subject types displayed on the monitor 20. For example, when “person B” is selected as the subject type, the control circuit 14 reads the selected subject type, and selects the selected type of subject from the still images recorded on the recording medium 24. The still image included is extracted and displayed on the monitor 20. When a plurality of still images are extracted, the control circuit 14 may display a reduced image of each still image on the monitor 20.

  When any one of the still images is selected by the user, the control circuit 14 reads out the image data of the selected still image from the recording medium 24 and reproduces it on the monitor 20.

  As described above, according to the first embodiment, an imaging unit that captures an image of a subject via the photographing lens to generate an image, an acquisition unit that acquires lens information of the photographing lens, and an image generated by the imaging unit. Generated by an image correction unit that corrects the geometric distortion of the image based on lens information, a motion extraction unit that extracts motion information of a subject included in the image from an image corrected by the image correction unit, and an imaging unit A recording unit that associates the recorded image with the motion information extracted by the motion extraction unit, and an image extraction unit that extracts an image having a predetermined feature from the image recorded in the recording unit based on the motion information. Prepare. Accordingly, it is possible to extract an appropriate feature amount (motion information) from the image, and to perform accurate search and editing of the image.

  In addition, according to the first embodiment, an imaging unit that captures an image of a subject via a photographic lens to generate an image, an acquisition unit that acquires lens information of the photographic lens, and a geometric shape of an image generated by the imaging unit An image correction unit that corrects the distortion based on the lens information, and a shape extraction unit that extracts the shape information of the subject included in the image from the image corrected by the image correction unit. Therefore, an appropriate feature amount (shape information) can be extracted from the image.

  Further, according to the first embodiment, based on the shape storage unit that stores shape information for each subject in advance, the shape information extracted by the shape extraction unit, and the shape information stored in the shape storage unit, the imaging unit Based on the identification unit that identifies the type of the subject included in the generated image, the image generated by the imaging unit, the recording unit that associates and records the type of the subject identified by the identification unit, and the type of the subject, An image extracting unit that extracts an image having a predetermined characteristic from the image recorded in the recording unit; Therefore, it is possible to accurately search and edit an image based on the extracted feature amount (shape information) and subject type information.

  According to the first embodiment, the image generated by the imaging unit and the feature amount (motion information, shape information) are recorded in the recording unit in association with each other, or the image correction is performed instead of the image generated by the imaging unit. The image processing apparatus further includes a selection unit that selects whether to record the image corrected by the unit and the feature amount in association with each other. Therefore, it is possible to discriminate between distortion due to the presence of undesirable aberrations in the photographing optical system and distortion intentionally added by the user for the purpose of video effects, and to record an appropriate image together with the feature amount.

  In the first embodiment, an example is shown in which motion information and shape information (subject type) are obtained as feature amounts, but only one of them may be used.

Second Embodiment
The second embodiment of the present invention will be described below with reference to the drawings. In the second embodiment, only different parts from the first embodiment will be described. The electronic camera of the second embodiment has the same configuration as the electronic camera 1 of the first embodiment. Therefore, the illustration and description of the configuration of the electronic camera of the second embodiment will be omitted, and description will be made using the same reference numerals as those in FIG. 1 of the first embodiment.

The outline of the operation at the time of shooting in the electronic camera 1 of the second embodiment will be described with reference to the flowchart shown in FIG. The flowchart of FIG. 4 is a process common to the three operation modes of “through image shooting operation”, “still image main shooting operation”, and “moving image shooting operation”. Note that through-image shooting is preliminary shooting performed as a pre-stage of still image main shooting. The still image main shooting is, for example, shooting performed in response to a release button full-press operation signal (shooting instruction) of the operation unit 15. The moving image shooting is shooting performed in response to an operation of the REC button of the operation unit 15, for example.

In step S21, the control circuit 14 determines whether imaging has been instructed. If the control circuit 14 determines that imaging has been instructed via the operation member 15, the control circuit 14 proceeds to step S22.

  In step S22, the control circuit 14 captures a subject image by controlling each unit.

  In step S <b> 23, the control circuit 14 acquires lens setting information (focal length, zoom magnification, etc.) at the time of shooting and aberration information of the shooting lens 11 from the shooting lens 11.

  In step S24, the control circuit 14 controls the data processing circuit 18 to correct the shape information (dictionary data) recorded in advance. The data processing circuit 18 stores dictionary data in which the relationship between the type of subject and various feature quantities is recorded. In this dictionary data, shape information in a state without distortion is stored corresponding to the type of subject. Therefore, this dictionary data is corrected to dictionary data with distortion. Based on the information acquired in step S23, the data processing circuit 18 determines longitudinal and lateral parameters so as to take this distortion into account, and corrects the dictionary data. As a result, as the corrected dictionary data, shape information in a state where there is distortion is calculated corresponding to the subject type.

  In step S25, the control circuit 14 controls the data processing circuit 18 to extract subject shape information included in the image captured in step S22.

  The data processing circuit 18 extracts the shape information of the subject included in the image data of the image captured in step S22 (for each of the frames when there are a plurality of frames). The data processing circuit 18 extracts shape information for each subject in the screen. The shape information can be extracted, for example, by extracting a closed (straight) line having a predetermined amount or more of edge components from the image data.

  In step S26, the control circuit 14 controls the data processing circuit 18 to identify the type of subject based on the subject shape information extracted in step S25.

  The data processing circuit 18 identifies the type of subject by comparing the dictionary data (shape information) corrected in step S24 with the shape information of the subject extracted in step S25. In step S26, the type of the subject is specified using the dictionary data corrected based on the distortion aberration, so that the subject can be appropriately specified even for an image including geometric distortion.

In step S27, the control circuit 14 records the image, the feature amount (shape information), and the subject type information in association with each other on the recording medium 24. The feature amount is the type of subject specified in step S26. The control circuit 14 records the feature quantity and the image generated by the imaging temporarily recorded in the internal memory or the like in association with each other on the recording medium 24. The feature amount is recorded in the form of text data, for example.
Yes.

  Details of the operations of “through image shooting operation”, “still image main shooting operation”, and “moving image shooting operation” are the same as those in the first embodiment, and thus the description thereof is omitted. Further, since processing for searching for and editing an image recorded in the recording medium 24 is the same as that in the first embodiment, description thereof is omitted.

  As described above, according to the second embodiment, the imaging unit that captures an image of a subject via the photographing lens to generate an image, the acquisition unit that acquires lens information of the photographing lens, and the shape information for each subject are obtained. Shape information stored in advance, shape information correction unit that corrects shape information stored in the shape storage unit based on lens information, and shape information of the subject included in the image are extracted from the image generated by the imaging unit And a specifying unit for specifying the type of subject included in the image based on the shape information extracted by the shape extracting unit and the shape information corrected by the shape information correcting unit. Therefore, it is possible to extract an appropriate feature amount (shape information, subject type) from the image, and to perform accurate search and editing of the image.

  Further, according to the second embodiment, the recording unit that records the image generated by the imaging unit in association with the type of the subject specified by the specifying unit, and the image recorded in the recording unit based on the type of the subject And an image extracting unit for extracting an image having a predetermined feature from the image. Therefore, it is possible to accurately search and edit an image based on the extracted feature amount (shape information) and subject type information.

  In each of the above-described embodiments, the moving image shooting has been described without limiting the format of the moving image, but the format of the moving image may be any. For example, when the present invention is applied to MPEG-7, it is preferable to record a feature quantity such as motion information of a subject or an imaging device or shape information of the subject in a metadata area.

  In each of the above embodiments, an example in which the motion vector of the subject is calculated as the motion information has been described. However, the motion (pan, tilt, etc.) of the imaging device may be obtained based on the image data. In this case, the motion information of the imaging device may be recorded in association with the image. The user can perform the same processing (search, edit, etc.) as described above based on the motion information (direction, direction, rotation, etc.) of the imaging device.

  In each of the above-described embodiments, the description has been made using the electronic camera to which the interchangeable photographing lens can be attached. However, the present invention can be similarly applied to an electronic camera in which the photographing lens and the main body are integrated. .

  In each of the above embodiments, an electronic camera has been described as an example of an imaging apparatus. However, the present invention can be similarly applied to an imaging apparatus other than an electronic camera.

  Moreover, it is good also as a structure which implement | achieves one part or all part of the process demonstrated by said each embodiment with apparatuses other than imaging devices, such as a computer. For example, when implemented in a computer, a program for realizing each process described in FIG. 2 and FIG. 4 may be recorded in the computer, and each process may be performed according to the program. In a computer, it is good also as a structure which acquires the lens information (design information, aberration information, etc.) of the taking lens used at the time of image generation together with the image to be processed, and is used at the time of image generation together with the image to be processed. The geometric distortion of the image to be processed is corrected in advance based on the lens information of the photographed lens, and the feature amount (motion vector and shape information) of the subject included in the image extracted from the corrected image to be processed It is good also as a structure to acquire.

DESCRIPTION OF SYMBOLS 1 ... Electronic camera, 11 ... Shooting lens, 12 ... Image sensor, 14 ... Control circuit, 18 ... Data processing circuit, 24 ... Recording medium

Claims (4)

An imaging apparatus including an imaging unit that captures an image of a subject via a photographic lens and generates an image,
An acquisition unit for acquiring lens information of the photographing lens;
A shape storage unit for storing shape information for each subject in advance;
An image correction unit that corrects geometric distortion of the image generated by the imaging unit based on the lens information;
A shape extraction unit that extracts shape information of a subject included in the image from the image corrected by the image correction unit;
The type of subject included in the image generated by the imaging unit based on the shape information extracted by the shape extraction unit from the corrected image and the shape information stored in the shape storage unit A specific part for identifying
A recording unit that records the image generated by the imaging unit in association with the type of the subject specified by the specifying unit;
Based on the type of the object, imaging, characterized in that from the recorded the image on the recording unit Ru and an image extraction unit configured to extract an image having a predetermined feature device. The imaging device according to claim 1,
The image correction unit corrects the geometric distortion by performing aberration correction on an image having a lower pixel number than an image obtained by still image shooting by the image pickup unit. . In the imaging device according to claim 1 or 2,
In the recording unit, the image generated by the imaging unit and the type of the subject are recorded in association with each other, or instead of the image generated by the imaging unit, the image corrected by the image correction unit and the image It further includes a selection unit that selects whether to record in association with the type of subject.
An imaging apparatus characterized by that. An image processing program for executing image processing on an image to be processed by a computer,
An acquisition procedure for acquiring an image to be processed and lens information of a photographing lens used when generating the image;
An image correction procedure for correcting geometric distortion of the image to be processed based on the lens information;
A shape extraction procedure for extracting shape information of a subject included in the image from the image to be processed after the correction by the image correction procedure;
The type of the subject included in the image to be processed based on the shape information extracted by the shape extraction procedure from the image to be processed after the correction and the shape information for each subject stored in advance. Specific steps to identify
A recording procedure for associating and recording the image to be processed and the type of the subject specified by the specifying procedure;
An image extraction procedure for extracting an image having a predetermined feature from the image to be processed recorded in the recording procedure based on the type of the subject;
An image processing program comprising:

Images