JP6292872B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  2. Description of the Related Art Conventionally, as an image processing apparatus, a digital camera that records image data obtained by imaging a subject on a recording medium as a still image or a moving image is known. In recent years, in such a digital camera, since a large amount of photography has become possible, a technique for improving the viewability is required. For example, as in Patent Document 1, a technique has been proposed in which only a scene with a high volume is extracted from a moving image such as a television program and reproduced.

JP 2007-267351 A

  However, in Patent Document 1, it is impossible to display information indicating a person or a place appearing in digest reproduction.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus capable of displaying information related to an extracted scene.

In order to achieve such an object, the image processing apparatus of the present invention provides :
Reproducing means for reproducing image data recorded on a recording medium, extracting means for extracting an image including a designated feature image, an image extracted by the extracting means, and an identifier associated with the designated feature image Display control means for displaying on the display unit, and it is possible to specify a plurality of feature images, and the display control means includes a case where the specified feature images are less than a predetermined number and a predetermined number or more. The display timing of the identifier is different depending on the case.

  According to the present invention, information related to an extracted scene can be displayed.

1 is a diagram illustrating a configuration of an imaging apparatus according to an embodiment. The figure which shows the operation | movement of the album reproduction | regeneration of a present Example. The conceptual diagram of the album reproduction | regeneration of a present Example. The conceptual diagram of the album reproduction | regeneration of a present Example.

  Hereinafter, examples of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments. The following embodiments do not limit the invention according to the claims, and all combinations of features described in the embodiments are not necessarily essential to the solution means of the invention.

  Note that each functional block described in the present embodiment is not necessarily separate hardware. That is, for example, the functions of some functional blocks may be executed by one piece of hardware. In addition, the function of one functional block or the functions of a plurality of functional blocks may be executed by some hardware linked operations. Further, the function of each functional block may be executed by a computer program developed on the memory by the CPU.

Example 1
In this embodiment, an imaging apparatus will be described as an example of an image processing apparatus. However, any apparatus may be used as long as it can reproduce a plurality of moving images and / or still images. For example, a mobile phone, a smartphone, a tablet information terminal, a notebook information terminal, a computer, or the like may be used. In this embodiment, a moving image file and a still image file are sometimes collectively referred to as an image file. In the present embodiment, moving image data and still image data may be collectively referred to as image data.

  The imaging apparatus according to the present embodiment can reproduce a moving image and / or a still image recorded on a recording medium. The imaging apparatus according to the present exemplary embodiment can extract images related to a designated person (or face) from a plurality of images recorded on a recording medium, and sequentially display the images. In this embodiment, a feature image such as a designated person's face and its identifier (name, nickname, etc.) are registered in advance in a “dictionary”, and the extracted image is sequentially displayed. Display when. Furthermore, in the present embodiment, the timing for displaying the identifier can be varied according to the number of persons included in the extracted image.

  With such a configuration, the imaging apparatus of the present embodiment can display information related to the extracted scene. Also, by changing the display timing of the identifier associated with the person (or face), for example, a slide show that is easy to see for the user or a slide show that is rich in change can be provided. Hereinafter, such an imaging apparatus will be described.

<Overall configuration>
First, the configuration of the imaging apparatus 100 of the present embodiment will be described with reference to FIG.

  As illustrated in FIG. 1, the imaging apparatus 100 according to the present exemplary embodiment includes a CPU 101, a RAM 102, a ROM 103, and an operation unit 104. In addition, the imaging apparatus 100 includes a first imaging unit 110, an image processing unit 111, a second imaging unit 112, a sound collection unit 120, and an audio processing unit 121. In addition, the imaging apparatus 100 includes a display unit 140, a display control unit 141, a recording / playback unit 150, a recording medium 151, and an attitude detection unit 160. CPU is an abbreviation for Central Processing Unit. RAM is an abbreviation for Random Access Memory. ROM is an abbreviation for Read Only Memory.

  In the imaging apparatus 100 according to the present exemplary embodiment, the CPU 101 develops various programs recorded in the ROM 103 using the RAM 102 as a work memory, and controls each block of the imaging apparatus 100 according to the program. The operation unit 104 includes switches for inputting various operations such as a power button, a record button, a zoom adjustment button, an autofocus button, a menu display button, a mode switch, and a determination button. Further, any type of operation element such as a cursor key, a pointing device, a touch panel, and a dial may be used. The operation unit 104 transmits an operation signal to the CPU 101 when these keys, buttons, and touch panel are operated by the user. Each operation member of the operation unit 104 is appropriately assigned a function for each scene by selecting and operating various function icons displayed on the display unit, and functions as various function buttons. Examples of the function buttons include an end button, a return button, an image advance button, a jump button, a narrowing button, and an attribute change button. For example, when a menu button is pressed, various setting menu screens are displayed on the display unit. The user can make various settings intuitively using the menu screen displayed on the display unit, and the four-way buttons and the SET button. Note that the operation unit 104 may be a touch panel that can detect contact with the display unit. The touch panel may be any of various types of touch panels such as a resistive film method, a capacitance method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, an image recognition method, and an optical sensor method. .

  In the first imaging unit 110, the second imaging unit 112 converts the optical image of the subject captured by the lens into an image signal using an imaging element such as a CCD sensor or a CMOS sensor by controlling the amount of light using a diaphragm. In the first imaging unit 110, the second imaging unit 112 converts the obtained analog image signal into a digital image signal and temporarily stores it in the RAM 102. The image processing unit 111 processes image data stored in the RAM 102 during recording, and processes image data read from a recording medium 151 (to be described later) during reproduction. The image processing unit 111 performs various processes such as white balance adjustment, gain adjustment, and color enhancement processing of image data. The image processing unit 111 transmits the processed image data to the display control unit 141 or to the recording / playback unit 150 or the communication unit. In addition, the image processing unit 111 is an H.264. H.264 / AVC, Motion JPEG2000, H.264, etc. An image can be compressed as a moving image using a moving image compression technique such as H.265, or an image can be compressed as a still image using a still image compression technique such as JPEG or JPEG 2000. As the compression format, other methods may be used regardless of these formats.

  The image processing unit 111 according to the present exemplary embodiment analyzes the images obtained by the first imaging unit 110 and the second imaging unit 112 and detects whether an image corresponding to a person or a face is included in the image. It is also possible to detect facial expressions. There are various detection methods. For example, the detection can be performed by performing pattern matching with a human or facial feature image. When a face is detected, the size and position are stored in the RAM 102. When a face is detected, the edge of the face image is detected and the smile level is determined. These pieces of detection information are stored in a header or the like of an image file including target image data when a still image or a moving image is shot.

  The audio processing unit 121 processes audio data obtained by collecting sound by the sound collecting unit 120 during recording, and processes compressed or uncompressed audio data read from a recording medium 151 described later during reproduction. . For example, audio data level adjustment processing, compressed audio data decompression processing, audio data compression processing, and the like are performed. The audio processing unit 121 transmits the processed audio data to the recording / reproducing unit 150 and the communication unit, or to the speaker during reproduction. Note that the sound collection unit 120 is, for example, a microphone that collects a plurality of omnidirectional sounds built in the housing of the imaging apparatus 100. Also, the audio processing unit 121 uses a known general audio compression method such as AC3, AAC, or the like as the audio compression method.

  The display unit 140 includes, for example, a liquid crystal display device or an organic EL display device, and displays an image under the control of the display control unit 141. The display unit 140 may be anything such as an LED display as long as it can provide an image to the user. The display control unit 141 displays an image on the display unit 140 based on the image data processed by the image processing unit 111. Further, the display control unit 141 may perform image signal processing such as matrix conversion, brightness adjustment, contrast adjustment, gamma adjustment, chroma gain adjustment, and sharpness adjustment on the video signal based on the digital image signal displayed on the display unit 140. Good.

  The recording / reproducing unit 150 records the image data and audio data processed by the image processing unit 111 on the recording medium 151 at the time of recording, and the image data and audio data recorded on the recording medium 151 at the time of reproduction. Reproduce. At the time of recording, the recording / reproducing unit 150 writes various types of information such as the shooting date, information related to compression, and settings of the imaging unit 110 to the recording medium 151 together with image data and audio data. The recording / reproducing unit 150 records image data and audio data on the recording medium 151 as a file in a form suitable for a file format such as FAT or exFAT, for example. Further, the recording medium 151 may be a recording medium built in the imaging apparatus or a removable recording medium. For example, the recording medium includes all types of recording media such as a hard disk, an optical disk, a magneto-optical disk, a CD-R, a DVD-R, a magnetic tape, a nonvolatile semiconductor memory, and a flash memory. When using removable recording media, the recording / reproducing unit 150 includes a mechanism for loading and unloading the removable recording media.

  The posture detection unit 160 detects the posture of the imaging apparatus 100 and is, for example, an angle sensor or an acceleration sensor. The posture detection unit 160 of the present embodiment detects whether the posture of the imaging device 100 is the normal position or the vertical position. For example, a rotation attitude angle (0 degree, 90 degrees, 180 degrees, 270 degrees) about the optical axis of the first imaging unit 110 of the imaging apparatus 100 can be detected. Here, 0 degrees is a so-called normal position (a state in which the upper surface of the imaging apparatus 100 faces the sky direction), and 180 degrees is a reverse position (a state in which the upper surface of the imaging apparatus 100 faces the ground direction). 90 degrees and 270 degrees are so-called vertical positions (a state in which either of the left and right surfaces of the imaging apparatus 100 faces the ground).

  Note that the first imaging unit 110 (first imaging unit) and the second imaging unit 112 (second imaging unit) of the present embodiment each capture subjects in different directions. Specifically, the second imaging unit 112 is a sub-camera for photographing a photographer (so-called cameraman), and the first imaging unit 110 is a subject (so-called subject) that the photographer intends to photograph. This is the main camera. Therefore, for example, the first image capturing unit 110 and the second image capturing unit 112 capture images in the reverse direction, but may not necessarily be in the reverse direction.

  Here, each of the image processing unit 111, the audio processing unit 121, the display control unit 141, and the recording / reproducing unit 150 may be a microcomputer equipped with a program for executing the above-described functions. Further, the program for executing the above-described processing recorded in the ROM 103 by the CPU 101 may be loaded into the RAM 102 and executed.

<Operation of Imaging Device 100>
Subsequently, the operation of the imaging apparatus 100 of the present embodiment will be described. The imaging apparatus 100 according to the present embodiment has a “moving image still image shooting mode” in addition to the “still image shooting mode”, the “moving image shooting mode”, and the “reproduction mode”. In each mode, the CPU 101 controls each block of the imaging apparatus 100 to perform the following operation.

  In the “still image shooting mode” and “moving image shooting mode”, only the image obtained by the first imaging unit 110 or the image obtained by the first imaging unit 110 and the first image until the shooting instruction is input. 2 The image obtained by combining the images obtained by the imaging unit 112 is displayed on the display unit 140. That is, an image obtained by each imaging unit is processed by the image processing unit 111, and the display control unit 141 is controlled to be displayed on the display unit 140. When a shooting instruction is input, the recording / reproducing unit 111 causes the image processing unit 111 to compress the image obtained by each imaging unit as a moving image or a still image, and sequentially record the compressed image on the recording medium 151. 150 is controlled. In the “still image shooting mode”, the image processing unit 111 is controlled so that one or a plurality of images obtained by each imaging unit are compressed using, for example, JPEG at a timing corresponding to a shooting instruction. Then, the recording / reproducing unit 150 is controlled to record the compressed still image data on the recording medium 151 as a still image file. On the other hand, in the “moving image shooting mode”, a moving image in which images sequentially obtained from images obtained by the respective imaging units at a timing corresponding to a shooting instruction are frame images is used, for example. The image processing unit 111 is controlled to perform compression using H.265. Then, the recording / playback unit 150 is controlled to record the compressed moving image data on the recording medium 151 as a moving image file. In the “moving image shooting mode”, this operation is continued until an instruction to end shooting is input. In the “still image shooting mode” and “moving image shooting mode”, a thumbnail image (still image data) based on the image shot in each mode is added to the still image file and the movie file and recorded on the recording medium 151. Is done.

  In “Movie Still Image Shooting Mode”, when a still image recording instruction is input, still image shooting and still image recording are performed, and moving image data for 4 seconds immediately before the still image recording instruction is input is recorded. Record on media. On the same day, when a plurality of still image recording instructions are input in the “moving picture still image shooting mode”, the moving image data for 4 seconds immediately before the input of the still image recording instruction is stored as one. Add to the movie file and record. By doing in this way, the moving image data just before the still image photography for 1 day can be stored in one moving image file, and browsing property can be improved. In the “moving picture still image shooting mode”, the CPU 101 performs only the image obtained by the first imaging unit 110 or the image obtained by the first imaging unit 110 and the second imaging until a shooting instruction is input. An image obtained by combining the images obtained by the unit 112 is displayed on the display unit 140. That is, an image obtained by each imaging unit is processed by the image processing unit 111, and the display control unit 141 is controlled to be displayed on the display unit 140. At this time, the image processing unit 111 combines only the image obtained by the first imaging unit 110 or an image obtained by combining the image obtained by the first imaging unit 110 and the image obtained by the second imaging unit 112. It is sequentially compressed as moving image data and temporarily stored in the RAM 102. The CPU 101 manages information in the RAM 102 so that the latest compressed video data for 4 seconds is always held in the RAM 102. For example, moving image data other than the latest 4 seconds may be discarded. Next, when a still image recording instruction is input, the image processing unit 111 is controlled to compress one or a plurality of images obtained by each imaging unit at a timing corresponding to the instruction using, for example, JPEG. To do. Then, the recording / reproducing unit 150 is controlled so as to record still image data compressed as a still image on the recording medium 151. On the other hand, in response to a still image recording instruction, the recording / reproducing unit 150 is controlled so that the latest 4-second compressed moving image data stored in the RAM 102 is also recorded on the recording medium 151. In addition, the image processing unit 111 is controlled so as to compress the image obtained by the still image recording instruction as a moving image (stop motion), and recorded on the recording medium 151 so as to be reproduced after the latest 4-second moving image data. To do. Thus, the recording / playback unit 150 includes the moving image data of the image obtained by the still image recording instruction in the same moving image file as the latest 4-second moving image data recorded on the recording medium. It should be noted that information regarding an operation during shooting of moving image data to be recorded, and information on the detected position, size, and smile level of a human face may be stored in a corresponding still image file header. Information indicating whether or not a stop motion video has been generated may also be stored in the header of the still image file.

  In the “playback mode”, the recording / playback unit 150 is controlled to read the file list and thumbnail images of the moving image file or still image file recorded on the recording medium 151. Then, the display control unit 141 is controlled so that the thumbnail image of each file reproduced by the recording / reproducing unit 150 is expanded by the image processing unit 111 and the expanded thumbnail image is displayed on the display unit 140. Then, the CPU 101 expands the still image file or the moving image file corresponding to the thumbnail image selected by the user operating the operation unit 104 to the image processing unit 111, and displays the obtained image on the display unit 140. The display control unit 141 is controlled.

  In the imaging apparatus 100 according to the present embodiment, it is possible to determine whether a person similar to a person image registered in advance is included in a captured image or a reproduced image during recording or reproduction. Therefore, the imaging apparatus 100 can register, for example, a feature image such as a face image for 12 people and an identifier such as a name or a nickname corresponding to the person (or face). In the registration method, the image processing unit 111 detects the feature amount of an image shot in the person registration mode, causes the user to input a name and a nickname, and the CPU 101 associates them with each other in a non-illustrated nonvolatile memory of the imaging apparatus 100. Record. In this embodiment, the information recorded in association with this may be expressed as a “dictionary”. In order to determine whether a person is registered in this “dictionary”, for example, there is a method of determining whether the face image is similar to the feature amount of a feature image such as a face of a person registered in advance. Other methods may be used. Thus, the image processing unit 111 can determine whether an image corresponding to a person (or face) registered in advance at the time of shooting or reproduction is included. At the time of recording, when determining which person (or face) image is included, the CPU 101 controls the recording / reproducing unit 150 to record the determination result in the header of the image file. In this way, it is possible to determine whether or not an image corresponding to the person registered at the time of reproduction is included by simply checking the header without performing image analysis.

<Play album>
Here, the imaging apparatus 100 according to the present embodiment can select “album playback” in “playback mode”. “Album playback” is a mode in which a predetermined number of image files are extracted from a plurality of image files recorded on a recording medium, and the images are sequentially displayed. In “album playback” in this embodiment, images are played back in the order of “opening”, “main part”, and “ending”. In the imaging apparatus 100 of the present embodiment, an image related to the designated person is specified and displayed as an image “main part” having a high evaluation value in the specified image. In the “album playback” in the present embodiment, an identifier for identifying a designated person is displayed at a predetermined timing. A plurality of feature images such as persons (or faces) can be specified, and the display timing is changed according to the number of specified persons. Specifically, when the number of designated persons is less than 4, the identifiers of the persons designated in the “opening” and registered in the dictionary are displayed for all members. On the other hand, when the number of designated persons is four or more, the identifier of the corresponding person registered in the dictionary is displayed according to the timing when the designated person is first displayed in the “main part”. . The identifier is text data input by the user at the time of dictionary registration, and is, for example, a name or nickname indicating a person (or face).

  Hereinafter, the operation of the imaging apparatus 100 in “album playback” will be described with reference to FIG. The flowchart in FIG. 2 shows processing executed by the CPU 101 controlling each block of the imaging apparatus 100.

<Target image identification>
(S201)
First, when “album playback” is designated, the CPU 101 controls the recording / playback unit 150 to read information included in the header of the still image file recorded on the recording medium 151. The CPU 101 determines whether or not a person registered in the dictionary is included based on information included in the header of the still image file, and displays an image or identifier indicating the included person on the display unit 140. The display control unit 141 is controlled as described above.

(S202)
Next, the CPU 101 specifies the image or identifier of the person selected by the user operation.

  The person selection method is not limited to the processing of S201 and S202. For example, the CPU 101 may control the display control unit 141 so that the image or identifier of a person (or face) registered in the dictionary is displayed on the display unit 140 and allow the user to select it. Further, a person registered in the dictionary may be automatically selected from the persons (or faces) included in the image reproduced immediately before switching to “album reproduction”.

(S203)
Next, the CPU 101 controls the display control unit 141 to display the target time setting screen on the display unit 140 in order to set the total display time of the image in “album playback”, and sets the target time to the user. Let Further, the target number of scenes may be set.

(S204)
Next, the CPU 101 calculates an evaluation value of an image including the person (or face) designated in S202. The CPU 101 identifies a still image file including the person (or face) designated in S202, refers to the additional information of the identified still image file, and calculates the evaluation value of the image of the identified still image file. . The additional information is generated based on information obtained when the image of the still image file is captured. For example, the image processing unit 111 includes the detected face image position, size information, smile level, and the like, but other information may be used. In S204, a higher evaluation value is calculated as the size of the face image is larger, a higher evaluation value is calculated as the position of the face image is closer to the center of the image, and a higher evaluation value is calculated as the smile level is higher. The still image data of the still image file may be analyzed by the image processing unit 111 at this time to detect the position, size, and smile level of the face image. Further, an evaluation value may be calculated at the time of shooting and recorded as additional information of the still image file. Further, the evaluation value may be calculated using shooting information including focus information, YUV histogram, posture information (gyro fluctuation information), zoom information, and the like.

(S205)
Next, the CPU 101 sorts the still image files in descending order of the calculated evaluation value.

(S206)
Next, the CPU 101 sets the target images for “album playback” in descending order of the calculated evaluation values.

  Through the processing so far, an image to be subjected to “album playback” is extracted. In the subsequent processing, “opening”, “main part”, and “ending” of “album playback” are generated based on these specified images.

(S207)
First, the CPU 101 determines whether or not the number of persons (or faces) designated in S202 is less than four.

(S208)
If the number of designated persons (or faces) is less than 4 in S207 (Yes in S207), the CPU 101 sets one still image file including images included in each of the designated persons (or faces). Select one by one. One still image file corresponding to each person (or face) is selected. For example, the still image file to be selected has the highest evaluation value of each person (or face). Therefore, the CPU 101 may calculate an evaluation value for each person (or face) using the face size and the smile level. In addition, the display control unit 141 may be controlled so that an image including each person (or face) is displayed on the display unit 140 and selected by the user.

(S209)
Next, the CPU 101 specifies an area corresponding to each person (or face) from the additional information included in the header of the still image file corresponding to each person (or face) selected in S208. Then, the image processing unit 111 is controlled so as to cut out an image of an area including each person (or face) from the image of the still image file. Then, the CPU 101 controls the image processing unit 111 so as to generate an image in which a name or a nickname (identifier) corresponding to each person (or face) is combined with the clipped image. That is, one image is generated by combining names and nicknames (identifiers) corresponding to each person (or face).

(S210)
Next, the CPU 101 generates an “opening” image set in which images in which names and nicknames (identifiers) corresponding to each person (or face) of each person generated in S209 are combined are reproduced in order.

(S211)
Next, the CPU 101 determines the number of images to be extracted based on the target reproduction time, extracts the determined number of images in descending order of evaluation values, and generates a “main part” that is displayed in chronological order. In this embodiment, the preset target reproduction time is 3 minutes, and the display time per image is 4 seconds. In this case, 3 (minutes) × 60 (second conversion) ÷ 4 (seconds) = 45 (sheets) is the number of images to be extracted. Then, the CPU 101 extracts the still image files sorted in descending order of evaluation values in S205 in descending order of evaluation values, and generates a “main part” image set for reproducing the images of each still image file in time series.

(S220)
Thereafter, the CPU 101 selects an “ending” image. The CPU 101 selects, as an “ending” image, an image in which the most images corresponding to the person (or face) designated in S202 are shown.

(S221)
Next, the CPU 101 sequentially displays the images of the “opening” image set, displays the images of the “main part” image set in order, and finally displays the “ending” image. To control.

  As described above, when the number of selected persons is less than four, the name or nickname (identifier) of the person corresponding to the selected person (or face) is displayed as the “opening” image for a predetermined period (opening period). ) “Album playback” is displayed continuously. A conceptual diagram of the “album playback” image generated in this way is shown in FIG. As shown in FIG. 3, in the opening, names and nicknames (identifiers) are continuously displayed along with images showing each person. In the main part, the selected scene is displayed, and the ending includes many designated persons. Display the image.

(S212)
On the other hand, if the designated person (or face) is not less than 4 in S207 (No in S207), the CPU 101 selects the image that contains the most images corresponding to the person (or face) designated in S202. To do.

(S213)
Next, the CPU 101 selects the oldest image and the newest image among the images extracted in S206 as the “opening” image. Then, the CPU 101 reads the shooting (recording) date and time of the oldest image and the newest image from the header of the still image file including each image. Based on the information, the image processing unit 111 is controlled so as to generate an image obtained by combining the information indicating the shooting period of the image to be reproduced with the image extracted in S212, so that an “opening” image is obtained. . In addition, an image obtained by combining the information indicating the period of the image extracted in S214, which will be described later, with the image extracted in S212 may be an “opening” image.

(S214)
Next, the CPU 101 determines the number of images to be extracted based on the target reproduction time, extracts the determined number of images in descending order of evaluation values, and generates a “main part” that is displayed in chronological order. As in S211, 45 (sheets) is the number of images to be extracted. Then, the CPU 101 extracts the still image files sorted in descending order of evaluation values in S205 in descending order of evaluation values, and generates a “main part” image set for reproducing the images of each still image file in time series.

(S215)
Here, for the images included in the “main part” image set, the CPU 101 displays for the first time an image corresponding to each person (or face) specified in S202 based on the additional information of the still image file included in each image. Identify the image to be played. Then, the CPU 101 controls the image processing unit 111 so as to generate an image in which information indicating a name or a nickname (identifier) corresponding to the person (or face) displayed for the first time on the identified image is generated. Then, for an image in which an image corresponding to each person (or face) is displayed for the first time, an image obtained by combining the corresponding name or nickname (identifier) is replaced with the image before combining in the “main part” image set. It should be noted that the name or nickname may not be synthesized only for the first appearance image, but the name or nickname may be synthesized for the second appearance image. Names and nicknames may be synthesized. In addition, the identifier is displayed at a timing based on the timing at which an image corresponding to each designated person (or face) is displayed for the first time, that is, immediately before or after the image is displayed, without being synthesized with the image. May be.

  Thereafter, the processes of S220 and S221 described above are performed.

  As described above, when the number of selected persons is not less than four, “album playback” that displays names and nicknames (identifiers) on the image corresponding to the person (or face) specified for the first time in the “main part” "I do. FIG. 4 shows a conceptual diagram of the “album playback” image generated in this way. As shown in FIG. 4, in the opening, information indicating a shooting period of an image to be displayed is displayed together with an image including many designated persons. In the main part, the selected scene is displayed, and a name and a nickname (identifier) are displayed for an image corresponding to a person (or face) designated for the first time in the “main part”. In the ending, an image including many designated persons is displayed. As described above, when the number of selected persons is not less than 4, the name or nickname (identifier) is displayed for the image corresponding to the person (or face) specified for the first time in the “main part”, and “opening” is performed. In, information indicating each person is not displayed.

  As illustrated in FIG. 2, the imaging apparatus according to the present exemplary embodiment is configured so that the designated person (or face) designated according to whether the number of designated persons (or faces) is less than the predetermined number or more than the predetermined number. ) The identifier display timing was changed. When the number is smaller than the predetermined number, the identifier of the specified person (or face) specified continuously within the predetermined period of “opening” is displayed. On the other hand, when the number is greater than or equal to the predetermined number, the identifier of the designated person (or face) designated at the timing of first displaying an image corresponding to the designated person in the “main part” instead of “opening” Was displayed. By doing in this way, it is possible to provide a slide show that is easy to see for the user who views “album playback” or a slide show that is rich in change.

  In addition, when the user confirms “play album” and performs “save” operation, image processing is performed so as to generate moving image data in which all the still image files have the expanded image as a frame image for 4 seconds. The unit 111 is controlled. Then, the image processing unit 111 is controlled to generate moving image data that reproduces the generated moving image data in time series, and the recording / reproducing unit 150 records the generated moving image data as a moving image file on the recording medium 151. To control.

  In this embodiment, “album playback” using a still image file has been described as an example, but “album playback” using a moving image file may be performed. Further, “album playback” using both a moving image file and a still image file may be performed. When “album playback” using a moving image file is performed, the scene may be extracted so as to reproduce 4 seconds of a scene having a high evaluation value in the moving image file. In addition, when the evaluation is performed based on the still image file shot in the moving image still image shooting mode of the present embodiment, the movie data of the scene corresponding to the extracted still image file is displayed as “album” instead of the still image file. It may be a target of “reproduction”.

  In addition, when moving image data shot in the “movie still image shooting mode” of this embodiment, still image data, and still image data shot in the “still image shooting mode” are targeted for “album playback”, For example, two or three of the ten selections are moving image data. In this embodiment, moving image data shot in the “moving image still image shooting mode” stores information on the operation during shooting, and information on the detected position, size, and smile level of the human face in the corresponding still image file. Has been. Therefore, the CPU 101 calculates an evaluation value of the moving image data based on the information, and selects a scene having a high evaluation value (data in the moving image corresponding to 4 seconds immediately before the instruction to record a still image) among the moving image data classified into a certain cluster. Extract. On the other hand, regarding still images, those having a high evaluation value among the still image files are extracted. If the extracted moving image data includes a stop motion moving image, the still image that is the same image may not be extracted even if the evaluation value is high.

  Further, in the present embodiment, with respect to the position where the information indicating the designated person (or face) is synthesized, the information indicating the person (or face) designated at the position corresponding to the human face area in the image is included. Change the layout so that they do not overlap. Information indicating the designated person (or face) is synthesized using a technique such as alpha blending so that the displayed image can be seen through.

  It should be noted that it may be possible to set a priority evaluation value as to which still image file has a higher evaluation value among the evaluation values.

  Also, by registering the date of birth of the specified person (or face) as a “dictionary”, the age at the time of shooting is displayed when displaying the identifier of the specified person (or face). It may be displayed. That is, the age is calculated from the date of birth registered in the dictionary and the shooting date and time of the displayed image, and displayed. Moreover, when it is younger than predetermined age, you may make it display to the moon age. In addition, when it is over a predetermined age, the age may not be displayed. Further, when the age changes during “album playback”, the name and age may be displayed again.

  In this embodiment, the image pickup apparatus has been described as an example. However, any apparatus may be used as long as it can reproduce a plurality of moving images and / or still images. For example, a mobile phone, a smartphone, a tablet information terminal, a notebook information terminal, a computer, or the like may be used.

(Other embodiments)
The above-described embodiment can also be realized in software by a computer of a system or apparatus (or CPU, MPU, etc.). Therefore, the computer program itself supplied to the computer in order to implement the above-described embodiment by the computer also realizes the present invention. That is, the computer program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The computer program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto. A computer program for realizing the above-described embodiment is supplied to a computer via a storage medium or wired / wireless communication. Examples of the storage medium for supplying the program include a magnetic storage medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is stored in the server. The program file may be an executable format or a source code. Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers. That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a storage medium in which the computer program for realizing the above-described embodiment is encrypted and distributed is distributed, and key information for decrypting is supplied to a user who satisfies a predetermined condition, and the user's computer Installation may be allowed. The key information can be supplied by being downloaded from a homepage via the Internet, for example. Further, the computer program for realizing the above-described embodiment may use an OS function already running on the computer. Further, a part of the computer program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. Good.

Claims (15)

Reproducing means for reproducing image data recorded on the recording medium;
Extraction means for extracting an image containing the specified feature image,
Have a display control means for displaying an identifier associated with the feature image which is the designated extracted image by said extraction means on a display unit,
It is possible to specify a plurality of the feature images,
The image processing apparatus , wherein the display control unit varies the display timing of the identifier depending on whether the designated feature image is less than a predetermined number or more than a predetermined number . The display control means may control, when the designated feature image is less than the predetermined number, the image processing apparatus according to claim 1, characterized in that continuously displaying the identifier. The display control means may control, when the designated feature image is less than the predetermined number, the image processing apparatus according to claim 1 or 2, characterized in that displaying the identifier for the first predetermined time period. The display control means may control, when the designated feature image is equal to or more than a predetermined number, image processing according to any one of claims 1 characterized in that it does not appear in the first predetermined period 3 apparatus. The display control means may control, when the designated feature image is equal to or more than a predetermined number, the identifier at the timing based on the timing of feature images of the identifier is the designation is displayed in the first predetermined period of time The image processing apparatus according to claim 1 , wherein the image processing apparatus displays the image. Before Kitoku symptom image, the image processing apparatus according to any one of claims 1, characterized in that it is designated 5 in response to the user's operation. Before Kitoku symptom image, the image processing apparatus according to any one of claims 1 to 6, characterized in that it is specified on the basis of the image being displayed. A reproduction step of reproducing the image data recorded on the recording medium;
An extraction step of extracting an image containing the specified feature image,
Have a display control step of displaying the identifier associated with the extracted image and the previous Kitoku symptoms image by the extraction step on a display unit,
A plurality of the feature images can be designated,
In the display control step, the display timing of the identifier is different depending on whether the designated feature image is less than a predetermined number or more than a predetermined number . Wherein the display control step, if said specified feature image is less than the predetermined number, the image processing method according to claim 8, characterized in that continuously displaying the identifier. Wherein the display control step, if said specified feature image is less than the predetermined number, the image processing method according to claim 8 or 9, characterized in that displaying the identifier for the first predetermined time period. In the display control step, if said specified feature image is equal to or more than a predetermined number, image processing according to any one of claims 8 characterized in that it does not appear in the first predetermined period of time 10 Method. In the display control step, if said specified feature image is equal to or more than a predetermined number, the identifier at the timing based on the timing of feature images of the identifier is the designation is displayed in the first predetermined period of time The image processing method according to claim 8, wherein the image processing method is displayed. Before Kitoku symptom image, image processing method according to any one of claims 8 to 12, characterized in that it is specified according to a user operation. Before Kitoku symptom image, the image processing apparatus according to any one of claims 8 to 13, characterized in that it is specified on the basis of the image being displayed. The program for making a computer perform the image processing method of any one of Claims 8 thru | or 14 .

Images