JP2011087104A - Apparatus and method for processing image, program, and medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus which can simply and appropriately detect a representative image (the best shot image) from a group of moving image data. <P>SOLUTION: A predetermined object is detected from the group of moving image data arranged in time series, and time in which the predetermined object is continuously detected is counted. Moving image data with long counted continuous detection time is specified as a section. The representative image is detected from the moving image data in the specified section. Thus, the moving image data with the long continuous detection time among the group of moving image data is considered as the moving image data for detecting the representative image, namely, the best shot image. Thereby, the desired number of best shot images are detected and printed from among the group of moving image data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method for detecting a representative image from moving image data, a program for realizing the image processing method, and a medium storing the program.

  In recent years, cameras have appeared that detect notable subjects such as faces from captured images, evaluate facial expressions on the detected faces, and perform various processes in accordance with the evaluation values.

  For example, Patent Document 1 proposes the following technique in order to enable a user to easily select and print an image with a good expression (best shot image) from photographed images. That is, a person's face is detected from the photographed image, and priority is given to the image using the smile evaluation value (smile evaluation value) for the detected face, and the images are displayed on the monitor in descending order of priority. An image evaluation apparatus has been proposed. The smile evaluation value is calculated based on an evaluation point for evaluating a shape such as lips in the face image.

JP 2004-046591 A

  However, when the facial expression of a person in a photographed image is evaluated with the smile evaluation value as in the above prior art, there are the following problems. That is, when the captured image is moving image data, all images whose smile evaluation value exceeds the threshold value are displayed on the monitor, and as a result, many images of similar scenes can be displayed on the monitor. There is sex. In such a case, there is a problem that it takes a long time until the user selects and prints the best shot image.

  In view of the above-described conventional problems, the present invention provides an image processing apparatus, an image processing method, a program, and a medium that can easily and accurately detect a representative image (best shot image) from a moving image data group. The purpose is to provide.

  In order to achieve the above object, the image processing apparatus of the present invention detects a predetermined subject from a group of moving image data arranged in time series, and obtains a continuous detection time acquisition time for continuously detecting the predetermined subject. Means for specifying a section in the moving image data group according to the continuous detection time acquired by the continuous detection time acquisition means, and detecting a representative image from the moving image data in the specified section And a representative image detecting means.

  According to the present invention, it is possible to easily and accurately detect a representative image from a moving image data group.

1 is a block diagram illustrating a configuration of a digital video camera that is an embodiment of an image processing apparatus of the present invention. It is a schematic diagram which shows an example showing a mode that a moving image data group changes with a time axis. It is a flowchart which shows operation | movement of the area specific | specification part based on 1st Embodiment. It is a figure which shows the screen and moving image data group which are used by 2nd Embodiment. It is a flowchart which shows operation | movement of the area specific | specification part based on 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
<Configuration of digital video camera>
1A and 1B are block diagrams showing the configuration of a digital video camera which is an embodiment of the image processing apparatus of the present invention. FIG. 1A is a block diagram showing the overall configuration. FIG. 5B is a block diagram showing the configuration of the characteristic part.

  As shown in FIG. 1A, the digital video camera 1 includes an imaging lens 101 and an imaging element 102 composed of a CCD image sensor or a CMOS image sensor, and a light beam representing a subject image is condensed by the imaging lens 101. Is incident on the image sensor 102. The image sensor 102 outputs an electrical signal corresponding to the intensity of the incident light beam in units of pixels. This electrical signal is a video signal.

  The analog signal processing unit 103 performs analog signal processing on the video signal output from the image sensor 102. The video signal output from the analog signal processing unit 103 is converted into a digital data format by the A / D conversion unit 104 and input to the image processing unit 106.

  An image extraction unit 105, a face detection unit 107, a continuous detection time measurement unit 108, a display unit 109, a recording medium 110, and a print processing unit 111 are connected to the image processing unit 106.

  The video signal output from the image processing unit 106, that is, moving image data is supplied to the face detection unit 107. The face detection unit 107 detects the face of a person in the image. A known face detection method is used as the detection method. Known techniques for face detection include a method of using knowledge about the face (skin color information, parts such as eyes, nose, mouth, etc.) and a method of configuring a classifier for face detection by a learning algorithm represented by a neural network. is there. In order to improve the recognition rate, face recognition is generally performed by combining them. Specifically, a face detection method using wavelet transform and image feature amount described in JP-A-2002-251380 may be used.

  The face information of the person detected by the face detection unit 107 is sent to the continuous detection time measurement unit 108. The continuous detection time measurement unit 108 (continuous detection time acquisition unit) measures the time when the face information is continuously detected, and supplies the continuous detection time information as a result thereof to the image processing unit 106. The image processing unit 106 adds the continuous detection time information supplied from the continuous detection time measurement unit 108 to the moving image data and supplies the moving image data to the image extraction unit 105.

  The moving image data output from the image processing unit 106 is also sent to the display unit 109. The display unit 109 is, for example, an LCD or an organic EL display, and displays moving image data. The moving image data output from the image processing unit 106 is recorded on a recording medium 110 (for example, a removable memory card). The moving image data output from the image processing unit 106 is also sent to the print processing unit 111, and the print processing unit 111 prints the image of the supplied moving image data.

  The operation of each component described above is controlled by the system control unit 100. In addition, an operation unit 120 for a user to input an instruction is connected to the system control unit 100.

  Next, with reference to FIG. 1B, a detailed configuration of the image extraction unit 105 that characterizes the present embodiment will be described.

  The image extraction unit 105 includes a continuous detection time acquisition unit 105a, a section identification unit 105b, a representative image detection unit 105c, and a control unit 105d. The continuous detection time acquisition unit 105a acquires continuous detection time information of the subject added to the moving image data, and supplies the continuous detection time information to the section specifying unit 105b. The control unit 105d supplies the print number information designated from the operation unit (not shown) to the section specifying unit 105b.

  The section specifying unit 105b specifies a predetermined section from the moving image data according to the number of printed sheets specified by the control unit 105d and the continuous detection time information supplied from the continuous detection time acquisition unit 105a, and specifies the specified section. Information is supplied to the representative image detection unit 105c. The representative image detection unit 105c detects a representative image, that is, a representative image (best shot image) from a specific section of the supplied moving image data. A known representative image detection method is used as the representative image detection method. This known technique includes, for example, a method in which a smile evaluation value of, for example, a smile is given to a detected face, and an image with the maximum smile evaluation value is detected as a representative image. Specifically, for example, there is a detection method described in Japanese Patent Application Laid-Open No. 2008-042319. In this detection method, a face is detected, and a facial expression evaluation value indicating the degree of closeness to one facial expression between a smiling face and a different facial expression such as a normal facial expression is calculated. For example, there is a method of recording an image signal on a recording medium when the value exceeds a predetermined threshold.

  The representative image detected by the representative image detection unit 105c is supplied to the print processing unit 111 through the image processing unit 106, and the print processing unit 111 prints the supplied representative image.

<Moving image data according to the first embodiment>
FIG. 2 is a schematic diagram showing an example of how the moving image data group 200 changes on the time axis.

  The moving image data group 200 includes moving image data 201, 202, 203, 204, and 205 arranged in time series. The moving image data 201 is moving image data from time code t1 to time code t2, and moving image data 202 is moving image data from time code t2 to time code t3. The moving image data 203 is moving image data from time code t3 to time code t4, and moving image data 204 is moving image data from time code t4 to time code t5. The moving image data 205 is composed of moving image data from time code t5 to time code t6.

  When the moving image data group 200 is input to the continuous detection time acquisition unit 105a, the continuous detection time acquisition unit 105a acquires subject continuous detection time information in the moving image data group 200. In the case of the moving image data group 200, for the moving image data 201 to 205, (t2-t1) time information, (t3-t2) time information, (t4-t3) time information, and (t5-t4) time information, respectively. , (T6-t5) time information is acquired.

<Operation of the section identifying unit according to the first embodiment>
Next, the operation of the section specifying unit 105b will be described with reference to FIG.

  FIG. 3 is a flowchart showing the operation (section specifying process) of the section specifying unit 105b according to the first embodiment.

  First, in step S301, the specific section total number zmax indicating the total number of sections existing in the entire moving image data group is acquired from the continuous detection time information output from the continuous detection time acquisition unit 105a. For example, in the case of the moving image data group 200 shown in FIG. 2, the specific section total number zmax = 5.

  In the next step S302, the number of printed sheets n is acquired from the control unit 105d, and in step S303, it is determined whether or not the number of printed sheets n is greater than the specific section total number zmax. If the number n of printed sheets is larger than zmax in step S303, the value of n is updated to n = zmax in step S304. If the number n of printed sheets is smaller than zmax in step S303, step S304 is skipped and the process proceeds to step S305.

  Next, in step S305, the specific section No. indicating the number of the specified section is displayed. Z is initialized to z = 0, and in the subsequent step S306, calculation processing of z = z + 1 is performed, and the value of z is updated. Next, in step S307, a zone in which the continuous detection time supplied from the continuous detection time acquisition unit 105a is the zth longest is specified. In the case of the moving image data group 200, z = the section from time code t5 to t6, which is the first detection time, that is, moving image data 205, is specified as the moving image data to be detected for representative image detection. It will be.

  In step S308, the specified section information is supplied to the representative image detection unit 105c. In subsequent step S309, n = n−1 is calculated, and the value of n is updated. Next, in step S310, it is determined whether or not n = 0. If n = 0, the process returns to step S306. If n = 0, the section specifying process ends.

  In the section specifying process described above, for example, when the number of printed sheets is specified as n = 2 in the moving image data group 200, the upper two sections with a long subject detection time are supplied to the representative image detection unit 105c. . In this example, information on the section from time code t5 to t6 and the section from time code t1 to t2 is supplied to the representative image detection unit 105c.

  The representative image detection unit 105c detects a total of two representative images, one representative image in the section from time code t5 to t6, and one representative image in the section from time code t1 to t2. The detection result is supplied to the print processing unit 111 through the image processing unit 106. The print processing unit 111 prints the supplied representative image.

<Advantages of First Embodiment>
According to the present embodiment, moving image data having a long continuous detection time in the moving image data group is set as the moving image data of interest for detecting the representative image, that is, the best shot image. Accordingly, it is possible to detect and print a desired number of best shot images from the moving image data group without performing face recognition of the subject.

<Modification according to the first embodiment>
In the present embodiment, the section in which the continuous detection time supplied from the continuous detection time acquisition unit 105a in step S307 is the zth longest is specified. At this time, if the supplied continuous detection times are the same or similar to each other, for example, a predetermined subject such as a large subject on the screen or a face facing the front is predetermined. The section is specified in consideration of the detection state of the subject.

[Second Embodiment]
In the second embodiment, the section specifying process by the section specifying unit 105b is different from the first embodiment.

  FIG. 4A is a screen diagram illustrating an example of a display screen of the display unit 109 according to the second embodiment.

  As shown in the figure, the screen of the display unit 109 displays options such as “number of printed sheets from the same shooting date / time image m sheets” and “number of printed sheets n sheets”. The user can designate the number m of prints from the same shooting date / time image and the number n of prints from the entire moving image data group for the moving image data by using the operation unit 120.

<Moving image data according to the second embodiment>
FIG. 4B is a schematic diagram illustrating an example in which the moving image data group 410 according to the second embodiment changes on the time axis.

  As shown in FIG. 4B, the moving image data group 410 includes time-series moving image data 411, 412, 413, 414, and 415, which are photographed at different dates and times. The moving image data 411 is moving image data from time code t1 to time code t4, and moving image data 412 is moving image data from time code t4 to time code t5. The moving image data 413 is moving image data from time code t5 to time code t6, and moving image data 414 is moving image data from time code t6 to time code t7. The moving image data 415 is moving image data from time code t7 to time code t8.

  The system control unit 100 groups the moving image data group 410 based on the shooting date / time information (attribute information), and divides the moving image data group 410 into groups 411 to 415 (grouping means).

  When the moving image data group 410 is input from the image processing unit 106 to the continuous detection time acquisition unit 105a, the continuous detection time acquisition unit 105a acquires the continuous detection time information of the subject in the moving image data group 410. That is, in the case of the moving image data group 410, for the groups 411 to 415, (t2-t1) time, (t4-t3) time, (t5-t4) time, (t6-t5) time, (t7-t6), respectively. ) And (t8-t7) time continuous detection time information is acquired (see FIG. 4B).

<Operation of Section Identification Unit According to Second Embodiment>
Next, the operation of the section identifying unit 105b according to the second embodiment will be described with reference to FIG.

  FIG. 5 is a flowchart showing the operation (section specifying process) of the section specifying unit 105b according to the second embodiment.

  First, in step S501, the specific section total number zmax indicating the total number of sections existing in the entire moving image data group is acquired from the continuous detection time information output from the continuous detection time acquisition unit 105a. In the next step S502, the number of sections Sn included in each moving image data group grouped by the control unit 105d and the maximum number of sections Smax are acquired.

  For example, in the case of the moving image data group 410, in the group 411, since two sections indicated by the moving image data 411a and 411b are detected, the number of sections S1 = 2. Since the group 412 detects one section indicated by the moving image data 412a, the number of sections S2 = 1, and since the group 413 detects one section indicated by the moving image data 413a, the number of sections S3 = 1. . In the group 414, since one section indicated by the moving image data 414a is detected, the number of sections S4 = 1. In the group 415, since one section indicated by the moving image data 415a is detected, the value of the number of sections S5 = 1 is acquired.

  In step S503, the control unit 105d acquires the number of printed sheets m from the same shooting date / time image and the number of printed sheets n from the entire moving image data group. The number of printed sheets m from the same shooting date / time image is limited to the maximum number of sections Smax ≧ m. Further, the number n of prints is limited to the total number of specific sections zmax ≧ n.

  In step S504, it is determined whether the number of sections Sn of the grouped moving image group is larger than the designated number m. If Sn is larger than the designated number m in step S504, the value of Sn is updated to Sn = m in step S505. If Sn is smaller than the designated number m in step S504, the process proceeds to step S506.

  In step S506, the specific section No. indicating the number of the specified section is displayed. Z is initialized to z = 0. In step S507, z = z + 1 is calculated and the value of z is updated. In a subsequent step S508, a zone in which the continuous detection time supplied from the continuous detection time acquisition unit 105a is the zth longest is specified. For example, in the case of the moving image data group 410, the section from the time code t7 to t8 of the moving image data 415 having the first detection time z = 1 is specified.

  In the next step S509, the value of Sn in the identified section is updated to Sn = Sn-1. For example, in the case of the moving image data group 410, since the section of S5 is specified in step S508, the calculation process of S5 = S5-1 is performed, and the value of S5 is updated. The values of S1 to S3 which are other sections are not updated. In the next step S510, information on the identified section is supplied to the representative image detection unit 105c.

  In a succeeding step S511, it is determined whether or not the value of Sn is 0. If Sn = 0, in the next step S512, the moving image data group of the specified Sn group is excluded from the range to be specified next. For example, in the case of the moving image data group 410, S5 = 0 is obtained by the calculation process of S5 = S5-1.

  If Sn = 0 is not satisfied in step S511, the process skips step S512 and proceeds to step S513. In the next step S513, calculation processing of n = n−1 is performed, and the value of n is updated. In subsequent step S514, it is determined whether or not n = 0. If n = 0 is not satisfied, the process returns to step S507. If n = 0 in step S514, the section specifying process is terminated.

  For example, in the moving image data group 410, when m = 1 and n = 3 are designated in step S503, it means that one print is designated from the image data having the same shooting date and time, and a total of three prints are designated. . That is, one representative image is detected from each of the sections of the moving image data 415 from the time code t7 to t8, the section of the moving image data 411 from the time code t1 to t2, and the section of the moving image data 413 from the time code t5 to t6. .

  If m = 2 and n = 3 are designated in step S503, it means that printing is performed for a total of three sheets, two sheets from the image data with the same shooting date and time. That is, representative images are detected one by one from the time code t7 to t8 section of the moving image data 415, from the time code t1 to t2 section of the moving image data 411, and from the time code t3 to t4 section of the moving image data 411. To do.

  The print processing unit 111 prints the representative image detected by the representative image detection unit 105c.

<Advantages of Second Embodiment>
According to the present embodiment, it is possible to easily detect and print a representative image corresponding to the shooting date and time.

<Modification according to Second Embodiment>
In the present embodiment, the moving image data group is grouped based on the shooting date / time information, but may be grouped based on the shooting end time information from the start of shooting.

  Moreover, in this Embodiment, the continuous detection time supplied from the continuous detection time acquisition part 105a in step S508 specified the z-th area. On the other hand, when the supplied continuous detection times are the same or similar, for example, the moving image data classified by the grouping means is not specified from the moving image data group of the already specified group. The section is specified taking into account the situation of the specified section of the group.

[Other embodiments]
In each of the above embodiments, a digital video camera has been described as an example. Needless to say, the present invention can also be applied to a reproduction apparatus and a print processing apparatus.

  The embodiment of the present invention can also be realized by executing software (program) acquired via a network or various storage media on a personal computer (CPU, processor).

105 Image Extraction Unit 105a Continuous Detection Time Acquisition Unit 105b Section Identification Unit 105c Representative Image Detection Unit 105d Control Unit 106 Image Processing Unit 108 Continuous Detection Time Measurement Unit

Claims (11)

Continuous detection time acquisition means for detecting a predetermined subject from a group of moving image data arranged in time series and acquiring a time at which the predetermined subject is continuously detected;
In accordance with the continuous detection time acquired by the continuous detection time acquisition means, a specifying means for specifying a section in the moving image data group;
An image processing apparatus comprising: representative image detection means for detecting a representative image from moving image data in the specified section.   The image processing apparatus according to claim 1, wherein the specifying unit specifies, as the section, a higher rank among the continuous detection times acquired by the continuous detection time acquisition unit.   The specifying unit specifies the section based on a detection state of a predetermined subject when the continuous detection times acquired by the continuous detection time acquisition unit are continuous detection times similar to each other. Item 3. The image processing apparatus according to Item 1 or 2. Grouping means for grouping the moving image data groups;
4. The image according to claim 1, further comprising: a unit that limits a number of sections specified by the specifying unit in the moving image data group grouped by the grouping unit. 5. Processing equipment.   In the case where the continuous detection times acquired by the continuous detection time acquisition unit are similar to each other, the specifying unit determines the status of the section specified in the moving image data group grouped by the grouping unit. The image processing apparatus according to claim 4, wherein the section is specified based on the information.   6. The image processing apparatus according to claim 4, wherein the grouping unit performs grouping based on attribute information of the moving image data group.   The image processing apparatus according to claim 6, wherein the attribute information is information on a shooting end time from a shooting start of the moving image data.   The image processing apparatus according to claim 6, wherein the attribute information is information on a shooting date and time of the moving image data. A continuous detection time acquisition step of detecting a predetermined subject from the moving image data group arranged in time series and acquiring a time at which the predetermined subject is continuously detected;
In accordance with the continuous detection time acquired by the continuous detection time acquisition step, a specifying step of specifying a section in the moving image data group,
An image processing method comprising: a representative image detection step of detecting a representative image from moving image data in the specified section. A computer-readable program for executing an image processing method,
A continuous detection time acquisition step of detecting a predetermined subject from the moving image data group arranged in time series and acquiring a time at which the predetermined subject is continuously detected;
In accordance with the continuous detection time acquired by the continuous detection time acquisition step, a specifying step for specifying a section in the moving image data group;
A program comprising: a representative image detecting step for detecting a representative image from moving image data in the specified section.   A medium for storing the program according to claim 10.