JP2007193592A - Image processing apparatus, image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suitably group image data in accordance with its generation time. <P>SOLUTION: A receiving terminal as an image recoding apparatus is provided with a control means 11 to be applied as: a differential value calculation means for calculating a differential value in the generation time of image data to be grouped on the basis of the date information of the image data to be grouped out of image data recorded in a recording medium M as a recording medium for recording image data having date information indicating the generation time as additional information; a threshold setting means for setting the threshold of a generation time interval for group division in accordance with the date information of the image data to be grouped; and a group setting means for grouping and setting the image data to be grouped by defining a position where the calculated differential value corresponding to the date information of the image data to be grouped is larger than the set group dividing threshold corresponding to the date information as a group dividing position. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  In recent years, digital cameras have been widely used in place of analog cameras. In addition, the memory (media) for recording and storing photographed image data is also increasing in capacity and price, and a huge number of image data can be recorded in one memory. Due to this increase in recorded image data, the need for image management has arisen, and for example, a technique for grouping a series of photographed images, such as travel, has been desired and studied.

As a method of grouping image data, a configuration in which the same group is set when the difference in photographing time between image data is smaller than a threshold value is considered (for example, see Patent Document 1).
JP 2001-228528 A

  However, in the conventional grouping method, there is a risk that the proper grouping desired by the user may not be achieved. Specifically, since the same threshold is used for old image data and new image data, old photographed image data that is less frequently ordered is grouped in the same way as new photographed image data, and a group of old photographed image data The number was larger than the number desired by the user.

  For this reason, there has been a problem in that the user's work time and work load increase during a print ordering operation for selecting image data to be printed from the image data of each group.

  An object of the present invention is to appropriately group image data according to the generation time thereof.

In order to solve the above problems, an image processing apparatus according to claim 1 is provided.
The generation time of the grouping target image data based on the date information of the grouping target image data among the image data recorded in the recording unit in which the image data having the generation time information indicating the generation time is recorded as additional information. Difference value calculating means for calculating the difference value of
Threshold setting means for setting a generation time interval threshold for group separation according to the date and time information of the image data to be grouped;
The image data of the grouping target is a position where the calculated difference value corresponding to the date / time information of the image data of the grouping target is larger than the set threshold for group separation corresponding to the date / time information of the grouping target. Group setting means for grouping and setting
It is characterized by providing.

The invention according to claim 2 is the image processing apparatus according to claim 1,
The grouping threshold value is larger as the date / time information is older.

The invention according to claim 3 is the image processing apparatus according to claim 1 or 2,
Output means for outputting the grouped image data for each group is provided.

The invention according to claim 4 is the image processing apparatus according to claim 3,
The output means displays the grouped image data for each group.

The invention according to claim 5 is the image processing apparatus according to claim 4,
Determining means for determining representative image data of the group from image data belonging to the group divided into groups;
The output means displays the representative image data.

The invention according to claim 6 is the image processing apparatus according to claim 5,
First input means for receiving selection input of the representative image data;
Generating means for generating thumbnail image data of image data belonging to a group of representative image data selected and input by the first input means,
The output means displays the generated thumbnail image data.

The invention according to claim 7 is the image processing apparatus according to any one of claims 1 to 6,
Second input means for receiving selection input of image data to be printed out of the grouped image data;
Transmitting means for transmitting the image data selected and input by the second input means to the image forming means as print image data;
It is characterized by providing.

An image processing method according to an eighth aspect of the present invention includes:
The generation time of the grouping target image data based on the date information of the grouping target image data among the image data recorded in the recording unit in which the image data having the generation time information indicating the generation time is recorded as additional information. A difference value calculating step for calculating a difference value of
A threshold setting step for setting a generation time interval threshold for group separation according to date and time information of the image data to be grouped;
The grouping target image data with the calculated difference value corresponding to the date / time information of the grouping target image data being larger than the set grouping threshold corresponding to the date / time information as a grouping position. A group setting process for grouping and setting
It is characterized by including.

The invention according to claim 9 is the image processing method according to claim 8,
The grouping threshold value is larger as the date / time information is older.

The invention according to claim 10 is the image processing method according to claim 8 or 9, wherein
An output step of outputting the grouped image data for each group is included.

The invention according to claim 11 is the image processing method according to claim 10,
The output step is a step of displaying the grouped image data for each group.

The invention according to claim 12 is the image processing method according to claim 11,
A determination step of determining representative image data of the group from the image data belonging to the group divided into groups,
In the output step, the representative image data is displayed.

The invention according to claim 13 is the image processing method according to claim 12,
A first input step for receiving selection input of the representative image data;
Generating thumbnail image data of image data belonging to the group of representative image data selected and input in the first input step, and
In the output step, the generated thumbnail image data is displayed.

The invention according to claim 14 is the image processing method according to any one of claims 8 to 13,
A second input step for receiving selection input of image data to be printed out of the grouped image data;
And a transmission step of transmitting the image data selected and input in the second input step as image data for printing to the image forming means.

The image processing program of the invention according to claim 15
Computer
Based on the date / time information of the grouping target image data among the image data recorded in the recording means in which the image data having the date / time information indicating the generation time is recorded as additional information, the generation time of the grouping target image data Difference value calculation means for calculating the difference value of
Threshold setting means for setting a generation time interval threshold for group separation according to the date and time information of the image data to be grouped;
The image data of the grouping target is a position where the calculated difference value corresponding to the date / time information of the image data of the grouping target is larger than the set threshold for group separation corresponding to the date / time information of the grouping target. Group setting means for grouping and setting
It is made to function as.

The invention according to claim 16 is the image processing program according to claim 15,
The grouping threshold value is larger as the date / time information is older.

The invention according to claim 17 is the image processing program according to claim 15 or 16,
The computer,
It is made to function as an output means for outputting the grouped image data for each group.

The invention according to claim 18 is the image processing program according to claim 17,
The output means displays the grouped image data for each group.

The invention according to claim 19 is the image processing program according to claim 18,
The computer,
From the image data belonging to the group divided into groups, function as a determining means for determining representative image data of the group,
The output means displays the representative image data.

The invention according to claim 20 is the image processing program according to claim 19,
The computer,
First input means for receiving selection input of the representative image data;
Functioning as generation means for generating thumbnail image data of image data belonging to the group of representative image data selected and input by the first input means,
The output means displays the generated thumbnail image data.

The invention according to claim 21 is the image processing program according to any one of claims 15 to 20,
The computer,
Second input means for receiving selection input of image data to be printed out of the grouped image data;
The image data selected and input by the second input unit is made to function as a transmission unit that transmits image data for printing to the image forming unit.

  According to the present invention, image data can be appropriately grouped according to the generation time.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

First, the configuration of this embodiment will be described with reference to FIGS.
FIG. 1 shows a configuration of an image processing system 100 of the present embodiment.

  As shown in FIG. 1, the image processing system 100 is a reception terminal 1 as an image processing apparatus of the present invention for receiving a print order from a customer (user), and manages information related to the print order of each customer. The order management apparatus 2 includes an image printer unit 4 that prints image data, and an image media writing unit 5 that writes image information on a medium. The order management device 2 is connected to a reception terminal 1, an image printer unit 4, and an image media writing unit 5. Note that the numbers of the receiving terminal 1, the image printer unit 4, and the image media writing unit 5 connected to the order management apparatus 2 are not limited to the example of FIG. 1. The order management device 2 and the image printer unit 4 function as image forming means.

  In FIG. 2, the principal part structure of the reception terminal 1 is shown. The reception terminal 1 is installed in a store for receiving a print order. As shown in FIG. 2, as a difference value calculation means, a threshold setting means, a group setting means, a determination means, a control section 11 as a generation means, and a display means Display section 12, first input means, operation input section 13 as second input means, media reading section 14, storage section 15, order confirmation slip issuing section 16, and communication section 17 as transmission means. These units are connected to each other via a bus 18.

  The control unit 11 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like. In the control unit 11, various processing programs stored in the storage unit 15 are read out and expanded in the RAM, and the operation of each unit constituting the reception terminal 1 is controlled in cooperation with the expanded program and the CPU. The

  The display unit 12 is configured by a display such as an LCD (Liquid Crystal Display), and performs a required display process according to a display control signal input from the control unit 11.

  The operation input unit 13 includes a numeric keypad, cursor keys, various function keys, and the like, and outputs an operation signal generated by pressing the key to the control unit 11. Further, the operation input unit 13 has a touch panel provided so as to cover the display display of the display unit 12, and detects a coordinate instructed to be touched by a coordinate reading principle such as an electromagnetic induction type, a magnetostriction type, a pressure sensitive type or the like. The detected coordinates are output to the control unit 11 as a position signal.

  The media reading unit 14 is a medium (recording device) such as a memory stick (registered trademark), smart media (registered trademark), compact flash (registered trademark), multimedia card (registered trademark), SD memory card (registered trademark), or PC card. Means) M is configured to be mountable, and reads data recorded on the mounted medium M. In the present embodiment, these media brought by the user are used as the recording means of the present invention. However, a hard disk built in or externally attached to the reception terminal 1, an image server connected to the reception terminal 1, etc. May be used as recording means.

  The storage unit 15 stores various processing programs executed by the control unit 11 and data used when these processing programs are executed. The storage unit 15 stores a first print acceptance program for executing the first print acceptance process.

  The order confirmation form issuing unit 16 prints out an order confirmation form or a receipt for allowing the user to confirm the contents of the print order in accordance with a control signal from the control unit 11.

  The communication unit 17 includes a modem, a terminal adapter, a LAN adapter, and the like, and performs communication with an external device (order management device 2) via a communication network such as a telephone line, an ISDN line, or a dedicated line. Take control.

  FIG. 3 shows a main part configuration of the order management apparatus 2. As shown in FIG. 3, the order management apparatus 2 includes a control unit 21, a display unit 22, an operation input unit 23, a storage unit 24, an I / F (interface) unit 25, a communication unit 26, and a DB reference unit 27. These units are connected to each other via a bus 28.

  The control unit 21 includes a CPU, a RAM, and the like. In the control unit 21, various processing programs stored in the storage unit 24 are read out and expanded in the RAM, and the operation of each unit constituting the order management apparatus 2 is controlled in cooperation with the expanded program and the CPU. Is done. Specifically, the control unit 21 executes a first print management process to be described later based on a first print management program stored in the storage unit 24.

  The display unit 22 is configured by a display such as an LCD, and performs a required display process according to a display control signal input from the control unit 21.

  The operation input unit 23 includes a numeric keypad, cursor keys, various function keys, and the like, and outputs an operation signal generated by pressing the key to the control unit 11. The operation input unit 23 may be provided with a touch panel so as to cover the display display of the display unit 22.

  The storage unit 24 stores various processing programs executed by the control unit 21 and data used when these processing programs are executed. The storage unit 24 stores a first print management program for executing the first print management process.

  The I / F unit 25 is a communication module capable of data communication with an external device, and outputs data to the image printer unit 4 and the image media writing unit 5 in accordance with a control signal from the control unit 21.

  The communication unit 26 includes a modem, a terminal adapter, a LAN adapter, and the like, and performs communication control for communicating with an external device (receiving terminal 1) via a communication network such as a telephone line, an ISDN line, or a dedicated line. I do.

  The DB reference unit 27 reads information stored in a DB (DataBase) unit 3 and writes information in accordance with an instruction from the control unit 21.

  Returning to FIG. The DB unit 3 stores service history information indicating a history of past print order services performed for the customer.

  The image printer unit 4 performs print output by a predetermined printing method (electrophotographic method, ink jet method, thermal sublimation method, etc.) according to an instruction from the order management device 2.

  The image media writing unit 5 is configured to be able to load a medium such as a CD-R, and writes image information to the loaded medium according to an instruction from the order management apparatus 2.

  Next, the operation in the present embodiment will be described with reference to FIGS. FIG. 4 shows a flow of a first print acceptance process executed in the reception terminal 1 and a first print management process executed in the order management apparatus 2. FIG. 5 shows the flow of the first grouping image display process.

  First, the first print acceptance process executed by the control unit 11 in the acceptance terminal 1 when accepting a print order will be described with reference to the flowchart of FIG.

  The first print acceptance process is executed by the control unit 11 triggered by the insertion of the medium M on which the image data is recorded into the media reading unit 14 of the reception terminal 1. First, the grouping target image data of the medium M in the media reading unit 14 is read, the image data is grouped based on the date / time information added to the grouping target image data, and the grouped image data is displayed on the display unit 12. A first grouping image display process to be displayed is performed (step S11).

  With reference to FIG. 5, the first grouping image display process (step S11) executed under the control of the control unit 11 will be described in detail.

  First, grouping target image data is read from the image data recorded on the medium M, and the number N of images of the grouping target image data (N is an integer equal to or greater than 0) is acquired (step S111). Then, it is determined whether or not the number N of images is larger than a predetermined number set in advance (step S112).

  When the image number N is equal to or less than the predetermined number (step S112; NO), the process proceeds to step S122.

  When the number N of images is larger than the predetermined number (step S112; YES), date / time information of the grouping target image data is acquired (step S113). The date and time information is information indicating the date and time of generation and update of image data, and if it is image data generated by shooting with a shooting device such as a digital camera, information indicating the date and time when a predetermined image was generated by shooting, So-called shooting date / time information is preferable. In addition, if the image data is generated using an editor such as a CG (Computer Graphics) image, the generation date / time or update date / time of the image data may be used as the date / time information.

  Regarding the acquisition of date and time information, existing tag information stipulated in various general-purpose image formats represented by JPEG (Joint Photographic Coding Experts Group), TIFF (Tagged Image File Format), Exif (Exchangeable Image File Format), etc. However, the present invention is not limited to this. For example, information set in a freely usable area such as a maker note may be used.

  When the date / time information of the grouping target image data is acquired, the image data of the grouping target image data is sorted in the date / time information order (generation order) based on the acquired date / time information (step S114). Numbers (1 to N) are assigned to the sorted image data in this sort order.

  Then, 1 is set as an initial value in the variable I (step S115). After execution of step S115, the difference between the date information of the I-th and (I + 1) -th image data is calculated (step S116), and a threshold value corresponding to the date-time information of the I-th and (I + 1) -th image data is set. (Step S117).

  The threshold value corresponding to the date / time information is based on the latest date / time information of the grouping target image data or the date on which the user performs order processing, and the difference between the reference date and the date / time information of each image data increases (that is, the old image It is preferable that the value is set to a larger value. For example, if the difference value is less than 3 months, the threshold is 2 days, if the difference value is 3 months or more and less than 1 year, 1 month is set, and if it is more than 12 months, the threshold is set to 12 months. Further, the threshold value is set in advance and stored in the storage unit 15, for example, and is acquired and set by reading the date / time information as a key in step S117.

  Then, it is determined whether or not the difference between the date information of the I-th and (I + 1) -th image data calculated in step S116 is larger than the threshold acquired in step S117 (step S118). If the difference between the date and time information is larger than the threshold (step S118; YES), a group break position is set between the I-th and (I + 1) -th image data (step S119). For example, the group break position is stored in the storage unit 15 as the grouping result together with the grouping target image data.

  And it is discriminate | determined whether the variable I is smaller than (N-1) (step S120). When the variable I is smaller than (N−1) (step S120; YES), the variable I is incremented by +1 (step S121), and the process proceeds to steps S116 and S117. If the difference between the date and time information is equal to or smaller than the threshold (step S118; NO), the process proceeds to step S120.

  When grouping target image data is not grouped (when no group separation position is set), all of the grouping target image data may be set as one group. If the grouping process is not grouped, for example, the grouping process condition is changed to a condition that facilitates grouping, for example, by setting the threshold value corresponding to the date and time information lower, and the grouping process is applied again. Also good.

  Further, not only the date and time information but also a method of grouping according to shooting locations using GPS (Global Positioning System) information or the like as described in, for example, JP-A-2005-174060 may be added.

  Further, it is preferable to add a step of determining whether or not the grouping process is necessary before step S111 and to prevent the grouping when the grouping process is unnecessary. If it does in this way, while being able to reduce the processing load of the reception terminal 1, waiting time of a user (customer) can be decreased.

  In addition, after grouping processing, the number of image data belonging to one image group is counted, and if the number is equal to or greater than a predetermined number, or if a user instruction is given from a button on the display screen, the grouped images You may apply the grouping process which changed conditions so that it might group into data group more finely. In addition, after the grouping process, when the shooting date range of the image data belonging to one image group is longer than a predetermined period, the grouping is performed by changing the conditions so that the grouped image data group is further divided into groups. Processing may be applied. As a grouping process whose conditions have been changed so as to be further finely grouped, for example, the grouping process conditions are changed to a condition that facilitates grouping, for example, by setting a threshold value corresponding to date and time information lower, and grouping is performed again. Processing may be applied.

  When the variable I is equal to or greater than (N-1) (step S120; NO), grouping target image data is displayed on the display unit 12 for each group (image group) as a grouping result (step S122), and the first grouping image is displayed. The display process ends. Various methods can be used to display the grouping result.

  As a display method of the grouping result, two-dimensional display is performed such that an image group sign or representative image data or head image data image is arranged in one direction and an image data image belonging to the image group is arranged in the other direction. May be. FIG. 6 shows an example of the grouping result display screen. For example, as shown in FIG. 6, the image of the representative image data of groups G1 to G3 is arranged in the vertical direction on the left side of the grouping result display screen, and the image of the image data belonging to each group G1 to G3 is arranged in the horizontal direction. It is good. In the grouping result display screen of FIG. 6, the display portions of the groups G1 to G3 have arrow key portions g1 to g3 as touch panels, respectively. When the user touches the arrow key portions g1 to g3, an undisplayed image can be displayed. In the example of FIG. 6, the arrow key portions g1 and g2 of the groups G1 and G2 with the undisplayed image accept touch input, and the arrow key portion g3 of the group G3 with no undisplayed image does not accept the touch input. The arrow key parts g1 and g2 are displayed in a different color and pattern from the arrow key part g3 so as to indicate that the touch input is being accepted. The grouping result display screen shown in FIG. 6 is configured so that image data to be printed can be selected corresponding to step S12 described later.

  As another example of the display method of the grouping result, a calendar may be displayed on the display unit 12, and a group indicator and / or representative image data may be displayed on the calendar for each group or each date. FIG. 7 shows an example of a calendar display screen as a grouping result. For example, the group G4 after grouping may be displayed as shown in FIG.

  The format of the calendar to be displayed is preferably set according to the date / time information of the image data included in the image group obtained by the grouping process, and the latest and oldest range of date / time information (hereinafter referred to as “group date / time range”). It is preferable to set according to. For example, in the case of image data generated by shooting, it may be set according to the latest and oldest range of shooting date and time.

  In setting the calendar format, it is preferable to set both the display unit of the calendar to be displayed (one-month calendar, two-month calendar, etc.) and the date and time range to be displayed (from when to when).

  As for the setting of the display unit of the calendar to be displayed, it is preferable to set a calendar format of a period including the group date / time range of all the grouped image groups, for a minimum of one month, two months, three months,. It is preferable to increase in units of one month. For example, if the group date / time range is 3 days, the 1-month format is used. If the total of the group date / time ranges of a plurality of image groups is 1 month and a half, the 2-month format is used. However, it goes without saying that the display unit and date / time range of the calendar to be displayed may be changed according to the request of the user (store) of the image processing system 100, the customer's usage record, and the like. For example, even if the group date and time range is 3 days, the format may be 2 weeks or 1 week instead of 1 month, or if the image data of one image group is captured within one day, the clock You can also set the time display format like a time table.

  Regarding the setting of the date and time range to be displayed on the calendar, when displaying on the display unit 12 in the above-described calendar format, an existing range in a general calendar (for example, setting) is set so that the user can easily grasp the group date and time range. If the calendar display unit is 1 month, the calendar display unit (for example, 1 month format) and the date and time including the group date and time range, regardless of the month of the month (1st to 31st) It is preferable to set the range. Furthermore, it is preferable to set the group date and time range so that it is located near the center of the calendar period so that the user can easily view the group date and time range on the display unit 12.

  For example, if the group date / time range is July 31 to August 6, the calendar display unit is set to one month, and the existing date / time range of the general calendar for July and August (July 1 Day to July 31, August 1 to August 31) is set as the date and time range of the calendar, and is displayed at the same time or by switching, or the display unit of the calendar is set to 2 months. July 1st to August 31st, which is the sum of the general calendars of the month and August, is not set as the date / time range displayed on the calendar, but July 31st to August 6th is 7 days. Therefore, the display unit of the calendar is set as one month, so that July 31 to August 6 are located near the center of one month, for example, as in group G4 shown in FIG. Setting to display 1 month from 15th to 14th of August The may be.

  Furthermore, when the group date / time range extends over a plurality of months (for example, July 31 to August 6), a plurality of months are continuously displayed without being divided into months as in the group G4 shown in FIG. It is preferable to set the date arrangement on the calendar. In this case, in order to distinguish each month, it is preferable to display a boundary line on the calendar or change the background color for each month. Moreover, you may make it mark on the date on which the customer visited the store where the reception terminal 1 was installed on a calendar.

  In this embodiment, in order to make the explanation easy to understand, the expression of one month, two months,... Is used as a calendar display or a calendar display unit, and the date and time range to be displayed on the calendar is set to 1. Although the month is explained as about 30 days, in actual processing, as in a general calendar, one month is 4 to 6 weeks (even if the number of days in each month and the date and day of the week vary) (It may be fixed as 5 weeks), and the date and time range displayed on the calendar is set to 4 to 4 so that the group date and time range is near the center (not about 30 days as described above). Needless to say, a period of 6 weeks may be set.

  The image group representative image data is selected by selecting the image data with the newest or oldest date / time information in the image group as the representative image data, or the feature quantity closest to the average value of the feature quantities of all the image data in the image group. For example, a method known in the art can be used, for example, image data having image data as representative image data (Japanese Patent Laid-Open No. 2004-810436). Here, the feature amount is, for example, any one of hue, saturation, and brightness of image data, or any combination of two or more thereof.

  However, in order to select image data that is easy for the user to see and image data that makes it easy to imagine the contents of the image group, the light distribution of the image data in each image group is calculated, and the light distribution that is most easily observable for the user is taken. The selected image data is preferably selected as representative image data. Specifically, image data photographed with direct light (image data with the highest light intensity) is preferable. If extraction is not possible, it is preferable to select strobe image data. As a light distribution calculation method, the imaging condition may be extracted using tag information such as Exif to calculate (detect) the light distribution, or may be calculated by performing an image analysis process. A known method can be used. A specific example of the light distribution calculation method will be described in detail later with reference to FIGS.

  Furthermore, since the representative image data is preferably image data that makes it easy to grasp the contents of the image data belonging to the image group, select image data that is captured with a person who is likely to remain in the user's memory as the main subject. It is preferable to do. When selecting image data in which a person is a main subject, selection may be made using tag information such as Exif, or may be selected using face extraction processing for extracting a person's face area from image data. . In order to select image data more suitable for representative image data, representative information based on the size, position, and brightness of the face area is used by using tag information such as Exif and face area information extracted by face extraction processing. It is preferable to select image data. The image data to which the face extraction processing is applied is image data with excellent light distribution, that is, image data taken with direct light (the light intensity is a predetermined value or more) because of extraction accuracy and ease of observation by the user. Image data). In selecting the representative image data, it is preferable to select based on the light distribution as described above, and it is preferable to select image data in which a person is the main subject. It is preferable to select based on whether or not is a person.

  As a method of face extraction processing, for example, a method using a two-dimensional histogram of hue and saturation described in JP-A-6-67320, JP-A-8-122944, JP-A-8-184925, A method using the pattern matching process described in Kaihei 9-138471, a pattern search method described in Japanese Patent Laid-Open No. 9-138471, and the like can be used. In addition, various methods such as a method for determining a human face candidate region using a neural network described in Japanese Patent Laid-Open Nos. 5-282457 and 6-214970 can be used. A specific example of face extraction processing using a neural network will be described later in detail with reference to FIGS. 10 and 21 to 24.

  Moreover, in the calendar display, it is preferable to display an index that allows the group date / time range to be visually recognized on the calendar. Specifically, the background color, the colored frame, the mark, and the like are displayed on the calendar for easy visual recognition. Further, when a plurality of image groups are obtained, it is preferable to assign a different index for each image group, such as changing a background color or a colored frame for each image group.

  Furthermore, it is preferable that a sign indicating the content of the image data included in the image group is displayed on the calendar. As such a marker, for example, representative image data of an image group, the number of images (number of images), a shooting location, and the like can be used. It is preferable to set a plurality of signs for one image group. For example, a plurality of representative image data can be selected and sequentially displayed in a slide show format. Alternatively, it is also preferable to set a combination of different types of signs such as representative image data and the number of images. Furthermore, this indicator is preferably set for each date in the image group. For example, for each date, the number of images on that day is displayed, or a representative image is selected for each date in the group date and time range. As shown in FIG. 7, thumbnail image data of representative image data for each date is displayed on the calendar. May be generated and displayed. When the representative image data is displayed on the calendar for each date, the corresponding representative image data may be enlarged and displayed by designating the date on the calendar.

  A plurality of types of signs may be used in one image group. For example, when the group date and time range is two days or more, a sign can be selected for each date, or a plurality of signs (for example, a plurality of representative image data) can be displayed as a slide show.

  Returning to FIG. When the grouping result is displayed on the display unit 12, an ordering operation by the user via the operation input unit 13 is performed based on the grouping result displayed on the display unit 12 (step S12). In step S12, for example, a button for ordering all image data included in the image group displayed on the calendar is arranged on the screen (touch panel), and the button is used for touch input or position input of a pointing device such as a mouse. Select the image data in the image group including the date by specifying the batch order method by specifying the date or by specifying the day on the calendar by the touch input or the position input. It is possible to use a method of ordering each time, a method of selecting an image or sign of representative image data and specifying it by the touch input or the position input. Also, you can select an image group by designating on the calendar, generate and display thumbnail image data of the image data of that image group, confirm all the image data in the image group, and place a batch order or necessary Depending on the situation, it is possible to select image data to be ordered.

  When the ordering operation is completed, a confirmation message for confirming the execution of printing is displayed on the display unit 12 (step S13). Then, it is determined whether or not the execution of printing has been confirmed by the operation of the operation input unit 13 by the user (step S14). When execution of printing is confirmed (step S14; YES), an order confirmation slip is issued from the order confirmation slip issuing unit 16 (step S15), and provisional order information (order information) is ordered via the communication unit 17. The data is transmitted to the management apparatus 2 (step S16), and the first print acceptance process ends. If execution of printing is not confirmed (step S14; NO), the process proceeds to step S12.

  Next, a first print management process executed by the control unit 21 in the order management apparatus 2 will be described. In response to step S16 of the first print acceptance process, the order management apparatus 2 that has received the temporary order information transmitted from the receiving terminal 1 via the communication unit 26, based on the received information, the image printer unit 4 Is output (step T1), and the first print management process ends. In the image printer 4, print output is performed based on the print output instruction signal output from the order management device 2. With the print output, the print order acceptance process is completed.

<Modification>
Next, a case where a print order for image data having solid identification basic information as additional information is performed will be described. The solid identification basic information is data such as a media unique number, a volume label, and a file name. In the service history information stored in the DB unit 3, data such as the file name of a printed image and the service usage status (usage frequency) are stored for each individual identification information (ID number).

  An operation according to a modification of the present embodiment will be described. FIG. 8 shows the flow of the second print acceptance process executed in the reception terminal 1 and the second print management process executed in the order management apparatus 2.

  With reference to FIG. 8, the second print acceptance process executed by the control unit 11 in the acceptance terminal 1 when the print order is accepted will be described. It is assumed that the second print acceptance process is executed by the control unit 11 based on the second print acceptance program stored in the storage unit 15.

  When the medium M on which the image data is recorded is inserted into the media reading unit 14 of the reception terminal 1, the solid identification basic information is acquired by searching the recorded content of the medium (step S 21). Individual identification basic information is transmitted to the order management apparatus 2 via the communication unit 17.

  In the reception terminal 1, it is determined whether or not there is matching information from the reception information received from the order management device 2 via the communication unit 17 (step S22). If it is determined in step S22 that there is matching information (step S22; YES), unprinted image data is selected from the image data recorded on the medium M based on the service history information received from the order management apparatus 2. Is selected (step S23). When it is determined that there is no match information (step S22; NO), all image data recorded on the medium M is selected (step S24).

  When image data is selected in step S23 or S24, a second grouping image display process for displaying images of the image data grouped on the display unit 12 based on the date and time information added to the image data is performed (step S25). ).

  With reference to FIG. 9, the second grouping image display processing in step S25 will be described. First, the image number N (N: an integer equal to or greater than 0) of the image data selected in step S23 or S24 is acquired (step S251). Then, it is determined whether or not the number N of images is larger than a predetermined number (step S252). When the image number N is equal to or less than the predetermined number (step S252; NO), the process proceeds to step S262.

  When the number N of images is larger than the predetermined number (step S252; YES), date / time information of the selected image data is acquired (step S253). When the date / time information of the selected target image data is acquired, each image data is sorted in the date / time information order (generation order) based on the acquired date / time information (step S254). Then, 1 is set as an initial value in the variable I (step S255). After execution of step S255, the difference between the date information of the I-th and (I + 1) -th image data is calculated (step S256), and a threshold value corresponding to the date-time information of the I-th and (I + 1) -th image data is set. (Step S257).

  Then, it is determined whether or not the difference between the date information of the I-th and (I + 1) -th image data calculated in step S256 is larger than the threshold acquired in step S257 (step S258). When the difference between the date and time information is larger than the threshold (step S258; YES), a group delimiter position is set between the I-th and (I + 1) -th image data (step S259). And it is discriminate | determined whether the variable I is smaller than (N-1) (step S260). When the variable I is smaller than (N−1) (step S260; YES), the variable I is incremented by +1 (step S261), and the process proceeds to steps S256 and S257. If the difference in date and time information is greater than the threshold (step S258; YES), the process proceeds to step S260.

  When the variable I is equal to or greater than (N-1) (step S120; NO), the selected image data is displayed on the display unit 12 for each group (image group) as a grouping result (step S262), and the second grouping is performed. The image display process ends. With respect to the second grouping image display process, each step of FIG. 9 except that the image data that is the grouping process target in the first grouping image display process of FIG. 5 is the image data selected in step S23 or S24. Since this process is substantially the same as steps S111 to S122 in FIG. 5, a detailed description thereof is omitted here.

  Returning to FIG. When the calendar is displayed on the display unit 12, the user performs an ordering operation based on the calendar displayed on the display unit 12 (step S26). When the ordering operation is completed, a confirmation message for confirming the execution of printing is displayed on the display unit 12 (step S27). It is determined whether or not the execution of printing is confirmed by the operation of the operation input unit 13 by the user (step S28). When execution of printing is confirmed (step S28; YES), an order confirmation slip is issued from the order confirmation slip issuing unit 16 (step S29), and provisional order information (order information) is ordered via the communication unit 17. The data is transmitted to the management apparatus 2 (step S29), and the second print acceptance process ends. If execution of printing is not confirmed (step S28; NO), the process proceeds to step S26.

  Next, the second print management process executed in the order management apparatus 2 will be described. It is assumed that the second print management process is executed by the control unit 21 based on the second print management program stored in the storage unit 24. Further, the second print management process is executed with the fact that the solid identification basic information is received via the communication unit 26 in the order management apparatus 2 corresponding to step S30 of the second print acceptance process.

  The order management device 2 searches the DB unit 3 for solid identification information that matches the received solid identification basic information, thereby reading service history information corresponding to the solid identification basic information (step T11). Information indicating the presence or absence (whether there is corresponding service history information) is transmitted to the receiving terminal 1 via the communication unit 26 (step T12). If there is service history information, in step T12, service history information is also transmitted in addition to the presence / absence information of the matching information.

  Then, in response to step S30 of the second print acceptance process, the order management apparatus 2 that has received the temporary order information via the communication unit 26 outputs a print to the image printer unit 4 based on the received information. An instruction signal is output (step T13), and the image printer 4 performs print output.

  Next, based on the presence / absence of matching information in step T11, it is determined whether or not the customer of the current print order is new (step T14). In step T11, when there is no solid identification information that matches the solid identification basic information received from the reception terminal 1 in the DB unit 3, it is determined as a new customer in step T14 (step T14; YES). New identification information is issued for the print order, and the identification information is registered in the DB unit 3 (step T15).

  After the execution of step T15, or when it is determined in step T14 that the customer of the current print order is not new (step T14; NO), or when the individual identification information is newly registered in step T14, Based on the print order, the service history information in the DB unit 3 is updated (step T16), and the second print order acceptance process ends.

<Process when representative image data is selected>
Next, a process executed under the control of the control unit 11 when selecting representative image data of a group (step S122 in FIG. 5 and step S262 in FIG. 9) will be described with reference to the flowchart in FIG. .

  First, for each photographic image data, the photographic image data is divided into predetermined image areas, and an occupancy ratio calculation process is performed to calculate an occupancy ratio indicating the ratio of each divided area to the entire photographic image data (step S40). The occupation rate calculation process in step S40 will be described in detail later with reference to FIGS.

  Next, based on the occupation ratio calculated in step S40 and a coefficient set in advance according to the shooting conditions, an index (index 1 to 5) for specifying the light distribution (light source state) is calculated, Based on the calculated index, the light distribution at the time of photographing (any one of forward light, backlight, strobe) is calculated (step S41). Here, the index for specifying the light distribution is a numerical value that quantitatively represents the light distribution at the time of photographing. The index calculation process and the light distribution calculation process in step S41 will be described in detail later.

  Next, processing conditions for face extraction processing for extracting a person's face from the captured image data are determined according to the light distribution calculated in step S41 (step S42), and face extraction processing is performed according to the determined processing conditions. Performed (step S43). The determination of the processing conditions in step S42 will be described in detail later. The face extraction process in step S43 will be described in detail later with reference to FIGS.

<Occupancy rate calculation process>
Next, the occupation rate calculation process shown in step S40 of FIG. 10 will be described in detail with reference to the flowchart of FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step S50). FIG. 12 shows an example of a conversion program (HSV conversion program) that obtains a hue value, a saturation value, and a lightness value by converting from RGB to the HSV color system using program code (c language). In the HSV conversion program shown in FIG. 12, the values of digital image data as input image data are defined as InR, InG, and InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, and the saturation The value is OutS, the brightness value is OutV, and the unit is defined as 0 to 255.

  Next, the photographed image data is divided into regions composed of combinations of predetermined brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S51). Hereinafter, the area division of the captured image data will be described in detail.

The lightness value (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200-224 (v6) , 225 to 255 (v7). Hue (H) is a skin hue hue area (H1 and H2) with a hue value of 0 to 39, 330 to 359, a green hue area (H3) with a hue value of 40 to 160, and a blue hue area with a hue value of 161 to 250 It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is known that the contribution to the light distribution calculation is small. The flesh-color hue area is further divided into a flesh-color area (H1) and other areas (H2). Hereinafter, among the flesh-colored hue regions (H = 0 to 39, 330 to 359), the hue '(H) that satisfies the following equation (1) is defined as the flesh-colored region (H1), and the region that does not satisfy the equation (1) is ( H2).
10 <Saturation (S) <175,
Hue '(H) = Hue (H) + 60 (when 0 ≤ Hue (H) <300),
Hue '(H) = Hue (H)-300 (when 300 ≤ Hue (H) <360),
Luminance Y = InR × 0.30 + InG × 0.59 + InB × 0.11
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (1)
Therefore, the number of divided areas of the captured image data is 4 × 7 = 28. In addition, it is also possible to use the brightness (V) in the formula (1).

When the two-dimensional histogram is created, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S52), and the main occupancy ratio calculation is performed. Processing ends. Assuming that the first occupancy ratio calculated in the divided area composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy ratio in each divided area is expressed as shown in Table 1.

Next, a method for calculating the index 1 and the index 2 will be described.
Table 2 shows the first coefficient necessary for calculating the index 1 that quantitatively indicates the accuracy of strobe shooting obtained by discriminant analysis, that is, the lightness state of the face area at the time of strobe shooting. Show. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupation ratio Rij of each divided area shown in Table 1 is multiplied.

  FIG. 13 shows a lightness (v) -hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the region (r1) distributed in the skin color hue region of high brightness in FIG. 13, and blue other than that is blue. A negative (−) coefficient is used for the first occupancy calculated from the hue region (r2). FIG. 15 shows a curve (coefficient curve) in which the first coefficient in the skin color area (H1) and the first coefficient in the other areas (green hue area (H3)) continuously change over the entire brightness. It is shown. According to Table 2 and FIG. 15, in the region of high brightness (V = 170 to 224), the sign of the first coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of the two are different.

従って、H1〜H4領域の和は、下記の式(2-1)〜式(2-4)のように表される。
H1領域の和＝R11×(-44.0)＋R21×(-16.0)＋（中略）...＋R71×(-11.3) …(2-1)
H2領域の和＝R12×0.0＋R22×8.6＋（中略）... ＋R72×(-11.1) …(2-2)
H3領域の和＝R13×0.0＋R23×(-6.3)＋（中略）...＋R73×(-10.0) …(2-3)
H4領域の和＝R14×0.0＋R24×(-1.8)＋（中略）...＋R74×(-14.6) …(2-4) When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Expression (2).

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (2-1) to (2-4).
H1 area sum = R11 x (-44.0) + R21 x (-16.0) + (omitted) ... + R71 x (-11.3) ... (2-1)
Sum of H2 area = R12 x 0.0 + R22 x 8.6 + (omitted) ... + R72 x (-11.1) ... (2-2)
H3 area sum = R13 x 0.0 + R23 x (-6.3) + (omitted) ... + R73 x (-10.0) ... (2-3)
Sum of H4 area = R14 x 0.0 + R24 x (-1.8) + (omitted) ... + R74 x (-14.6) ... (2-4)

The index 1 is defined as Expression (3) using the sum of the H1 to H4 regions shown in Expressions (2-1) to (2-4).
Index 1 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 4.424 (3)

Table 3 shows, for each divided region, the second coefficient necessary for calculating the index 2 that quantitatively indicates the accuracy of backlight photographing obtained by discriminant analysis, that is, the brightness state of the face region at the time of backlight photographing. Show. The coefficient of each divided area shown in Table 3 is a weighting coefficient that is multiplied by the first occupation ratio Rij of each divided area shown in Table 1.

  FIG. 14 shows a lightness (v) -hue (H) plane. According to Table 3, a negative (-) coefficient is used for the occupation ratio calculated from the region (r4) distributed in the intermediate lightness of the flesh color hue region in FIG. 14, and the low lightness (shadow) region ( A positive (+) coefficient is used for the occupation ratio calculated from r3). FIG. 16 shows the second coefficient in the skin color area (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 3 and FIG. 16, the sign of the second coefficient of the intermediate lightness region of the flesh color hue region having the lightness value of 85 to 169 (v4) is negative (−), and the lightness value is 26 to 84 (v2, It can be seen that the sign of the second coefficient in the low brightness (shadow) region of v3) is positive (+), and the sign of the coefficient in both regions is different.

従って、H1〜H4領域の和は、下記の式(4-1)〜式(4-4)のように表される。
H1領域の和＝R11×(-27.0)＋R21×4.5＋（中略）...＋R71×(-24.0) …(4-1)
H2領域の和＝R12×0.0＋R22×4.7＋（中略）... ＋R72×(-8.5) …(4-2)
H3領域の和＝R13×0.0＋R23×0.0＋（中略）...＋R73×0.0 …(4-3)
H4領域の和＝R14×0.0＋R24×(-5.1)＋（中略）...＋R74×7.2 …(4-4) When the second coefficient in the lightness area vi and the hue area Hj is Dij, the sum of the Hk areas for calculating the index 2 is defined as in Expression (4).

Therefore, the sum of the H1 to H4 regions is expressed by the following formulas (4-1) to (4-4).
H1 area sum = R11 x (-27.0) + R21 x 4.5 + (omitted) ... + R71 x (-24.0) ... (4-1)
Sum of H2 area = R12 x 0.0 + R22 x 4.7 + (omitted) ... + R72 x (-8.5) ... (4-2)
Sum of H3 area = R13 x 0.0 + R23 x 0.0 + (omitted) ... + R73 x 0.0 (4-3)
H4 area sum = R14 x 0.0 + R24 x (-5.1) + (omitted) ... + R74 x 7.2 ... (4-4)

The index 2 is defined as in Expression (5) using the sum of the H1 to H4 regions shown in Expressions (4-1) to (4-4).
Index 2 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 1.554 (5)
Since the index 1 and the index 2 are calculated based on the brightness of the captured image data and the distribution amount of the hue, they are effective in calculating the light distribution when the captured image data is a color image.

  Next, the occupation rate calculation process (step S40 in FIG. 10) for calculating the index 3 will be described in detail with reference to the flowchart in FIG.

  First, the RGB values of the captured image data are converted into the HSV color system (step S60). Next, the photographed image data is divided into regions each composed of a combination of the distance from the outer edge of the photographed image screen and brightness, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S61). Hereinafter, the area division of the captured image data will be described in detail.

  18A to 18D show four regions n1 to n4 that are divided according to the distance from the outer edge of the screen of the captured image data. A region n1 shown in FIG. 18A is an outer frame, a region n2 shown in FIG. 18B is a region inside the outer frame, and a region n3 shown in FIG. A region n4 shown in FIG. 18D is an inner region, and is a central region of the captured image screen. Further, the lightness is divided into seven regions v1 to v7 as described above. Therefore, when the captured image data is divided into regions composed of combinations of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

When the two-dimensional histogram is created, a second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S62), and the main occupancy ratio calculation is performed. The process ends. If the second occupancy calculated in the divided area composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each divided area is expressed as shown in Table 4.

図１９は、画面領域ｎ１〜ｎ４における第３の係数を、明度全体に渡って連続的に変化する曲線（係数曲線）として示したものである。 Next, a method for calculating the index 3 will be described.
Table 5 shows the third coefficient necessary for calculating the index 3 for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupancy Qij of each divided area shown in Table 4 is multiplied, and is obtained by discriminant analysis.

FIG. 19 shows the third coefficient in the screen areas n1 to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

従って、n1〜n4領域の和は、下記の式(6-1)〜式(6-4)のように表される。
n1領域の和＝Q11×40.1＋Q21×37.0＋（中略）...＋Q71×22.0 …(6-1)
n2領域の和＝Q12×(-14.8)＋Q22×(-10.5)＋（中略）...＋Q72×0.0 …(6-2)
n3領域の和＝Q13×24.6＋Q23×12.1＋（中略）...＋Q73×10.1 …(6-3)
n4領域の和＝Q14×1.5＋Q24×(-32.9)＋（中略）...＋Q74×(-52.2) …(6-4) If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in Expression (6).

Accordingly, the sum of the n1 to n4 regions is expressed as the following formulas (6-1) to (6-4).
n1 area sum = Q11 x 40.1 + Q21 x 37.0 + (omitted) ... + Q71 x 22.0 (6-1)
Sum of n2 areas = Q12 x (-14.8) + Q22 x (-10.5) + (omitted) ... + Q72 x 0.0 (6-2)
n3 area sum = Q13 x 24.6 + Q23 x 12.1 + (omitted) ... + Q73 x 10.1… (6-3)
n4 area sum = Q14 x 1.5 + Q24 x (-32.9) + (omitted) ... + Q74 x (-52.2) ... (6-4)

The index 3 is defined as in Expression (7) using the sum of the N1 to H4 regions shown in Expressions (6-1) to (6-4).
Index 3 = sum of n1 regions + sum of n2 regions + sum of n3 regions + sum of n4 regions−12.6201 (7)
Since the index 3 is calculated based on the compositional feature (distance from the outer edge of the screen of the captured image data) according to the distribution position of the brightness of the captured image data, the light distribution of not only the color image but also the monochrome image is calculated. It is also effective to do.

The index 4 is defined as Expression (8) using the indices 1 and 3, and the index 5 is defined as Expression (9) using the indices 1 to 3.
Index 4 = 0.565 x index 1 + 0.565 x index 3 + 0.457 (8)
Index 5 = (− 0.121) × Index 1 + 0.91 × Index 2 + 0.113 × Index 3-0.072 (9)
Here, the weighting coefficient by which each index is multiplied in Expression (8) and Expression (9) is set in advance according to the shooting conditions.

このように指標４、５の値により、配光性を定量的に算出することができる。 <Calculation method of light distribution>
Next, the light distribution calculation method (step S41 in FIG. 10) will be described.
In FIG. 20, 60 images were taken under each condition of forward light, backlight, and strobe, and indexes 4 and 5 were calculated for a total of 180 digital image data, and the values of indexes 4 and 5 under each condition were plotted. Is. According to FIG. 20, when the value of index 4 is greater than 0.5, there are many strobe scenes, and when the value of index 4 is 0.5 or less and the value of index 5 is greater than −0.5, the backlight scene is I understand that there are many. Table 6 shows the calculation contents of the light distribution by the values of the indexes 4 and 5.

Thus, the light distribution can be calculated quantitatively based on the values of the indexes 4 and 5.

<Determination of processing conditions for face extraction processing>
Next, the method for determining the processing conditions for the face extraction process shown in step S42 in FIG. 10 will be described in detail.

  The determination of the processing conditions for the face extraction process executed according to the light distribution at the time of shooting is performed in order to improve the extraction accuracy of the face extraction process. As a processing method of the face extraction processing, a face candidate area is extracted from captured image data, and it is determined whether or not the extracted face candidate area is a person's face. A human face is included in the face candidate area. For example, there is a method for determining whether or not it is. When such a method is used, the extraction accuracy of the face extraction process includes the extraction accuracy of the face candidate region and whether the face candidate region is a human face (or whether a face is included in the face candidate region). Both of the determination accuracy contribute to determining whether or not the face extraction processing conditions are determined. When the processing conditions for the face extraction process are determined, the processing conditions for the process of determining the face of the person from the face candidate area Either one or both are preferably determined according to the light distribution calculation result.

  Hereinafter, the process of extracting a face candidate area from captured image data and the process of determining whether or not the extracted face candidate area is a person's face are referred to as a “face extraction process”. Of these, the part of the process for determining whether or not the face candidate area is a person's face is referred to as “face determination process”.

First, a method for determining the extraction condition of the face candidate area will be described.
As a method for extracting a face candidate area, a method can be used in which an initial point corresponding to a skin color is searched from captured image data, and a skin color area is extracted as a face candidate area using a simple area expansion method. The simple area expansion method is an image processing method for extracting a specific image area by expanding adjacent areas with pixels adjacent to each other whose data difference between pixels is equal to or less than a threshold value belonging to the same image area. . In other words, the simple area expansion method starts from an initial pixel that matches a specified condition, and a data difference between adjacent pixels (either 4-connected or 8-connected) is equal to or less than a threshold value. In this case, it is assumed that the adjacent pixel and the initial pixel belong to the same image area, and the same determination is performed for a pixel adjacent to the pixel that belongs to the same pixel. In this manner, starting from the initial pixel, a specific image region is extracted by gradually expanding the same image region. In the present embodiment, it is further preferable to use a process of extracting a face component region by extracting an edge component in an image and ending simple region expansion when the image edge is reached.

  When a skin color area is extracted as the face candidate area, even if the subject is the same person, the color of the skin color area varies significantly depending on the light distribution at the time of shooting. For example, based on the skin color of an image taken in front light, the skin tone becomes whiter (especially the closer it is) in strobe shooting, and if it is shooting in backlight, Skin tone becomes darker. In consideration of such a situation, it is preferable to switch and / or adjust (stepwise or continuously) the search condition for the initial pixel in accordance with the light distribution calculation result.

  For example, for a predetermined basic search condition for searching for an initial pixel, if the light distribution is “forward light”, the basic search condition is used. If the light distribution is “strobe”, the basic search condition is used. If the search condition with the brightness corrected (brighter) is used and the light distribution is “backlight”, the search condition with the brightness reduced (darker) from the basic search condition may be used. As described above, when extracting a face candidate region using the simple region expansion method, the search condition for the initial pixel may be switched according to the light distribution at the time of photographing. Moreover, since the edge intensity | strength which complete | finishes simple area expansion differs with light distribution, it is preferable to adjust the edge extraction conditions for ending simple area expansion according to light distribution.

  When the calculation result of the light distribution is obtained as an index (index 4, 5) indicating the light distribution stepwise or continuously, such as strobe degree, backlight intensity, etc., depending on the value of the obtained index The search condition for the initial pixel may be determined by adjusting the degree of increasing the brightness (in the case of a strobe) and the degree of decreasing in the brightness (in the case of backlight). Therefore, the simple region expansion method using the initial pixel search condition can be determined as the extraction condition for the face candidate region.

  In the above description, the example in which the search condition for the initial pixel is corrected using the fact that the brightness difference is generated in the skin color area according to the light distribution is shown. However, not only the brightness but also saturation is used. Thus, the search condition correction requirement may be increased. For example, with flash and flash photography, the saturation is lower than with direct light. Further, not only the initial pixel search condition but also the area expansion condition may be corrected. Furthermore, it is preferable to correct the initial pixel search condition and the region expansion condition for various processing conditions for extracting the face candidate region, reflecting the difference in the characteristics of the skin color region obtained for each light distribution.

  Next, a method for determining a face determination processing condition for determining a face from a face candidate area will be described. In the following, a method for determining a face determination processing condition when a feature amount is extracted from a face candidate region and face determination processing is performed using a neural network based on the extracted feature amount will be described.

  First, prior to a specific description of a method for determining face determination processing conditions, a neural network used in face determination processing will be briefly described. As shown in FIG. 21, this neural network is a hierarchical neural network (hereinafter simply referred to as a neural network) having an input layer, an intermediate layer, and an output layer. The neural network preferably uses a digital neural network chip, but can also be realized by using a general-purpose DSP (Digital Signal Processor) and a dedicated emulation program, or using a normal CPU and an emulation program. It doesn't matter.

ここで、ｆ（ｘ）は一般に式（１１）に示すような非線形のシグモイド関数が用いられるが、必ずしもこれに限定されるわけではない。

Each layer of the neural network is composed of structural units called neurons. Except for the input layer, as shown in FIG. 22, an arbitrary neuron receives a large number of inputs called synapses, and multiplies each input value x _i by a predetermined weight w _i called connection strength to sum the sum. It functions as a functional element that obtains and evaluates a result obtained by evaluating it with a predetermined output function f as an output y. The output function f is expressed as in Expression (10).

Here, f (x) is generally a non-linear sigmoid function as shown in Equation (11), but is not necessarily limited thereto.

Here, the input weight w _i of each neuron is important information for determining the relationship between the input and output of the neural network, in other words, the operation of the neural network, and is determined by performing a learning operation on the neural network. In FIG. 21, only one intermediate layer is shown, but this can be configured in one or more arbitrary layers. Hierarchical networks are known to be able to achieve any function that can be synthesized if there is only one intermediate layer, but increasing the number of intermediate layers is advantageous in terms of learning efficiency and the number of neurons. It has been known.

  As a feature of the neural network, generalization ability as a system is raised. This is not necessarily teacher data (a pair of an input pattern and a desired output pattern corresponding to the input pattern is usually prepared by learning, and a plurality of them are usually prepared. The operation of determining a desired output pattern for a specific input pattern is generally performed by an expert. This is the ability to give a correct output even for unlearned input patterns that are not given as a result of subjective judgment.

  Various learning methods using teacher data for neural networks are known, for example, by performing a sufficient number of learning operations using the widely used error back propagation learning method (Back Propagation learning method), An output approximate to the teacher data can be obtained.

  Next, a method for determining the face determination processing condition will be specifically described. If the same face determination processing conditions are used even though the light distribution is different, a difference in face determination accuracy occurs. This is because the feature amount contributing to the face determination process is different for each light distribution and the extraction condition for optimally extracting the feature amount is different. Therefore, it is preferable to determine the face determination processing condition according to the light distribution.

  Specifically, it is preferable to determine the learning result of the neural network used for the design of the neural network and the face determination process according to the light distribution. Here, the design of the neural network indicates, for example, designing the type and number of input parameters of the neural network, the number of intermediate layers, and the like. The input parameter includes a feature amount extracted from the face candidate region. The learning result of the neural network represents a weight (connection strength) assigned to each neuron constituting the neural network, a slope value of the sigmoid function, and the like.

  Regardless of the nature of the feature quantity used as the input parameter of the neural network and the light distribution, if the feature quantity is extracted from the face candidate region under the same extraction condition, the extraction accuracy differs. Further, depending on the nature of the feature quantity, a difference occurs in the separability between a person's face and a part other than the face according to the light distribution. For this reason, it is preferable to change the feature amount extracted from the face candidate region and the feature amount extraction condition in accordance with the light distribution.

  For example, using features that make up the face such as eyes and mouth as feature values, and extraction conditions for feature values, differences in skin and eyes or mouth color (skin color and almost black, skin color and redness) and contours of parts In the case of using the extraction condition using the edge of the backlight, there is a tendency for the backlight to be backlit and the backlight to be less accurate in extracting feature quantities. That is, in the backlight, the entire face of the person is photographed darker than the normal light, so that the difference between the color values of the skin and the part is small, and the edge of the contour is difficult to extract compared to the normal light.

  In this way, the feature quantity extraction conditions may be switched or the type and number of input parameters of the neural network may be changed according to the light distribution. Further, the feature quantity extraction condition may be adjusted stepwise or continuously in accordance with the calculated index value (strobe degree, backlight intensity, etc.). It is also preferable to change the number of intermediate layers and the number of neurons in the neural network according to the number of input parameters.

  Further, when determining the face determination processing conditions, it is preferable to switch the neural network previously learned in each of the forward light, strobe, and backlight images according to the light distribution. In addition, if the calculation result of light distribution is a system in which the light distribution is obtained as an indicator that indicates stepwise or continuously, such as strobe degree, backlight intensity, etc., a neural network can be used for a strong strobe. If it is prepared step by step according to the degree of light distribution, such as for strobes at high degrees, for strong backlights, and for weak backlights, the determination accuracy of the face determination process can be improved. In addition, it is preferable to operate the system by appropriately updating it using a version upgrade method of the self-learning neural network.

<Face extraction processing>
Next, the face extraction process shown in step S43 of FIG. 10 will be described in detail with reference to the flowchart of FIG.

  First, in accordance with the extraction condition of the face candidate area determined in step S42 of FIG. 10, a face candidate area is extracted from photographed image data having a light intensity equal to or greater than a predetermined value (step S70), and the face candidate area is included in the photographed image data. It is determined whether or not there is (step S71).

  If it is determined in step S71 that there is no face candidate area in the captured image data (step S71; NO), the face extraction process ends. If it is determined in step S71 that there is a face candidate area in the captured image data (step S71; YES), face determination is performed to determine whether the face candidate area extracted in step S70 is a human face. Processing is performed (step S72), and the face extraction processing ends. The face determination process in step S72 will be described in detail later with reference to FIG.

  In the above description, the simple area expansion method is used as an example of the face candidate area extraction process. However, the face candidate area extraction method is not limited to this. For example, it is possible to use a method of extracting an edge from photographed image data and extracting a specific shape pattern such as an ellipse using a technique such as Hough transform or template matching.

  Furthermore, as disclosed in JP-A-4-346332, an original image (photographed image data) is divided into a large number of pixels, and a histogram of hue values and saturation values is created from the RGB values of each pixel, A method is used in which a histogram is divided from the shape, an image is divided into areas composed of pixels corresponding to the divided parts, and an area corresponding to a human face is estimated from the divided areas. be able to.

  Further, as disclosed in JP-A-6-309433, JP-A-6-67320, JP-A-5-158164, and JP-A-5-165120, a hue value, a saturation value, and a luminance value are disclosed. A method of determining a skin color area based on a histogram such as the above and using the skin color area as a face candidate area can be used. Further, a line process as disclosed in JP-A-8-221567, JP-A-2000-20694, JP-A-2000-32272, JP-A-2000-201158, and JP-A-2000-20769. The face candidate region can be extracted using the method.

  In addition, an edge and a low frequency image are generated from a color digital image, and predetermined parameters (for example, saturation value, hue value, and lightness value) are initially matched with predetermined initial conditions (values in each predetermined range). Extract a pixel, perform simple area expansion from the extracted initial pixel for the low-frequency image, and forcibly end simple area expansion when the image edge is reached to extract a face candidate area May be.

  Next, the face determination process shown in step S72 of FIG. 23 will be described with reference to the flowchart of FIG.

  First, a feature value is extracted from the face candidate area according to the feature value determined in step S42 of FIG. 10 and its extraction condition (step S80). Next, the neural network determined according to the light distribution (for example, when the light distribution is backlit) using the feature amount extracted in step S80 as the input signal (input parameter) and the parameter representing the facial appearance as the output signal. The neural network learned for backlighting is used to determine whether the face candidate area is a human face (step S81), and the face determination process ends.

  As described above, according to the present embodiment, the reception terminal 1 calculates the difference value of the generation time of the grouping target image data based on the date / time information of the grouping target image data among the plurality of image data recorded on the medium M. In addition, a threshold value of the generation time interval of the grouping target image data is set according to the date and time information, and when the difference value of the grouping target image data is larger than the set threshold value, the grouping result is set. Is displayed on the display unit 12. Therefore, the image data can be appropriately grouped according to the generation (photographing) time (date and time information), and the grouping target image data can be displayed for each group.

  Further, the user can easily select and input image data for print ordering by referring to grouped image data after grouping displayed on the display unit 12. Further, by increasing the generation time interval threshold as the date and time information becomes older, it is possible to reduce the number of groups in which image data is unlikely to be ordered and to reduce the work time and work load of the user for selecting image data. Can be reduced.

  Further, in the display of the grouping result on the display unit 12, if the representative image data is displayed for each group, the user can easily grasp the representative image data for each group. Furthermore, when thumbnail image data of grouping target image data is generated and predetermined representative image data is selected and input, thumbnail image data of all image data belonging to the group of the representative image data is displayed. By referring to the image data, it is possible to more easily select and input image data for print order.

  The description in the above embodiment is an example of a suitable image processing apparatus, image processing method, and image processing program according to the present invention, and the present invention is not limited to this.

  For example, in the above-described embodiment, the case where the processing related to the print order is generally performed on the reception terminal 1 side is shown, but when the processing capability of the reception terminal 1 is low, an image to be processed is transmitted to the external device, It is possible to distribute the processing load by performing processing with a high processing load on the external device side (for example, light distribution calculation processing and face extraction processing in representative image selection). On the other hand, depending on the processing capability of the reception terminal 1, the function of the order management device 2 may be added to the reception terminal 1 side.

  In the above embodiment, the grouping result is displayed on the display unit 12 and output. However, the present invention is not limited to this. For example, the grouping target image data may be recorded and output on a recording medium such as a CD-R (Compact Disc Recordable) for each group as a grouping result. Specifically, for example, grouping target image data and information on the group separation position are recorded on a recording medium. Further, the grouping target image data may be recorded on a recording medium for each group, and information such as a label, an index, and representative image data of each group may be recorded together.

  Further, the detailed configuration and detailed operation of each part constituting the image processing system 100 in the above embodiment can be appropriately changed without departing from the spirit of the present invention.

1 is a diagram showing a configuration of an image processing system 100 according to an embodiment of the present invention. It is a block diagram which shows the principal part structure of the reception terminal 1 which concerns on this Embodiment. It is a block diagram which shows the principal part structure of the order management apparatus 2 which concerns on this Embodiment. 5 is a flowchart showing a first print acceptance process executed in the reception terminal 1 and a first print management process executed in the order management apparatus 2. It is a flowchart which shows a 1st grouping image display process. It is a figure which shows an example of a grouping result display screen. It is a figure which shows an example of the calendar display screen as a grouping result. 10 is a flowchart illustrating a second print acceptance process executed in the reception terminal 1 in a modified example and a second print management process executed in the order management apparatus 2; It is a flowchart which shows a 2nd grouping image display process. It is a flowchart which shows the process performed in the reception terminal 1 at the time of selection of the representative image of an image group. It is a flowchart which shows the occupation rate calculation process which calculates the 1st occupation rate for every area | region of lightness and hue. It is a figure which shows an example of the program converted from RGB to HSV color system. It is a figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r1 and the area | region r2 on a VH plane. It is a figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r3 and the area | region r4 on a VH plane. It is a figure which shows the curve showing the 1st coefficient by which the 1st occupation rate for calculating the parameter | index 1 is multiplied. It is a figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. It is a flowchart which shows the occupation rate calculation process which calculates a 2nd occupation rate based on the composition of picked-up image data. It is a figure which shows the area | region n1-n4 determined according to the distance from the outer edge of the screen of picked-up image data. It is a figure which shows the curve showing the 3rd coefficient by which the 2nd occupation rate for calculating the parameter | index 3 is multiplied for every area | region (n1-n4). It is the plot figure of the parameter | index 4 and the parameter | index 5 calculated according to light distribution (forward light, strobe, back light). It is a figure which shows typically the structure of a hierarchical neural network. It is a figure which shows the structure of the neuron which comprises a neural network. It is a flowchart which shows a face extraction process. It is a flowchart which shows a face determination process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image processing system 1 Reception terminal 11 Control part 12 Display part 13 Operation input part 14 Media reading part 15 Storage part 16 Order confirmation slip issuing part 17 Communication part 2 Order management apparatus 21 Control part 22 Display part 23 Operation input part 24 Storage part 25 I / F unit 26 Communication unit 27 DB reference unit 3 DB unit 4 Image printer unit 5 Image media writing unit

Claims (21)

The generation time of the grouping target image data based on the date information of the grouping target image data among the image data recorded in the recording unit in which the image data having the generation time information indicating the generation time is recorded as additional information. Difference value calculating means for calculating the difference value of
Threshold setting means for setting a generation time interval threshold for group separation according to the date and time information of the image data to be grouped;
The grouping target image data with the calculated difference value corresponding to the date / time information of the grouping target image data being larger than the set grouping threshold corresponding to the date / time information as a grouping position. Group setting means for grouping and setting
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the group separation threshold is larger as the date information is older.   The image processing apparatus according to claim 1, further comprising an output unit that outputs the grouped image data for each group.   The image processing apparatus according to claim 3, wherein the output unit displays the grouped image data for each group. Determining means for determining representative image data of the group from image data belonging to the group divided into groups;
The image processing apparatus according to claim 4, wherein the output unit displays the representative image data. First input means for receiving selection input of the representative image data;
Generating means for generating thumbnail image data of image data belonging to a group of representative image data selected and input by the first input means,
The image processing apparatus according to claim 5, wherein the output unit displays the generated thumbnail image data. Second input means for receiving selection input of image data to be printed out of the grouped image data;
Transmitting means for transmitting the image data selected and input by the second input means to the image forming means as print image data;
The image processing apparatus according to claim 1, further comprising: The generation time of the grouping target image data based on the date information of the grouping target image data among the image data recorded in the recording unit in which the image data having the generation time information indicating the generation time is recorded as additional information. A difference value calculating step for calculating a difference value of
A threshold setting step for setting a generation time interval threshold for group separation according to date and time information of the image data to be grouped;
The grouping target image data with the calculated difference value corresponding to the date / time information of the grouping target image data being larger than the set grouping threshold corresponding to the date / time information as a grouping position. A group setting process for grouping and setting
An image processing method comprising:   The image processing method according to claim 8, wherein the group separation threshold is larger as the date information is older.   The image processing method according to claim 8, further comprising an output step of outputting the grouped image data for each group.   The image processing method according to claim 10, wherein the output step is a step of displaying the grouped image data for each group. A determination step of determining representative image data of the group from the image data belonging to the group divided into groups,
The image processing method according to claim 11, wherein the representative image data is displayed in the output step. A first input step for receiving selection input of the representative image data;
Generating thumbnail image data of image data belonging to the group of representative image data selected and input in the first input step, and
13. The image processing method according to claim 12, wherein, in the output step, the generated thumbnail image data is displayed. A second input step for receiving selection input of image data to be printed out of the grouped image data;
14. The image according to claim 8, further comprising: a transmission step of transmitting the image data selected and input in the second input step to the image forming unit as print image data. Processing method. Computer
The generation time of the grouping target image data based on the date information of the grouping target image data among the image data recorded in the recording unit in which the image data having the generation time information indicating the generation time is recorded as additional information. Difference value calculation means for calculating the difference value of
Threshold setting means for setting a generation time interval threshold for group separation according to the date and time information of the image data to be grouped;
The grouping target image data with the calculated difference value corresponding to the date / time information of the grouping target image data being larger than the set grouping threshold corresponding to the date / time information as a grouping position. Group setting means for grouping and setting
Image processing program to function as   The image processing program according to claim 15, wherein the threshold for group separation is larger as the date information is older. The computer,
17. The image processing program according to claim 15, wherein the image processing program functions as an output unit that outputs the grouped image data for each group.   The image processing program according to claim 17, wherein the output unit displays the grouped image data for each group. The computer,
From the image data belonging to the group divided into groups, function as a determining means for determining representative image data of the group,
The image processing program according to claim 18, wherein the output unit displays the representative image data. The computer,
First input means for receiving selection input of the representative image data;
Functioning as generation means for generating thumbnail image data of image data belonging to the group of representative image data selected and input by the first input means,
The image processing program according to claim 19, wherein the output unit displays the generated thumbnail image data. The computer,
Second input means for receiving selection input of image data to be printed out of the grouped image data;
21. The image according to claim 15, wherein the image data selected and input by the second input unit functions as a transmission unit that transmits the image data to the image forming unit as print image data. Processing program.

Images