JP6307964B2 - Management server, photo sticker creating apparatus, image management method, control program, and recording medium - Google Patents

Description

  The present invention relates to a management server connected to a photo sticker creating apparatus via a network.

  In recent years, as a service for providing a user with an image generated by a photo sticker creation apparatus, an image is displayed by displaying the image on a mobile terminal owned by the user together with a service for providing a user with a print sticker on which an image is printed. The services that provide users with the Internet are expanding.

  In Patent Document 1, a photo sticker creation apparatus allows a user to select a relation attribute of users A and B at the time of shooting, and a server obtains the photo sticker creation apparatus in a predetermined display form corresponding to the relation attribute. Describes a technique for generating a site page that displays an image.

JP 2012-39413 A (published February 23, 2012)

  However, in the conventional technology, the server can recognize the relationship attribute between a plurality of persons included in the image, but recognizes who each of the plurality of persons included in the image is. I can't.

  The present invention has been made in order to solve the above-mentioned problems, and its main object is to create a management server that can use feature information about a person in an image, or to create a photo sticker that enables it. To provide an apparatus.

  A management server according to an aspect of the present invention is a management server connected to a photo sticker creation device via a network, and includes a feature amount of a human face included in a photographed image taken by the photo sticker creation device. A feature amount acquisition means for acquiring, a correction image acquisition means for acquiring a correction image generated by correcting a face included in the photographed image in the photograph sticker creating apparatus, the correction image, and the feature amount. And image management means for storing them in a database in association with each other.

  According to the above configuration, the feature amount in the captured image before the face is corrected is associated with the corrected image after the face is corrected and stored in the database. Therefore, the original characteristics of the person included in the corrected image can be saved. Depending on the correction applied, the facial features will vary. Even in such a case, according to one aspect of the present invention, for example, the feature amount before correction can be used to determine whether the persons included in the plurality of corrected images are the same person. Therefore, the management server can provide useful services to the user.

  The management server may include a personal information acquisition unit that acquires personal information about the person, and the image management unit may store the personal information in a database in association with the feature amount.

  According to the above configuration, since the person information is associated with the feature amount, the person information can be provided for the person included in the corrected image.

  The image management means stores the corrected image in the database in association with a user who is the registrant of the corrected image, and the management server has user information management means for managing user information related to the user; A person determination unit that determines whether the feature amount is a feature amount of the face of the user who is the registrant of the corrected image, based on the person information associated with the amount and the user information; The image management unit may be configured to store the determination result of the person determination unit in the database in association with the feature amount.

  Photo sticker creation devices can generally be used together by multiple users. For this reason, a single corrected image often includes a plurality of persons. Even in such a case, according to the above configuration, it is possible to determine whether or not a certain person included in the corrected image is a user who has registered the corrected image in the database.

  The management server may be configured to include image extraction means for extracting the correction image including a specific person who is not the user from the plurality of correction images associated with the user.

  According to the above configuration, a corrected image including a specific person can be extracted. Therefore, the corrected image can be classified according to the person included.

  The image extraction unit compares the feature amount determined not to be the feature amount of the user's face with the feature amount of the face included in the corrected image, so that the specific person is included in the corrected image. The structure which determines whether it is contained may be sufficient.

  Depending on the correction applied, the facial features will vary. According to the above configuration, it is possible to accurately determine whether or not the specific person is included in the corrected image by comparing the characteristic amount of the specific person with the characteristic amount associated with another corrected image. Can do.

  The management server specifies the position of the face corresponding to the feature amount using the feature amount associated with the correction image in the correction image, and the management server specifies the position of the face specified in the correction image. It may be configured to include image processing means for performing predetermined image processing.

  According to said structure, the position of the face in a correction image can be pinpointed using the feature-value linked | related with the correction image. Therefore, image processing can be performed on the face of the corrected image.

  The image processing means may be configured to perform the image processing for synthesizing the person information associated with the feature amount at a position corresponding to the face.

  According to said structure, person information can be synthesize | combined to the position corresponding to the face of the person as if the stamp was pressed to the correction image.

  The management server may include a first transmission unit that transmits the corrected image and the feature amount to a user terminal.

  According to said structure, the feature-value linked | related with the correction image is transmitted to the user terminal which performs the image process with respect to a correction image, for example. The user terminal can perform image processing on the corrected image using the received corrected image and feature amount.

  The person information acquisition means may be configured to acquire the person information input by the user to the photo sticker creating apparatus from the photo sticker creating apparatus.

  The person information acquisition unit may be configured to acquire the person information from a user terminal of the user.

  The feature amount may be at least one of the position of the face, the size of the face, the shape of the face, the position of the face part, the size of the part, and the shape of the part. .

  A photograph sticker creating apparatus according to an aspect of the present invention includes a photographed image generating unit that generates a photographed image of a person by photographing, a feature amount identifying unit that identifies a feature amount of a person's face included in the photographed image, A correction image generation unit that generates a correction image by correcting a face included in a captured image, and a second transmission unit that associates the correction image with the feature amount and transmits the correlation image to the database server. .

  The photo sticker creating apparatus includes a face image extracting unit that extracts a face image included in the photographed image or the corrected image, a face image presenting unit that presents the face image to the user, and It may be configured to include personal information input means for receiving input of personal information related to a person.

  According to the above configuration, even if the corrected image includes a plurality of persons, the facial feature amount and the person information can be easily combined.

  An image management method according to an aspect of the present invention includes a feature amount specifying step for specifying a feature amount of a person's face included in a photographed image photographed by the photograph sticker creating device, and the photographing by the photograph sticker creating device. A correction image generation step of generating a correction image by correcting a face included in the image, and an image management step of associating the correction image with the feature amount and storing them in a database are provided.

  The management server according to each aspect of the present invention may be realized by a computer. In this case, the management server that realizes the management server by a computer by causing the computer to operate as each unit included in the management server. A control program and a computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  According to one aspect of the present invention, it is possible to use feature information related to a person in an image.

It is a block diagram which shows the principal part structure of the management server which concerns on 1st Embodiment of this invention. It is a figure which shows the structure of the picked-up image management system which concerns on 1st Embodiment of this invention. It is a perspective view which shows the structural example of the external appearance of the photograph sticker production apparatus which concerns on 1st Embodiment of this invention. It is the perspective view which looked at the external appearance of the photograph sticker production apparatus concerning a 1st embodiment of the present invention from other angles. It is a figure explaining the movement of the user in 1st Embodiment of this invention. It is a figure which shows the structural example of the imaging | photography part with which the photograph sticker production apparatus which concerns on 1st Embodiment of this invention is equipped. It is a figure which shows the structural example of the background part with which the photograph sticker production apparatus which concerns on 1st Embodiment of this invention is equipped. It is a figure which shows the structural example of the edit part with which the photograph sticker production apparatus which concerns on 1st Embodiment of this invention is equipped. It is a figure which shows the structural example of the aft customer service part with which the photograph sticker production apparatus which concerns on 1st Embodiment of this invention is equipped. It is a block diagram which shows the example of an internal structure of the photograph sticker production apparatus which concerns on 1st Embodiment of this invention. It is a functional block diagram which shows the detailed structural example of the control part with which the photograph sticker production apparatus which concerns on 1st Embodiment of this invention is equipped. It is a block diagram which shows the principal part structure of the portable terminal which concerns on 1st Embodiment of this invention. It is a screen transition figure which shows an example of the web page displayed on the display part of a portable terminal at the time of the member registration process in 1st Embodiment of this invention. It is a sequence diagram which shows an example of the flow of a series of processes from photography to image preservation | save in 1st Embodiment of this invention. It is a screen transition diagram which shows an example of the web page displayed on the display part of a portable terminal at the time of the image preservation | save process to the management server in 1st Embodiment of this invention. It is a sequence diagram which shows an example of the flow of a series of processes regarding the image preservation | save to the portable terminal in the picked-up image management system which concerns on 1st Embodiment of this invention. It is a screen transition diagram which shows an example of the web page displayed on the display part of a portable terminal at the time of the image preservation | save process to the portable terminal in 1st Embodiment of this invention. It is a figure which shows an example of the data preserve | saved in the image database in 1st Embodiment of this invention. It is a transition diagram which shows an example of the procedure of a person information input in 1st Embodiment of this invention. It is a figure which shows the image before and behind stamp processing in 1st Embodiment of this invention.

Embodiment 1
A first embodiment of the present invention will be described below with reference to FIGS. Descriptions of configurations other than those described in the following specific embodiments may be omitted as necessary, but are the same as those configurations when described in other embodiments. For convenience of explanation, members having the same functions as those shown in each embodiment are given the same reference numerals, and the explanation thereof is omitted as appropriate. Further, the shape of the configuration described in each drawing and the dimensions such as length, size, and width do not reflect the actual shape and dimensions, but are changed as appropriate for clarity and simplification of the drawings. doing.

(Captured image management system 10)
FIG. 2 is a diagram showing a configuration of the captured image management system 10 according to the first embodiment of the present invention. As shown in FIG. 2, the captured image management system 10 includes a photo sticker creating apparatus 1, a mobile terminal 6 (user terminal), and a management server 4.

  An outline of the captured image management system 10 will be described below. The captured image management system 10 provides a user with services for realizing user photography, image editing, image printing as a photo sticker, and image storage on an image providing site. Photo shooting, image editing, and photo sticker printing services are provided by the photo sticker creating apparatus 1 installed in a store such as a game center. The image storage service is provided by the management server 4 that manages the image providing site. The user can store his / her captured image in the image providing site by operating the mobile terminal 6 possessed by the user.

  In order to store images on the image providing site, prior member registration is required. The user registers himself / herself as a free member in the image providing site. Free members do not need to pay membership fees, but image providers are limited in the number of images they can store. In this embodiment, the free member can store only one image in the image providing site for each game.

  The user can change his / her member attribute from a free member to a paid member. Paid members can save an unlimited number of images on an image providing site instead of having to pay a certain membership fee continuously. In the present embodiment, all the captured images generated by a single photo shoot can be stored in the image providing site.

  The flow from photography to image storage in the photographed image management system 10 will be briefly described below. One or more users pay the price to the photo sticker creating apparatus 1 and play a game relating to photography and image editing. The user takes a picture of himself as a subject during the game. The user selects several images to be edited from the images generated by shooting and edits them as desired. Further, the user inputs his / her e-mail address to the photo sticker creating apparatus 1 in order to save the image in the image providing site later. A game relating to photography and image editing can be played together (cooperatively) by a plurality of users. Multiple users can enter their respective email addresses.

  After the game is over, the user gets a photo sticker. On the other hand, the photo sticker creating apparatus 1 associates the generated user's email address with the generated image and transmits the image to the management server 4. When a plurality of e-mail addresses are input, the photo sticker creating apparatus 1 associates the input e-mail addresses of the plurality of users with the generated image and transmits them to the management server 4. The management server 4 transmits a mail for guiding the received image to be stored in the image providing site to the received mail address. The user receives this mail at his / her mobile terminal 6. The e-mail describes a procedure for storing an image on an image providing site. The user performs an operation for saving the image on the portable terminal 6 according to the procedure. As a result, the image is stored in the image storage area dedicated to the user at the image providing site.

  After the image is stored in the image providing site, the user can log in to the image providing site using the portable terminal 6 or other desired device and download the image.

(Configuration of photo sticker creation device)
The configuration and function of each device constituting the captured image management system 10 will be described in detail below. First, details of the photo sticker creating apparatus 1 will be described with reference to FIGS. FIG. 3 is a perspective view showing a configuration example of the appearance of the photo sticker creating apparatus 1 according to the first embodiment of the present invention. As shown in this figure, the photo sticker creating apparatus 1 mainly includes a photographing unit 11, an editing unit 12, and a subsequent customer service unit 13. The photo sticker creating apparatus 1 is installed in a state in which the photographing unit 11 and the editing unit 12 are in contact with each other, and the editing unit 12 and the subsequent customer service unit 13 are in contact with each other.

(Shooting unit 11)
The photographing unit 11 includes a photographing unit 21 and a background unit 22. The photographing unit 21 and the background unit 22 are installed with a predetermined distance apart. Shooting control is performed in a shooting space that is a space formed between the shooting unit 21 and the background unit 22.

  The photographing unit 21 is a device that allows a user to perform photographing control. The photographing unit 21 enters the photographing space and is located in front of the user who is performing the photographing control. A front panel 41 of the photographing unit 21 that constitutes a surface in which the photographing space is desired in front is provided with a touch panel monitor used by the user during photographing control. Assuming that the left side is the left side and the right side is the right side as viewed from the user in the imaging space, the left side of the imaging unit 21 is configured by the side panel 42A, and the right side is configured by the side panel 42B.

  The background portion 22 is attached to the left end of the rear panel 51 and the rear panel 51, which are plate-like members positioned on the rear side of the user who is performing the shooting control facing the front, and has a narrower width than the side panel 42A. And a side panel 52B (FIG. 4) which is a plate-like member attached to the right end of the rear panel 51 and having a narrower width than the side panel 42B.

  The side panel 42A constituting the left side surface of the photographing unit 21 and the side panel 52A of the background unit 22 are provided on substantially the same plane, and their upper portions are connected by a connecting part 23A that is a plate-like member. Further, the side panel 42B constituting the right side surface of the photographing unit 21 and the side panel 52B of the background portion 22 are provided on substantially the same plane, and their upper portions are connected by a connecting portion 23B which is a plate-like member.

  An opening formed by being surrounded by the side panel 42 </ b> A of the photographing unit 21, the connecting part 23 </ b> A, and the side panel 52 </ b> A of the background part 22 becomes an entrance G <b> 1 of the photographing space. An opening formed by being surrounded by the side panel 42B of the imaging unit 21, the connecting unit 23B, and the side panel 52B of the background unit 22 is also an entrance G2 of the imaging space.

  The background curtain unit 25 is provided above the background portion 22 so as to be supported by the back panel 51, the side panel 52A, and the side panel 52B. The background curtain unit 25 stores a plurality of roll-up type background curtains having different colors or patterns and used for the background. The background curtain unit 25 operates in conjunction with shooting by a camera or the like provided in the shooting unit 21. At the time of shooting, for example, the curtain of the color selected by the user is lowered and the other curtains are wound up.

  As the background curtain stored in the background curtain unit 25, an elevating curtain that uses a plurality of curtains as a single curtain for curtains may be prepared. In addition, a blackboard curtain is pasted on the rear panel 51 that is the back of the shooting space, and a plurality of background images for synthesis are prepared. In editing control described later, a user-desired background image is used as a curtain portion. It may be possible to synthesize.

  An opening surrounded by the front panel 41 of the photographing unit 21, the connecting part 23A, the connecting part 23B, and the background curtain unit 25 is formed above the photographing space, and the ceiling strobe unit 24 covers the part of the opening. Provided. One end of the ceiling strobe unit 24 is fixed to the connecting portion 23A, and the other end is fixed to the connecting portion 23B. The ceiling strobe unit 24 has a built-in strobe that emits light toward the photographing space in accordance with photographing. A fluorescent lamp is provided inside the strobe constituting the ceiling strobe unit 24, and also functions as illumination of the photographing space.

(Editor 12)
The editing unit 12 is a device that allows a user to perform editing control, which is processing for editing an image obtained by shooting control. The editing unit 12 is provided so as to be connected to the photographing unit 11 so that one side surface thereof is in contact with the back surface of the front panel 41 of the photographing unit 21. The editing unit 12 is provided with a configuration such as a tablet built-in monitor used by the user during editing control.

  If the configuration of the editing unit 12 shown in FIG. 3 is a front side configuration, the configuration used for editing control is provided on the front side and the back side of the editing unit 12 so that two sets of users can edit simultaneously. Provided.

  The front side of the editing unit 12 is a surface perpendicular to the floor surface, and is composed of a surface 71 that is substantially parallel to the side panel 42A, and a slope 72 formed above the surface 71 for editing control. The configuration used is provided on the slope 72. On the left side of the slope 72, a support portion 73A that has a columnar shape and supports one end of the lighting device 74 is provided. A support portion 75 that supports the curtain rail 26 is provided on the upper surface of the support portion 73A. A support portion 73 </ b> B (FIG. 4) that supports the other end of the lighting device 74 is also provided on the right side of the slope 72.

  A curtain rail 26 is attached above the editing unit 12. The curtain rail 26 is configured by combining three rails 26A to 26C so that the shape when viewed from above is a U-shape. One end of the rails 26A and 26B provided in parallel is fixed to the connecting portion 23A and the connecting portion 23B, respectively, and both ends of the remaining one rail 26C are joined to the other end.

  A curtain is attached to the curtain rail 26 so that the inside of the space in front of the editing unit 12 and the space in front of the back cannot be seen from the outside. The space in front of the editing unit 12 and the space in front of the back of the editing unit 12 surrounded by the curtain attached to the curtain rail 26 are editing spaces in which the user performs editing control.

(Subsequent customer service department 13)
FIG. 4 is a perspective view of the external appearance of the photo sticker creating apparatus as seen from another angle. The subsequent customer service unit 13 is a device that performs the subsequent customer service and allows the user to perform an operation on the subsequent customer service. Subsequent customer service includes processing for transmitting images (including edited images) to the management server 4, processing for causing a user to play a mini game, processing for answering a questionnaire, and the like.

  As shown in FIG. 4, the post-service customer unit 13 has a plate-shaped housing having a predetermined thickness, and is connected to the editing unit 12 so that the back surface thereof is in contact with the left side surface of the editing unit 12. Provided. On the front side of the post-service customer section 13, there are provided a tablet built-in monitor used by the user at the time of operation for the post-service customer, and a discharge port for discharging sticker paper on which an image (including an edited image) is printed. The space in front of the post-service section 13 is a post-service area where a user waiting for printing on the sticker paper to perform an operation on the post-service.

(User movement)
The flow of the photo sticker creation game and the accompanying user movement will be described with reference to FIG. FIG. 5 is a diagram illustrating the movement of the user.

  The user of the photo sticker creating apparatus 1 takes a picture formed between the photographing part 21 and the background part 22 from the entrance / exit G1 as indicated by the white arrow # 1 or from the entrance / exit G2 as indicated by the white arrow # 2. The camera enters the space A1 and performs shooting control using a camera and a touch panel monitor provided in the shooting unit 21.

  It should be noted that, in the shooting of the user's image, not all shooting is to capture an image printed on sticker paper, but for transmission to the mobile terminal 6 (or so that it can be viewed on the mobile terminal 6, In some cases, the image is taken only for transmission to the management server 4. This is because the user's preference may be different between the image to be printed and the image transmitted to the mobile terminal 6, and the user wants to take an image in consideration of transmission to the mobile terminal 6 at the time of the user's shooting. It is in response to the user's intention.

  The user who has finished the shooting control uses the doorway G1 to move out of the shooting space A1 as indicated by the white arrow # 3 and moves to the editing space A2-1 or opens the doorway G2 as indicated by the white arrow # 4. Use to move out of the shooting space A1 and move to the editing space A2-2.

  The editing space A2-1 is an editing space on the front side of the editing unit 12, and the editing space A2-2 is an editing space on the back side of the editing unit 12. Which of the editing space A2-1 and the editing space A2-2 is moved is guided by the screen display of the touch panel monitor of the photographing unit 21 or the like. For example, the empty space of the two editing spaces is guided as the destination. The user who has moved to the editing space A2-1 or the editing space A2-2 starts editing control. The user of the editing space A2-1 and the user of the editing space A2-2 can perform editing control at the same time.

  After the editing control is finished, printing of an image selected from images (including edited images) is started. During printing of the image, the user who has finished the editing control moves from the editing space A2-1 to the subsequent customer service space A3 as indicated by the white arrow # 5 when the editing control is performed in the editing space A2-1. The customer receives the subsequent customer service and performs an operation on the subsequent customer service. In addition, when editing control is performed in the editing space A2-2, the user who has finished editing control moves from the editing space A2-2 to the post-service area A3 as indicated by the white arrow # 6. In the space A3, a subsequent customer service is received and an operation is performed on the customer service.

  When the printing of the image is finished, the user receives the sticker paper from the outlet provided in the post-service section 13 and finishes a series of photo sticker creation games.

(Configuration of each member)
Next, the configuration of each member provided in the photo sticker creating apparatus 1 will be described.

(Photographing unit 21)
FIG. 6 is a diagram illustrating a configuration example of the photographing unit 21 provided in the photo sticker creating apparatus 1. The photographing unit 21 is configured by attaching a front panel 41, a side panel 42A, and a side panel 42B to a base unit 43 having a box shape.

  The front panel 41 is positioned in front of the user who performs shooting control in the shooting space A1, and the side panel 42A and the side panel 42B are positioned on the left and right sides of the user who performs shooting control in the shooting space A1, respectively. The connecting portion 23A is fixed above the side panel 42A, and the connecting portion 23B is fixed above the side panel 42B.

  An imaging / display unit 81 is provided in the approximate center of the front panel 41. The photographing / display unit 81 includes a camera 91, a front strobe 92, and a touch panel monitor 93.

  The camera 91 is configured by an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and photographs a user in the photographing space A1. The moving image captured by the camera 91 is displayed on the touch panel monitor 93 in real time. An image captured by the camera 91 at a predetermined timing such as when a user gives an instruction to shoot is saved as an image (still image).

  The touch panel monitor 93 provided on the lower side of the camera 91 includes a monitor such as an LCD (Liquid Crystal Display) and a touch panel stacked thereon. The touch panel monitor 93 has a function as a live view monitor that displays a moving image captured by the camera 91 and a function that displays various GUI (Graphical User Interface) images and accepts a user's selection operation through the touch panel. Yes. On the touch panel monitor 93, a moving image after an image such as a background is appropriately displayed is displayed.

  On the front panel 41, an upper strobe 82 is installed on the upper side with respect to the position of the photographing / display unit 81. A left strobe 83 is installed on the left side and a right strobe 84 is installed on the right side. A lower strobe 85 is provided at a position below the photographing / display unit 81 such that a part of the upper surface of the base portion 43 protrudes.

  The upper strobe 82 irradiates the user from the upper front. The left strobe 83 irradiates the user from the left front, and the right strobe 84 irradiates the user from the right front. The lower strobe 85 irradiates the user from the lower front.

  The base unit 43 is also provided with a foot strobe 86 that irradiates the user's foot. The upper strobe 82, left strobe 83, right strobe 84, lower strobe 85, and foot strobe 86 are provided with fluorescent lamps. Used as illumination in the space A1. As will be described later, by adjusting the light emission amount of each fluorescent lamp, or by adjusting the number of fluorescent lamps to emit light, the brightness in the shooting space A1 becomes the content of the shooting control performed by the user. It is adjusted accordingly.

  On the right side of the foot strobe 86, a coin insertion / return opening 87 through which a user inserts money is provided.

  Spaces 43A and 43B formed on the left and right sides of the upper surface of the base portion 43 with the lower strobe 85 interposed therebetween are used as a luggage storage place for a user who performs shooting control to place baggage and the like. For example, near the ceiling of the front panel 41, a speaker for outputting sound such as guidance sound for shooting control, BGM (Back Ground Music), and sound effects is also provided.

(Background part 22)
FIG. 7 is a diagram illustrating a configuration example of the background unit 22 provided in the photo sticker creating apparatus 1. As described above, the background curtain unit 25 is provided above the back panel 51. Near the center of the background curtain unit 25, a rear center strobe 101 that irradiates a user who is performing shooting control in the shooting space A1 from the rear center.

  A rear left strobe 102 that irradiates a user who is performing shooting control in the shooting space A1 from the left rear is provided above the rear panel 51 and on the doorway G1 side. Further, a rear right strobe 103 that irradiates the user who is performing the photographing control in the photographing space A1 from the right rear is provided above the rear panel 51 and on the entrance / exit G2 side.

(Editor 12)
FIG. 8 is a diagram illustrating a configuration example of the editing unit 12 provided in the photo sticker creating apparatus 1. As shown in the figure, in the editing unit 12, a tablet built-in monitor 131 is provided in the approximate center of the slope 72. A touch pen 132A is provided on the left side of the tablet built-in monitor 131, and a touch pen 132B is provided on the right side.

  The tablet built-in monitor 131 is configured by superimposing a tablet capable of inputting an operation using the touch pens 132A and 132B on a monitor such as an LCD. On the tablet built-in monitor 131, for example, an editing screen, which is a screen used for editing an image selected as an image to be edited, is displayed. When two people perform editing work simultaneously, the touch pen 132A is used by one user, and the touch pen 132B is used by the other user.

  It should be noted that the image editing performed in the editing unit 12 is not necessarily an editing on an image to be printed, but an image that is assumed to be transmitted only to the mobile terminal 6 by the user (that is, an image that is not printed). May be an edit to This is because there is a difference in user preference between an image to be printed and an image transmitted to the mobile terminal 6 and used on the web. For example, an image used for a blog or the like is printed. This is to respond to the user's intention to edit differently from the editing of the image.

(Subsequent customer service department 13)
On the left side of the editing unit 12, a post-service unit 13 is provided. FIG. 9 is a diagram illustrating a configuration example of the post-service unit 13 provided in the photo sticker creating apparatus 1.

  A monitor 161 with a built-in tablet is provided at the upper center of the post-service section 13. A touch pen 162A is provided on the left side of the tablet built-in monitor 161, and a touch pen 162B is provided on the right side. Similarly to the tablet built-in monitor 131, the tablet built-in monitor 161 is configured by superimposing a tablet capable of performing an operation input using the touch pen 162A or 162B on a monitor such as an LCD. On the tablet built-in monitor 161, a screen used in a post-service process such as a mini game is displayed.

  A non-contact type IC reader / writer 163 is provided under the tablet built-in monitor 161, and a sticker paper discharge port 164 is provided under the non-contact type IC reader / writer 163. A speaker 165 is provided below the sticker paper discharge port 164.

  The non-contact type IC reader / writer 163 communicates with a non-contact type IC that is in close proximity, reads data from the non-contact type IC, and writes data to the non-contact type IC.

  In the sticker paper discharge port 164, sticker paper on which an image created by the editing space A2-1 user performing editing control is printed, or created by the editing space A2-2 user performing editing control. The sticker paper on which the image is printed is discharged. A printer is provided inside the editing unit 12, and printing of images is performed by the printer.

  The speaker 165 outputs sound such as guidance voice, BGM, and sound effects at the time of subsequent customer service.

(Internal structure of photo sticker making apparatus 1)
Next, the internal configuration of the photo sticker creating apparatus 1 will be described. FIG. 10 is a block diagram illustrating an internal configuration example of the photo sticker creating apparatus 1. The same code | symbol is attached | subjected to the same structure as the structure mentioned above. The overlapping description will be omitted as appropriate.

  The control unit 201 includes a CPU (Central Processing Unit) and the like, and executes a program stored in a ROM (Read Only Memory) 206 or a storage unit 202 to control the overall operation of the photo sticker creating apparatus 1. A storage unit 202, a communication unit 203, a drive 204, a ROM 206, and a RAM (Random Access Memory) 207 are connected to the control unit 201. The control unit 201 is also connected with the components of the photographing unit 21, the editing unit 12, the post-service unit 13, and the printing unit 211.

  The storage unit 202 includes a non-volatile storage medium such as a hard disk or a flash memory, and stores various setting information supplied from the control unit 201. Information stored in the storage unit 202 is appropriately read out by the control unit 201.

  A communication unit 203 is an interface of a network such as the Internet, and communicates with an external device according to control by the control unit 201.

  A removable medium 205 made of an optical disk or a semiconductor memory is appropriately attached to the drive 204. The computer program and data read from the removable medium 205 by the drive 204 are supplied to the control unit 201 and stored in the storage unit 202 or installed.

  The ROM 206 stores programs and data executed by the control unit 201. The RAM 207 temporarily stores data and programs processed by the control unit 201.

  The imaging unit 21 performs processing related to the user's imaging control in the imaging space A1. The imaging unit 21 includes a coin processing unit 221, a background control unit 222, an illumination control unit 223, a camera 91, a touch panel monitor 93, and a speaker 224.

  The coin processing unit 221 detects the insertion of coins into the coin insertion / return port 87. When the coin processing unit 221 detects that a predetermined amount of coins such as 300 yen has been inserted, the coin processing unit 221 outputs an activation signal indicating that to the control unit 201.

  The background control unit 222 raises and lowers the background curtain stored in the background curtain unit 25 according to the background control signal supplied from the control unit 201. The background curtain may be manually selected by the user.

  The illumination control unit 223 controls the light emission of each strobe in the imaging space A1 according to the illumination control signal supplied from the control unit 201. As described above, in the photographing space A1, in addition to the strobe of the ceiling strobe unit 24, an upper strobe 82, a left strobe 83, a right strobe 84, a lower strobe 85, a foot strobe 86, and a background portion provided in the photographing unit 21 are provided. 22 is provided with a back center strobe 101, a back left strobe 102, and a back right strobe 103. A fluorescent lamp is also provided inside each strobe.

  In addition, the illumination control unit 223 adjusts the light emission amount of the fluorescent lamp provided in each strobe, or adjusts the number of fluorescent lamps to emit light, thereby performing the shooting control stage performed by the user. Accordingly, the brightness in the shooting space A1 is adjusted.

  The camera 91 shoots according to the control by the control unit 201 and outputs an image (image) obtained by the shooting to the control unit 201.

  The editing unit 12A performs processing related to user editing control in the editing space A2-1. The editing unit 12A includes a tablet built-in monitor 131, touch pens 132A and 132B, and a speaker 231 provided on the front side of the editing unit 12. The editing unit 12B has the same configuration as the editing unit 12A, and performs processing related to user editing control in the editing space A2-2.

  The tablet built-in monitor 131 displays an editing screen according to control by the control unit 201 and detects a user operation on the editing screen. A signal representing the content of the user's operation is supplied to the control unit 201, and the image to be edited is edited.

  The subsequent customer service section 13 performs processing related to the subsequent customer service in the subsequent customer service space A3. The subsequent customer service unit 13 includes a tablet built-in monitor 161, touch pens 162A and 162B, a non-contact type IC reader / writer 163, and a speaker 165.

  The printing unit 211 includes a printer 241 and a sticker unit 242 that are configured to perform printing control. The printer 241 prints the image or the edited image obtained by the editing control by the user on the sticker paper stored in the sticker paper unit 242 attached to the printer 241 and discharges it to the sticker paper discharge port 164. . The subsequent customer service unit 13 and the printing unit 211 that are surrounded by a one-dot chain line are configured to perform processing related to the subsequent customer service for the user in the subsequent customer service space A3.

  In addition, the printing unit 211 acquires the image transmitted to the management server 4 outside the margin (such as the edge of the sticker paper) in addition to printing the photographed image (or the edited image) on the sticker paper in the process related to the print control. URL (Uniform Resource Locater) and a seal ID (image acquisition ID) may be printed. The sticker ID is an identifier for uniquely identifying one or a plurality of images (including a photographed image and an edited image) generated during one game in the photo sticker creating apparatus 1. This is an identifier for acquiring the generated image. The user accesses a URL for acquiring an image and inputs a sticker ID, whereby a web page (image acquisition page) for acquiring an image generated during one game in the photo sticker creating apparatus 1 is displayed. It can be displayed on the portable terminal 6.

  This is because, for example, when receiving an e-mail address input from the user at the post-service department 13 or the like, a correct address cannot be received due to a time-out or error input, and information such as a URL for acquiring an image is transmitted to the user. This is to allow the user to acquire an image by accessing the URL printed on the sticker paper and inputting the sticker ID when the user cannot do so.

  Note that the printing unit 211 may print, for example, a predetermined mail address or a QR code (registered trademark) as the information for acquiring an image instead of the URL. When a predetermined mail address is printed, the user receives a mail describing a URL for acquiring an image from the management server 4 by sending a blank mail to the predetermined mail address. Can do. In addition, when the QR code is printed on the sticker paper, the user can directly access the image acquisition page by accessing the web page indicated by the QR code using the mobile terminal 6.

(Configuration of control unit)
FIG. 11 is a functional block diagram illustrating a detailed configuration example of the control unit 201 provided in the photo sticker creating apparatus 1. As shown in this figure, the control unit 201 includes a communication control unit 311 (second transmission means), an imaging control unit 312, an editing control unit 313, a print control unit 314, and a post-service customer control unit 315.

  The communication control unit 311 controls communication with an external device by the communication unit 203.

  The shooting control unit 312 relates to a process related to a charge inserted when starting the photo sticker creation game by the shooting unit 21 and a user shooting control in the photo sticker creation game such as shooting a user group according to the user shooting control. Control processing. The shooting control unit 312 can also generate date / time information related to the date / time when the image was shot. The photographing control unit 312 (photographed image generating means) generates a photographed image of a person by photographing.

  The editing control unit 313 controls processing related to user editing control in the photo sticker creation game, such as graffiti editing of a captured image by the editing unit 12A and the editing unit 12B. The editing control unit 313 also controls processing for performing face correction on the captured image generated by the imaging control unit 312 in the live view display at the time of shooting and the confirmation image display after the shooting is determined. The edit control unit 313 includes a feature amount specifying unit 313a (feature amount specifying unit) and an image generating unit 313b (corrected image generating unit).

  The feature amount specifying unit 313a first recognizes a facial part included in an image by face recognition processing, and specifies a feature amount of a person's face included in a captured image (captured image). The facial feature amount is, for example, at least one of a face position, a face size, a face outline (shape), a face part position, a face part size, and a face part shape. Represents. For example, the feature amount specifying unit 313a specifies the eye position, the eye size, the eye shape, and the like as the face feature amount.

  The image generation unit 313b generates a corrected image in which a face included in the captured image is corrected based on preset setting information and a user instruction (editing work) from a captured image obtained by capturing. For example, the image generation unit 313b corrects the skin color and the eye size in the captured image. The corrected image is synthesized with an image (foreground image) indicating a handwritten character or the like input from the tablet built-in monitor 131 by the user and a background image selected by the user. The background image is synthesized by photographing the user with a curtain of a color such as green or blue and applying a chroma process for identifying a person area.

  The print control unit 314 controls processing related to user print control in the photo sticker creation game, such as sticker sheet printing by the printing unit 211.

  The post-service customer control unit 315 controls processing related to the post-service in the photo sticker creation game, such as processing to serve a user group that has finished the editing work by the post-service unit 13. The post-service customer control unit 315 includes a face image extraction unit 315a (face image extraction unit), a face image presentation unit 315b (face image presentation unit), and a person information input unit 315c (person information input unit).

  The face image extraction unit 315a extracts a face image of a person included in the corrected image or the captured image. When the image includes a plurality of human faces, the face image extraction unit 315a extracts each face image.

  The face image presenting unit 315b displays the extracted face image on the tablet built-in monitor 161 in order to present the face image to the user.

  The person information input unit 315c receives input of person information related to the person of the face image displayed from the user via the tablet built-in monitor 161.

(Management server 4)
Next, details of the management server 4 will be described with reference to FIG. FIG. 1 is a block diagram showing a main configuration of the management server 4 according to the first embodiment of the present invention. As shown in this figure, the management server 4 mainly includes a control unit 401, a storage unit 402, and a communication unit 403. The control unit 401 performs overall control of each member provided in the management server 4. The storage unit 402 stores various data used in the management server 4. The communication unit 403 communicates with other devices constituting the captured image management system 10.

(Details of control unit 401)
As illustrated in FIG. 1, the control unit 401 includes a communication control unit 411 (a feature amount acquisition unit, a corrected image acquisition unit, a person information acquisition unit, a first transmission unit), a mail generation unit 412, a web page generation unit 413, an image. A management unit 414 (image management unit, person determination unit) and an image processing unit 415 (image processing unit) are provided.

(Communication control unit 411)
The communication control unit 411 controls communication by the communication unit 403.

(Mail creation unit 412)
The mail creating unit 412 creates a mail to be sent to the mail address of the user who played with the photo sticker creating apparatus 1. In the present embodiment, the mail creation unit 412 creates a mail that guides the storage of an image that is transmitted to the mail address input from the communication control unit 411 using the data stored in the mail database 421. And output to the communication control unit 411.

(Web page creation unit 413)
The web page creation unit 413 creates a web page displayed on the mobile terminal 6. In the present embodiment, the web page creation unit 413 creates a web page corresponding to the access request received from the mobile terminal 6 using the data stored in the web page database 423 and outputs the web page to the communication control unit 411. . Although details will be described later, the web page creation unit 413 creates a web page corresponding to the user's membership type.

  In the following description, “page” refers to data that the management server 4 transmits to a communication terminal such as the mobile terminal 6 for display, such as a web page. The page usually includes text data and image data, and may further include audio data and moving image data.

(Image management unit 414)
The image management unit 414 manages images provided to the user through the image providing site. In the present embodiment, the image management unit 414 stores, in the image database 424, an image designated by the user among images created by the photo sticker creating apparatus 1. The image management unit 414 manages facial feature amount information corresponding to a person included in an image and person information related to the person. For each user, the image management unit 414 stores the image (corrected image), facial feature amount, and personal information of the user in the image database 424 in association with each other. Here, the feature amount of the face is a feature amount specified from the captured image before the face is corrected. The image management unit 414 has a function of confirming or correcting information stored in the image database 424 such as personal information.

(Image processing unit 415)
The image processing unit 415 performs various image processing on the images stored in the image database 424.

(Details of storage unit 402)
As shown in FIG. 1, the storage unit 402 includes a mail database 421, a member database 422, a web page database 423, and an image database 424 (database).

(Mail database 421)
The mail database 421 stores various data for creating a mail to be transmitted to the user.

(Member database 422)
The member database 422 stores various types of information related to users who are members of the image providing site. In the present embodiment, the member database 422 stores information (user specifying information) for specifying a user who uses the photo sticker creating apparatus 1 for each user. Examples of the user specifying information include a user's member ID, e-mail address, nickname, date of birth, address, member attribute, membership date, and terminal ID of a mobile terminal held by the user. Moreover, as a member attribute, a pay member, a free member, etc. can be mentioned, for example.

(Web page database 423)
The web page database 423 stores various data for creating a web page of an image providing site displayed on the mobile terminal 6.

(Image database 424)
The image database 424 stores images instructed by the user to be stored in the image providing site among the user images (corrected images) created by the photo sticker creating apparatus 1. In the image database 424, images are stored separately for each user registered as a member in the member database 422. Further, the image database 424 stores an image (corrected image), a facial feature amount, and person information of a person corresponding to the feature amount in association with each other. Note that the image database 424 may store not only the corrected image but also the captured image before correction.

  Although not shown, the storage unit 402 also has a temporary storage database that temporarily stores data such as images received from the photo sticker creating apparatus 1.

(Details of communication unit 403)
In this embodiment, the photo sticker creating apparatus 1 and the mobile terminal 6 are communication partners of the communication unit 403. The communication unit 403 receives data from another device or transmits data to another device under the control of the communication control unit 411.

(Mobile terminal 6)
FIG. 12 is a block diagram showing a main configuration of the mobile terminal 6 according to Embodiment 1 of the present invention. As shown in this figure, the mobile terminal 6 mainly includes a control unit 601, a storage unit 602, a communication unit 603, an input unit 604, and a display unit 605. The control unit 601 performs overall control of each member provided in the mobile terminal 6. The storage unit 602 stores various data used in the mobile terminal 6. The mobile terminal 6 communicates with other devices constituting the captured image management system 10. The input unit 604 is a device for a user to input information to the mobile terminal 6 such as a touch panel. The display unit 605 is a device for displaying information, such as a liquid crystal display device.

(Details of control unit 601)
As illustrated in FIG. 12, the control unit 601 includes a communication control unit 611, an input control unit 612, and a display control unit 613.

  The communication control unit 611 controls communication by the communication unit 603. The input control unit 612 controls information input in the input unit 604. The display control unit 613 controls information display on the display unit 605. For example, when the display control unit 613 outputs the web page input from the communication control unit 611 to the display unit 605, the web page can be displayed on the display unit 605.

(Member registration process)
In the photographed image management system 10, a user who has registered as a member on the image providing site can save the photographed image on the image providing site. The user registers himself / herself as a free member in the image providing site according to a predetermined procedure. The procedure will be described below with reference to FIG. FIG. 13 is a screen transition diagram illustrating an example of a web page displayed on the display unit 605 of the mobile terminal 6 during the member registration process.

  The user accesses the image providing site using the portable terminal 6 in order to register as a member in the image providing site. As a result, the top page 1001 of the image providing site as shown in FIG. 13A is displayed on the display unit 605 of the mobile terminal 6. The top page 1001 includes at least a link image 1002 for proceeding with the member login process and a link image 1003 for proceeding with the free member registration process.

  The user operates the input unit 604 of the mobile terminal 6 to select a link image 1003 for proceeding with free member registration. As a result, the portable terminal 6 communicates with the management server 4, and as a result, the web page 1011 for free member registration as shown in FIG. 13B is displayed on the display unit 605.

  The web page 1011 includes an input form 1012 for inputting information (user specifying information) for specifying a user who is registered as a free member, and a link image 1013 for transmitting the input user specifying information to the management server 4. Including at least. The user operates the input unit 604 of the mobile terminal 6 to input various personal information such as the user's member ID, password, and email address to the input form 1012 as user identification information. The user selects the link image 1013 after completing the input of the user specifying information. As a result, the mobile terminal 6 transmits the user specifying information to the management server 4 and requests the management server 4 to register the user for free membership.

  The communication control unit 411 of the management server 4 receives the user specifying information transmitted from the mobile terminal 6 through the communication unit 403. The communication control unit 411 outputs the received user identification information to the mail creation unit 412 and instructs the mail creation unit 412 to create a member registration email. In response to this instruction, the mail creation unit 412 creates a member registration mail 1021 as shown in FIG. The member registration mail 1021 is a mail addressed to the mail address included in the received user identification information, and includes a URL 1022 to a web page indicating completion of free member registration. The mail creation unit 412 outputs the created member registration mail 1021 to the communication control unit 411. The communication control unit 411 transmits the input member registration mail 1021 to the user's mail address via the communication unit 403.

  The portable terminal 6 receives the transmitted member registration mail 1021 and displays it on the display unit 605. The user confirms the displayed member registration mail 1021 and selects the URL 1022. Thereby, the mobile terminal 6 requests the management server 4 for a web page specified by the URL 1022.

  In response to this request, the management server 4 completes free member registration. Specifically, the management server 4 creates an account area for a user (a storage area for storing member-specific data) in the member database 422, and stores the user specifying information in the account area. At this time, information indicating that the user is a free member is added to the account area. Finally, the management server 4 creates a web page indicating that the free member registration is completed and transmits it to the mobile terminal 6.

  The user can register himself / herself as a paying member in the image providing site by logging in to the image providing site by entering a member ID and password after registering a free member and following a predetermined procedure. After this registration is completed, information indicating that the user is a paid member is added to the user's account area.

(Service flow)
Next, details of the flow of services provided by the captured image management system 10 will be described below with reference to FIGS. 14 and 15. FIG. 14 is a sequence diagram illustrating an example of a flow of a series of processes from photography to image storage in the photographed image management system 10.

(photography)
As shown in FIG. 14, the user starts playing the game in the photo sticker creating apparatus 1 (step S1). Specifically, the user stands in front of the front panel 41 of the photo sticker creating apparatus 1 and inserts coins having a fee required for playing the game into the coin insertion / return slot 87. The imaging control unit 312 determines whether or not a predetermined amount of coins has been inserted based on a signal supplied from the coin processing unit 221.

  If it is determined that the camera has been inserted, the shooting control unit 312 controls the shooting unit 21 to display the moving image of the subject captured by the camera 91 on the touch panel monitor 93 and display it in the shooting space A1 according to the shooting control by the user. The user is photographed as a subject (step S2). The shooting control unit 312 generates a shot image in which a person is shot. This photographing control is performed a plurality of times within the photographing processing time.

  In addition, as a process in the imaging control, a predetermined number of times may be taken as an image for transmission to the mobile terminal 6 instead of an image for printing on a sticker sheet. In this case, as an image suitable for an image to be transmitted to the mobile terminal 6, an image obtained by chroma-combining the image for synthesis and an image subjected to image processing such as filtering processing using a color filter are created at the time of shooting. Sometimes. At this time, the face correction process selected by the user or the face correction process set in advance may be performed.

  After the photographing is finished, the photographing control unit 312 guides the user in the photographing space A1 to move to the editing space A2-1 or the editing space A2-2. This guidance is performed by displaying a guidance screen on the touch panel monitor 93 of the photographing unit 21 or by outputting voice guidance from the speaker 224.

(edit a picture)
After the end of the guidance, the editing control unit 313 controls the editing unit 12 corresponding to the editing space of the editing space A2-1 and the editing space A2-2 that is the destination of the user who has finished the shooting control. Thus, the editing unit 12 is caused to edit the photographed image (step S3). Specifically, the edit control unit 313 causes the user to select a synthesis image to be synthesized with the captured image selected or acquired as the editing target image, and synthesizes the selected synthesis image with the captured image. The obtained image is edited according to the editing operation by the user.

  The feature amount specifying unit 313a of the edit control unit 313 specifies the feature amount of the person's face included in the captured image. Here, the facial feature amount in the photographed image is the facial feature amount in the image before the face is corrected.

  The editing control unit 313 receives a correction instruction related to face correction from the user via the editing unit 12. For example, the user inputs correction instructions such as skin color, eye size, eye shape, eyelash density, facial contour, and jaw shape to the editing unit 12. Note that the editing control unit 313 also receives an editing instruction such as synthesizing handwritten characters (doodles) from the user.

  The image generation unit 313b of the editing control unit 313 corrects the face included in the captured image based on the user's correction instruction. In the face correction, the image generation unit 313b recognizes the position of the face and the position of the face parts (eyes, nose, mouth, etc.) based on the specified facial feature amount, and the size of each part. Correct (change) the shape and color. Thereby, the image generation unit 313b generates a corrected image in which a human face included in the captured image is corrected. Note that a composite image and handwritten characters (graffiti) are also synthesized with the corrected image.

  In addition, as editing for an image for transmission to the mobile terminal 6, for example, processing such as processing for trimming an image to a predetermined size, filtering processing using a color filter, and collage synthesis processing for finishing a plurality of images as one image Can also be done.

  After the editing, the editing control unit 313 guides the user who has finished editing control in the editing space A2-1 or the editing space A2-2 to move to the post-service space A3. This guidance is performed by displaying a guidance screen on the tablet built-in monitor 131 or by outputting guidance voice from the speaker 231.

(Photo sticker printing)
After the guidance is completed, the print control unit 314 outputs the edited image (corrected image) selected by the user to the printer 241 and prints it on the sticker sheet (step S4). When printing is completed, the customer service control unit 315 displays that the printing has been completed on the tablet built-in monitor 161 so that the printer 241 can acquire images from the management server 4 and the photo sticker creation apparatus 1 in the same procedure. The sticker ID created in the photo sticker creation device 1 and image acquisition information (URL of the image acquisition site, e-mail address for acquisition, etc.) are printed out of the box, and a predetermined image is printed in the center. The sealed sheet is discharged to the sheet discharge port 164.

(Subsequent customer service)
During printing, the subsequent customer service control unit 315 controls the subsequent customer service unit 13 and controls processing related to the subsequent customer service for the user who is waiting for the completion of printing (step S5). Specifically, the post-service customer control unit 315 manages information such as images (including edited images) and user email addresses in the photo sticker creation game as processing related to post-service by the post-service customer unit 13. Control the process to send to.

  Specifically, the customer service control unit 315 causes the tablet built-in monitor 161 to display a selection screen for an image to be transmitted. The user selects an image to be transmitted to the management server 4 by inputting an instruction to the tablet built-in monitor 161 using the touch pen 162A or the touch pen 162B.

  FIG. 19 is a transition diagram illustrating an example of a procedure for inputting personal information. Here, the description will be made assuming that the corrected image P1 shown in FIG. 19A is selected for transmission. In FIG. 19, the alphabet which shows a person is shown in the person of an image for easy understanding.

  The face image extraction unit 315a extracts a face image included in the image from the image (corrected image) P1 selected for transmission. When a plurality of human faces are included in the image, the face image extraction unit 315a extracts a plurality of face images PA and PB (FIG. 19B). The face image PA is an image including the face of the person A, and the face image PB is an image including the face of the person B. Note that the face image extraction unit 315a may extract a face image from a captured image corresponding to the corrected image.

  The face image presenting unit 315b displays one or more extracted face images on the tablet built-in monitor 161 in order to present the face image to the user. In addition, the face image presenting unit 315b displays on the tablet built-in monitor 161 screens 1061 and 1064 that allow the user to input person information corresponding to the person in the face image. Here, input of a person's nickname and person's mail address is accepted as person information. The e-mail address is used, for example, to provide an image generated by the photo sticker creating apparatus 1 to the user at the image providing site.

  The person information input unit 315c receives input of person information related to the person of the face image displayed from the user via the tablet built-in monitor 161. In the photo sticker creating apparatus 1, a plurality of users often shoots together. For example, when a plurality of users A and B are included in the image, the users A and B input their nicknames and mail addresses. It should be noted that input of person information of some persons among a plurality of persons can be omitted.

  As shown in FIG. 19C, the extracted face images PA and PB are displayed on the nickname input screen 1061. On the nickname input screen 1061, a nickname input area 1062 and a software keyboard 1063 are displayed corresponding to each face image. The user inputs a nickname using the software keyboard 1063. The input characters are displayed in the nickname input area 1062. When the input of the nickname is completed, the display on the tablet built-in monitor 161 is switched to the mail address input screen 1064.

  As shown in FIG. 19 (d), the extracted face image PA / PB, the mail address input area 1065, and the software keyboard 1063 are displayed on the mail address input screen 1064. The user inputs a mail address using the software keyboard 1063. The input characters are displayed in the mail address input area 1065.

(Image transmission)
The communication control unit 311 associates the corrected image, the facial feature amount specified in the pre-correction captured image corresponding to the corrected image, and the input personal information (nickname and e-mail address), and the communication unit 203. To the management server 4 (step S6). When a plurality of persons are included in one corrected image, the facial feature amounts of the plurality of persons and the person information of the plurality of persons are associated with one corrected image. The feature amount of each person and the person information of each person are associated with each other. The image to be transmitted is a selected corrected image or a corrected image determined by the control unit 201 as a transmission target. Further, an identifier (image ID) for uniquely identifying the image is associated with the image transmitted at this time. The communication unit 403 of the management server 4 receives the image, the facial feature amount, the person information (nickname and mail address), and the image ID transmitted from the photo sticker creating apparatus 1 (step S7), and sends them to the communication control unit 411. Output. The communication control unit 411 stores the received image, face feature amount, personal information, and other data in a temporary storage database (not shown) of the storage unit 402. The stored data such as an image is continuously stored in the temporary storage database until a predetermined time limit elapses, and is automatically deleted from the temporary storage database when the time limit elapses.

  Note that the communication control unit 311 may transmit all images to the management server 4 in consideration of not only a predetermined number of images selected by the user but also a future paying member. As a result, the user can acquire all images taken in the state of the free member when the user becomes a paying member on the site after shooting in the state of the free member.

(Compose and send a guide email)
The communication control unit 411 outputs the received email address and image ID to the email creation unit 412. The mail creating unit 412 creates a guide mail that is transmitted to the mail address received from the communication control unit 411 and guides the user to save the image. The procedure is as follows. The mail creation unit 412 outputs the image ID to the web page creation unit 413 and requests the web page creation unit 413 to generate a URL 1102 indicating a web page for storing the image specified by the image ID. The web page creation unit 413 generates the requested URL 1102 and outputs it to the mail creation unit 412.

  The mail creation unit 412 acquires data for creating a guide mail from the mail database 421. Then, the guide mail 1101 shown in FIG. 15A is created using the data, the URL 1102 and the user's mail address. A URL 1102 is described in the text of the guide mail 1101. The mail creation unit 412 outputs the created guide mail 1101 to the communication control unit 411. The communication control unit 411 transmits the guide mail 1101 to the user's mail address via the communication unit 403 (step S8).

  Note that the mail creation unit 412 may transmit a guide mail including a URL indicating a web page that only allows the user to input a seal ID instead of a web page that collects the image ID and the image related to the image ID. Good. In this case, the management server 4 receives an input of a seal ID printed outside the margin of the sticker paper from the mobile terminal 6 on the web page indicated by the URL included in the guidance mail, and is transmitted to the management server 4. You may make it display the image which is.

(Request and send web page for saving images)
The communication unit 603 of the portable terminal 6 receives the transmitted guide mail 1101 (step S9) and outputs it to the communication control unit 611. The communication control unit 611 outputs the input guide mail 1101 to the display control unit 613. The display control unit 613 outputs the input guide mail 1101 to the display unit 605. The user confirms the guide mail 1101 displayed on the display unit 605, and selects the URL 1102 using the input unit 604 in order to save the image on the image providing site (step S10).

  The input control unit 612 receives a selection input from the input unit 604 and notifies the communication control unit 611 that the URL 1102 has been selected. In response to this, the communication control unit 611 requests the management server 4 for a web page corresponding to the selected URL 1102. The specific procedure is as follows. The communication control unit 611 acquires the user member ID stored in the storage unit 602, and transmits a predetermined request for providing a web page including the member ID to the management server 4 through the communication unit 603.

  The communication control unit 411 receives the transmitted request through the communication unit 403 (step S11) and outputs it to the web page creation unit 413. The web page creation unit 413 determines the member type of the user by referring to the member database 422 for the member ID included in the request (step S12). In the present embodiment, the web page creation unit 413 determines whether the user is a free member or a paid member, and creates a web page according to the determination result. If the web page creation unit 413 determines that the user is a free member, the web page creation unit 413 creates a web page 1103 for the free member to store images as shown in FIG. On the other hand, when it is determined that the user is a paying member, a web page 1107 for the paying member to save the image is created as shown in FIG.

  As shown in FIG. 15B, the web page 1103 includes one thumbnail image 1104, a link image 1105 that prompts the user to register a paying member, and a link image 1106 for storing one image on the image providing site. Is included. If the user is a free member, the number of images that can be stored is one, and the thumbnail image 1104 is a thumbnail of the one image. Note that the single image is an image selected by the user in the photo sticker creating apparatus 1. On the other hand, when there are a plurality of images, the thumbnail image 1104 is any one of the plurality of images.

  As shown in FIG. 15C, the web page 1107 includes one thumbnail image 1104 and a link image 1108 for storing all the images.

  In this example, since the user is a free member, the web page creation unit 413 creates a web page 1103 for a free member. The web page creation unit 413 outputs the created web page 1103 to the communication control unit 411. The communication control unit 411 transmits the input web page 1103 to the mobile terminal 6 through the communication unit 403 (step S13).

(Save image)
The portable terminal 6 receives the web page 1103 through the communication unit 603 (step S14). The web page 1103 is output to the display control unit 613 via the communication control unit 611. The display control unit 613 outputs the input web page 1103 to the display unit 605. The user confirms the web page 1103 displayed on the display unit 605. When the user is a free member, even if there are a plurality of images, only one of them can be stored in the image providing site. When the user wants to save one image, the user selects the link image 1106 through the input unit 604. Note that the image to be stored is not limited to the image selected as the image to be transmitted to the mobile terminal 6 in the photo sticker creating apparatus 1, and is designated as an image to be stored in the mobile terminal 6 accessing the image providing site, for example. It may be an image.

  The input control unit 612 receives a selection input from the input unit 604 and notifies the communication control unit 611 that the link image 1106 has been selected. In response to this, the communication control unit 611 transmits an image storage instruction to the management server 4 (step S15).

  The communication control unit 411 receives the transmitted storage instruction through the communication unit 403 (step S16). As a result, the communication control unit 411 instructs the image management unit 414 to store the image. In response to this instruction, the image management unit 414 stores the user image stored in the temporary storage database in the image database 424 in association with the user member ID. At that time, if there is only one image, the image is saved. On the other hand, if there are a plurality of images, one of them (for example, an image corresponding to the thumbnail image 1104) is stored.

  Further, the image management unit 414 stores the facial feature amount and the person information associated with the stored image in the image database 424 in association with the image (step S17). When a plurality of persons are included in one image, a plurality of feature amounts and person information are associated with one image. Specifically, the correspondence between each person included in the image, each feature amount, and each person information is specified in the photo sticker creating apparatus 1, and the correspondence is stored in the image database 424. For example, the image management unit 414 uses the lower left corner of the image as the origin (0, 0) and assigns numbers (person numbers) to the persons in order of increasing x-coordinate values at the center of the person's face area. Associate quantity and person information. The coordinates of the center of the face area may be included in the feature amount. When the x-coordinate values are the same, the person numbers are assigned to the persons in the order from the smallest y-coordinate value. Also, the facial feature amount of a person among the plurality of persons and the person information of the person are stored in the image database 424 in association with each other. Further, as will be described later, the image management unit 414 regards a plurality of persons included in the image as being in a friend relationship, and uses the feature amount and person information of the persons appearing together in the image database 424 as friend information. save. Note that the image management unit 414 may cut out a face image including each person's face from the image, associate the face image with the corresponding feature amount and the corresponding person information, and save the image in the image database 424 separately. The image is stored in the image database 424 in association with the member ID of the user who is the registrant of the image. The registrant of the image is a user who has saved the image in the image database 424.

  FIG. 18 is a diagram illustrating an example of data stored in the image database 424. In FIG. 18, for easy understanding, an alphabet indicating a person is shown in the person of the image. The image database 424 stores images (corrected images), image IDs, facial feature amounts, nicknames, e-mail addresses, and person attributes, classified by member ID. In FIG. 18, for example, information associated with five images P1 to P5 of the user A is shown. The image ID is information that is generated in the photo sticker creating apparatus 1 and uniquely identifies the image. The image ID may include the date and time when the image was captured, information indicating the photo sticker creation apparatus 1 that captured the image, and an arbitrary character string. The location where the image was taken can also be specified by information indicating the photo sticker creating apparatus 1 that has taken the image. A plurality of nicknames may be stored for one person as necessary. For example, when a person (same person) having the same member ID is included in a plurality of images, nicknames associated with the images (input by the user) do not always match. Therefore, a plurality of nicknames may all be associated with one person.

  The facial feature amount, nickname, e-mail address, and person attribute are associated with the image for each person included in the image. In FIG. 18, in order to simplify the description, the facial feature values are shown in alphabetic combinations. Actually, the facial feature quantity is a multidimensional vector (feature quantity vector) representing the facial feature of a person. That is, the feature amount is a multidimensional vector (feature amount vector) including a plurality of values (data) representing features of each part of the face. The nickname and e-mail address are a nickname and e-mail address of a person included in the image input by the user. Like the images P2, P4, and P5, the nickname and / or mail address may be omitted. The person attribute indicates a human relationship viewed from the user A of the person included in the image. The person attribute is set as “Principal” for the user A and “Friend” for persons other than the user A. The method for determining the person attribute will be described later.

  Note that a user who is a free member selects the link image 1105 when he wants to save all images. In this case, a series of processing for paying member registration is performed in the captured image management system 10. When registration is completed, all images are stored in the image database 424 in association with the member ID of the user.

  In addition, the saving process when the user is a paying member is essentially the same as the saving process when the user is a free member. However, the user selects the link image 1108 included in the web page 1107. As a result, the management server 4 stores all images in the user image storage area in the image database 424. Note that the selection of an image to be saved by the user (one image for a free member and all images for a paid member) may be accepted by the photo sticker creating apparatus 1. In this case, the image management unit 414 may store the image selected in the photo sticker creating apparatus 1 in the image database 424.

(Determination of person attributes)
When the image management unit 414 stores the image P1 (see FIG. 18), each mail address associated with the image P1 of the user A is registered in the member database 422 (user information management unit). It is determined whether or not it matches the mail address (user information). When the mail address (xxx@xxx.xx) associated with the image P1 matches the mail address of the user A, the image management unit 414 determines that the person corresponding to the mail address (xxx@xxx.xx) is The user A is identified. On the other hand, when the mail address (yy@xxx.xx) associated with the image P1 does not match the mail address of the user A, the image management unit 414 displays the person corresponding to the mail address (yy@xxx.xx). Specifies that it is a friend of user A (that is, not user A himself) as a person who has taken a picture with user A. If no e-mail address is input, the image management unit 414 specifies that the person is a friend of the user A. The image management unit 414 stores the identified person attribute in the image database 424 in association with each feature amount (step S18). Note that the image management unit 414 may store the specified person attribute in the image database 424 as friend information in association with the person information.

  Note that if the person whose face feature value is “KK” has already been identified from the image stored in the past as the user A himself, the image management unit 414 determines the face feature value associated with the image. May identify the person corresponding to “KK” as the “person”. In addition, the image management unit 414 may specify a person whose face feature amount does not correspond to “KK” as a “friend”. Note that the image management unit 414 considers an error of the feature amount of the captured face and determines that the feature amount corresponds (“person”) if the difference in the feature amount is within a predetermined range.

  Note that the image management unit 414 may identify a member ID of a person who is not the user A himself / herself (for example, the person B) with reference to the member database 422 from the mail address of the person. The image management unit 414 may store the identified member ID in the image database 424 in association with the feature amount, nickname, and the like of the person.

  Further, the management server 4 obtains the nickname “B-chan” of the friend of the user A from the personal information (nickname) of the person (friend) included in the accumulated friend information of the user A and the member ID corresponding to the nickname. It may be estimated which member ID corresponds to the person. Further, the management server 4 determines the member ID of the person based on the feature amount of all the past friend information of the user A and the feature amount of the person who is included in the image but does not input the e-mail address or nickname. You may guess. The management server 4 stores the estimated correspondence relationship in the database. A simple nickname may be used by a plurality of persons, but by doing so, it is possible to determine which member ID is “B-chan” as viewed from the user A. 4 can be specified (guessed). When a new image is stored in the album (image providing site) of the user A, the management server 4 may notify the user B who is a friend of the user A to that effect. The notification may be performed on a screen logged in to the image providing site or on an e-mail or the like. The notification on the image providing site may be displayed until the user B browses the newly arrived image.

(Image saving to portable terminal 6)
Hereinafter, a flow of processing when an image stored in the image providing site is stored in the user's portable terminal 6 will be described with reference to FIGS. 16 and 17. FIG. 16 is a sequence diagram illustrating an example of a flow of a series of processes related to image storage in the mobile terminal 6 in the captured image management system 10. FIG. 17 is a screen transition diagram illustrating an example of a web page displayed on the display unit 605 of the mobile terminal 6 during the image saving process on the mobile terminal 6.

  The user accesses the image providing site in order to log in to the image providing site. As a result, the top page 1001 of the image providing site as shown in FIG. 17A is displayed on the display unit 605 of the mobile terminal 6. The user operates the input unit 604 of the mobile terminal 6 to select a link image 1002 for proceeding with member login processing to the image providing site. The input control unit 612 receives this selection operation and notifies the communication control unit 611 that the link image 1002 has been selected. As a result, the communication control unit 611 reads the user's member ID and password from the storage unit 602, transmits them to the management server 4 via the communication unit 603, and requests the management server 4 to log in to the image providing site (step S21). . Note that when the link image 1002 is selected, the communication control unit 611 may acquire a web page for inputting the user's member ID and password. In this case, the management server 4 may accept input of the member ID and password on the web page.

  The communication control unit 411 of the management server 4 receives this request through the communication unit 403 (step S22). Next, the communication control unit 411 outputs the received member ID and password to the web page creation unit 413 and instructs the web page creation unit 413 to create the member top page 1031.

  The web page creation unit 413 determines whether or not the user is registered as a member on the image providing site by checking the input member ID and password against the member database 422. If it is determined that it is registered, data for generating the member top page 1031 is acquired from the web page database 423. The web page creation unit 413 generates a member top page 1031 as shown in FIG. 17B using the data and outputs the member top page 1031 to the communication control unit 411. The member top page 1031 includes a link image 1032 for displaying a list of user images stored in the image providing site. The communication control unit 411 transmits the member top page 1031 to the mobile terminal 6 through the communication unit 403 (step S23).

  The communication control unit 611 of the mobile terminal 6 receives the member top page 1031 through the communication unit 603. The communication control unit 611 outputs the received member top page 1031 to the display control unit 613. The display control unit 613 outputs the member top page 1031 to the display unit 605. As a result, the member top page 1031 is displayed on the display unit 605 (step S24).

  The user selects the link image 1032 by operating the input unit 604 of the mobile terminal 6 (step S25). The input control unit 612 receives this selection operation and notifies the communication control unit 611 that the link image 1032 has been selected. As a result, the communication control unit 611 transmits a request for providing a web page including a list of user images stored in the image providing site to the management server 4 through the communication unit 603.

  The communication control unit 411 of the management server 4 receives the transmitted request through the communication unit 403 (step S26). Next, the communication control unit 411 instructs the web page creation unit 413 to generate an image list page 1041 including a list of user images stored in the image database 424.

  The web page creation unit 413 acquires data for generating the image list page 1041 from the web page database 423. The web page creation unit 413 uses the data and refers to all the images of the user stored in the image database 424 to generate an image list page 1041 as shown in FIG. 411. A detailed method by which the web page creation unit 413 creates the image list page 1041 will be described later. The image list page 1041 includes a plurality of thumbnail images 1042 that individually represent a plurality of images of the user stored in the image database 424. The communication control unit 411 transmits the image list page 1041 to the mobile terminal 6 through the communication unit 403 (step S27).

  As shown in FIG. 17C, in this embodiment, the image list page 1041 includes a plurality of album pages 1043 that can be switched by tabs. In the example shown in FIG. 17C, the image list page 1041 includes an “all” tab 1044, a “B-chan” tab 1045, a “C-chan” tab 1046, an “you” tab 1047, and a “three or more” tab 1048. A plurality of album pages that can be switched by, etc. are included. FIG. 17C shows a state where the “B-chan” tab 1045 is clicked. On the album page 1043 of the “B-chan” tab 1045, thumbnail images 1042 of a plurality of images including the person B among all the images of the user stored in the image database 424 are displayed. The image may be classified for each person included in the image by the tab, or the image may be classified according to the number of persons included in the image.

  In the image list page, instead of the image list as shown in FIG. 17C, the images displayed every predetermined time may be switched to display images one by one in a slide show format. Further, face images (images of face regions) may be cut out from a predetermined number of images, and a plurality of face images may be combined into one large image and displayed. When an individual face image is selected by clicking or the like from there, a page (“B-chan”) that displays a list of a plurality of images (images generated by the photo sticker creating apparatus 1) including the person of the selected face image. The display may be changed to the album page 1043 of the “tab 1045”.

  The communication control unit 611 of the portable terminal 6 receives the image list page 1041 through the communication unit 603. The communication control unit 611 outputs the received image list page 1041 to the display control unit 613. The display control unit 613 outputs the image list page 1041 to the display unit 605. As a result, the image list page 1041 is displayed on the display unit 605 (step S28).

  The user operates the input unit 604 of the mobile terminal 6 to select a thumbnail image 1042 corresponding to an image to be stored in the mobile terminal 6 among the displayed thumbnail images 1042 (step S29). The input control unit 612 receives this selection operation and notifies the communication control unit 611 that the thumbnail image 1042 has been selected. As a result, the communication control unit 611 transmits a request for providing an image corresponding to the selected thumbnail image 1042 to the management server 4 through the communication unit 603.

  The communication control unit 411 of the management server 4 receives the transmitted request through the communication unit 403 (step S30). Next, the communication control unit 411 instructs the image management unit 414 to output the image of the user requested to be provided from the portable terminal 6 to the communication control unit 411.

  The image management unit 414 acquires the instructed image from the image database 424 and outputs the acquired image to the communication control unit 411. The communication control unit 411 transmits the input image to the mobile terminal 6 through the communication unit 403 (step S31).

  The communication control unit 611 of the portable terminal 6 receives the transmitted image through the communication unit 603 and stores it in the storage unit 602 (step S32). Thus, the image storage process on the mobile terminal 6 is completed.

  In the above description, the configuration in which the mobile terminal 6 acquires an image by logging in to the image providing site has been described as an example. However, in the present embodiment, an image can also be acquired using a seal ID. . For example, the mobile terminal 6 accesses the web page indicated by the URL printed on the sticker paper (or the URL included in the guide mail), and accepts an input operation of the sticker ID by the user, thereby obtaining the image acquisition page. To access.

  At this time, if the user is a paying member, the image acquisition page displays a list of all the images specified by the seal ID and generated during one game so that the user can select. If the user is a free member, only one image of all images specified by the seal ID may be displayed in a list so that the user can select. Further, the image acquisition page may display only one image selected in advance in the photo sticker creating apparatus 1 when the user is a free member. Note that it is preferable that only a list of edited images is displayed on the image acquisition page. For example, when there are captured images for which no edited image is generated, the edited image and the captured image for which the edited image is not generated are displayed. Images may be displayed as a list, or all edited images and captured images may be displayed as a list, and there is no particular limitation.

  The portable terminal 6 acquires an image selected by the user from the management server 4 among the images displayed as a list on the image acquisition page, and stores the acquired image in the storage unit 602. When the user is a paying member, the mobile terminal 6 manages all the images displayed in the list without accepting the user's selection operation for the images displayed in the list on the image acquisition page. And may be stored in the storage unit 602.

(Create image list page)
A detailed method for the web page creation unit 413 to create the image list page 1041 will be described with reference to FIGS. The web page creation unit 413 creates a tab corresponding to the nickname associated with the image of the user A in the image database 424 in the image list page 1041. At this time, the web page creation unit 413 may create tabs for all nicknames, or may create tabs for some nicknames having a high appearance frequency. In addition, when a plurality of nicknames are associated with the same friend as judged from the feature amount, images including the friend may be combined into one tab. Conversely, when the same nickname is associated with a plurality of different friends as judged from the feature amount, the image including each is divided into different tabs. In this case, in order to distinguish a plurality of friends, a subscript (number) is added to the nickname displayed on the tab. Here, the web page creation unit 413 creates a tab on which a nickname is displayed for a nickname whose appearance count is greater than a predetermined threshold. The web page creation unit 413 creates an “all” tab 1044, a “you” tab 1047, a “three or more” tab 1048, and the like. The “all” tab 1044 is a tab for displaying all images of the user A in the image database 424. The “you” tab 1047 is a tab for displaying an image including only the user A himself / herself. The “3 or more” tab 1048 is a tab for displaying an image including three or more persons.

  When the user selects a tab of the image list page 1041 displayed on the mobile terminal 6, the mobile terminal 6 transmits an instruction to specify a tab to be displayed to the management server 4.

  The selection of the tab may be selected by voice recognition. The portable terminal 6 also acquires, from the management server 4, the reading corresponding to, for example, the “B-chan” tab together with the image list page. The application that operates on the mobile terminal 6 may recognize the user's voice and perform processing of selecting the “B-chan” tab when the voice matches the reading corresponding to the “B-chan” tab.

  When the communication control unit 411 of the management server 4 receives an instruction to specify a tab to be displayed from the user's portable terminal 6, the communication control unit 411 instructs the web page creation unit 413 to create an album page 1043 corresponding to the tab to be displayed. Here, a case where “B-chan” tab 1045 for displaying an image including person B is selected will be described.

  The web page creation unit 413 acquires the face feature amount “LL” of the person B from information (see FIG. 18) associated with a part of the images (for example, the image P1) stored in the image database 424. The web page creation unit 413 searches for the feature quantity “LL” of the face of the person B, using the feature quantity stored in the image database 424 as a search target. The web page creation unit 413 compares the feature amount “LL” of the face of the person B with the feature amount stored in the image database 424. Here, the feature quantity “LL” omitted for simplicity represents a plurality of features of the face. For example, the web page creation unit 413 compares a plurality of feature amounts such as the shape of the face and the shape of each part as the feature amount to be compared. If the degree of matching of the feature amounts is equal to or greater than a predetermined value, the web page creation unit 413 determines that the feature amounts of the two persons are similar. As a result, the web page creation unit 413 includes an image including a face person having a feature amount similar to the face feature amount “LL” of the person B among the plurality of images of the user A stored in the image database 424. P1, P3, and P4 are extracted.

  The web page creation unit 413 generates an album page 1043 including thumbnail images of the extracted images P1, P3, and P4. Note that a nickname of a person included in the image may be displayed next to the image. The web page creation unit 413 generates an image list page 1041 including the album page 1043 and outputs it to the communication control unit 411.

  Here, the image stored in the image database 424 is a corrected image obtained by correcting the face in the photo sticker creating apparatus 1, and the associated feature quantity “LL” is the face feature quantity before correction. The features of the face included in the corrected image can vary depending on the type of correction to be applied. For this reason, the corrected feature quantity of the face of the person B included in the corrected image P1 and the corrected feature quantity of the face of the person B included in the other corrected image P3 do not necessarily match (similar). In addition, the corrected feature values of different persons may be similar due to the correction. However, in the present embodiment, since the facial feature amount before correction is associated with each image, a plurality of images having similar facial feature amounts before correction can be extracted. That is, the management server 4 can extract a plurality of images including the same person with high accuracy.

  Even for the same person, different nicknames may be input for each game, such as “B-chan” in the image P1 and “B-san” in the image P3. Even in such a case, the management server 4 can extract a plurality of images including the person B based on the face feature value “LL” before correction. In addition, input of person information (nickname and mail address) may be omitted as in the images P2 and P5. Even when the person information is missing in the image database 424, the management server 4 can extract a plurality of images including the person C based on the face feature value “MM” before correction.

  Even when another tab (“C-chan” tab 1046) is selected, the web page creation unit 413 generates an album page in the same procedure. When the “you” tab 1047 is selected, the web page creation unit 413 extracts an image whose person attribute is only the user A himself / herself. When the “three or more” tab 1048 is selected, the web page creation unit 413 extracts an image (for example, the image P4) in which the feature amounts of three or more persons are associated with one image.

  Further, by searching the image database 424 using the feature amount, it is possible to extract an image under a composite condition such as an image including the person B and not including the person C.

  Note that if the photo sticker creating apparatus 1 transmits the corresponding original photographed image together with the corrected image to the management server 4, the communication amount and the load on the management server 4 increase. In the present embodiment, the photo sticker creating apparatus 1 transmits the corrected image and the feature amount extracted from the corresponding captured image to the management server 4. Since the feature amount data can be smaller than the image data, the communication amount and the load on the management server 4 can be reduced.

  The web page creation unit 413 may be provided with a tab for classifying and displaying images according to the shooting date. The web page creation unit 413 can specify the shooting date and time from the image ID. The web page creation unit 413 classifies and displays images according to the date and time when the image is stored in the management server 4 or the date and time when the user first acquires (views) the image on the image browsing site. Tabs may be provided.

(Image processing)
The management server 4 can perform various image processing using the feature amount associated with the image stored in the image database 424. Here, stamp processing for synthesizing a character string of the person's nickname under the person's face will be described as an example of image processing.

  FIG. 20 shows images before and after stamp processing. The image list page 1041 shown in FIG. 17C is provided with a stamp processing button 1051 corresponding to the thumbnail image 1042. When the user presses (selects) the stamp processing button 1051 on the portable terminal 6, the communication control unit 611 of the portable terminal 6 transmits a request corresponding to the stamp processing button 1051 to the management server 4.

  When the communication control unit 411 of the management server 4 receives a request corresponding to the stamp processing button 1051, the communication control unit 411 outputs the request to the image processing unit 415. The image processing unit 415 acquires from the image database 424 the image P1 corresponding to the stamp processing button 1051 and the feature amount and person information associated with the image. The image P1 is an image to be processed.

  The image processing unit 415 generates a character image TP representing a nickname of personal information (FIG. 20 (a)). The image processing unit 415 specifies the face position FP in the processing target image P1 based on the feature amount (face position or position of each part) (FIG. 20A). The image processing unit 415 generates a stamp image P11 by synthesizing the character image TP representing the nickname corresponding to the person on the lower side (position corresponding to the face) of the person's face in the processing target image P1. . When the processing target image P1 includes a plurality of persons, the image processing unit 415 synthesizes a character image TP representing a corresponding nickname under each face. As a result, a stamp image P11 in which a nickname stamp is pressed on the image is generated.

  When there are a plurality of nicknames for a friend, the most recently saved nickname may be the character image TP, or the user may be allowed to select a nickname to be used for the character image TP. Also, the user's nickname or specific name (kanji name, initials, English character signature, etc.) may be newly input (additionally) by the user, accumulated as personal information of the friend, and stored. Good.

  Note that the character image TP generated from the person information may be arranged based on the positions of parts other than the face. For example, the image processing unit 415 may arrange the character image TP next to the mouth or at the position of the forehead. Further, instead of the character image, an illustration of a predetermined pattern may be synthesized.

  The image processing unit 415 stores the stamp image P11 in the image database 424. In addition, the image processing unit 415 outputs the stamp image P11 to the web page creation unit 413. The web page creation unit 413 creates a web page including the stamp image P11 and outputs the web page to the communication control unit 411. The communication control unit 411 of the management server 4 transmits a web page including the stamp image P11 to the mobile terminal 6. The user can check the stamp image P11 on the portable terminal 6.

  Here, the feature amount associated with the corrected image is a feature amount corresponding to the face of the photographed image before correction. However, since the face position and the like are hardly changed by face correction, image processing can be performed on the corrected image using this feature amount. When the feature amount of the face position is not stored in the image database 424, the image processing unit 415 newly extracts a plurality of face feature amounts included in the corrected image, and uses them as feature amounts associated with the corrected image. You may specify the position of the face which has the similar feature-value. Even if the face is corrected, the feature values before and after correction may be somewhat similar.

  In addition to the stamp processing, the image processing unit 415 performs arbitrary image processing (face concealment processing) such as blurring or image composition on the face area by using the facial feature amount. It is also possible to generate an image with the face part hidden. For a user who does not want to reveal the face of an image to others, the management server 4 can also generate an image with at least a part of the face hidden and use it for publication to others.

  An application that operates on the mobile terminal 6 may perform the above image processing. In this case, the management server 4 transmits the selected image, the feature amount associated with the image, and the person information to the mobile terminal 6. The application of the portable terminal 6 can perform image processing on the image using the received feature amount.

  The photo sticker creating apparatus 1 may perform the above-described image processing in image editing. In this case, at the start of the game, the photo sticker creating apparatus 1 accepts an input such as a member ID from the user and downloads personal information (such as a nickname) from the management server 4. The photo sticker creating apparatus 1 may display stamps (character images TP) corresponding to a plurality of nicknames in a predetermined format at the image editing stage, and allow the user to select a character image TP to be used as a stamp. Furthermore, the photo sticker creating apparatus 1 may identify the user's face in the captured image by comparing the feature amount corresponding to the person information with the feature amount of the captured image. Then, a character image (stamp) such as the acquired nickname may be synthesized with the specified face.

  The management server 4 may transmit the captured image before correction, the corrected image subjected to face correction, and the feature amount to the mobile terminal 6. The mobile terminal 6 may present to the user how much the face changes due to the correction by displaying the captured image and the corrected image. Further, a difference in feature amount between the captured image and the corrected image may be displayed. In addition, a set of a captured image and a corresponding corrected image may be classified and displayed according to a feature amount difference (correction amount). These displays may be performed on the image providing site.

  The management server 4 identifies the face of the user A himself / herself included in the image based on the feature amount of the face included in the image in the image display in the image list or the like, and is the person himself / herself in the face of the user A himself / herself. An image may be displayed with a mark indicating that.

(Enter or change personal information)
In the above description, the configuration is described in which the user inputs personal information in the photo sticker creating apparatus 1 and the personal information is transmitted from the photo sticker creating apparatus 1 to the management server 4. Not only this but the management server 4 can also receive the input of personal information from a user via the portable terminal 6. FIG.

  The image list page 1041 shown in FIG. 17C is provided with a person information input button 1052 corresponding to the thumbnail image 1042. When the user presses (selects) the personal information input button 1052 on the mobile terminal 6, the communication control unit 611 of the mobile terminal 6 transmits a request corresponding to the personal information input button 1052 to the management server 4.

  When receiving the request corresponding to the person information input button 1052, the communication control unit 411 of the management server 4 outputs the request to the web page creation unit 413. The web page creation unit 413 acquires an image corresponding to the person information input button 1052 from the image database 424. The web page creation unit 413 extracts a face image from the image, and creates a web page that allows the user to input person information of the person for each face image. The web page for inputting personal information may have the same screen configuration as that shown in FIG. However, the software keyboard may not be provided on the web page for inputting the personal information.

  The web page creation unit 413 outputs a web page for inputting personal information to the communication control unit 411. The communication control unit 411 of the management server 4 transmits a web page for inputting personal information to the mobile terminal 6.

  The display control unit 613 of the portable terminal 6 displays a web page that allows the display unit 605 to input person information. The user inputs personal information to the mobile terminal 6, and the communication control unit 611 of the mobile terminal 6 transmits the personal information input by the user to the management server 4.

  The communication control unit 411 of the management server 4 outputs the received person information to the image management unit 414. The image management unit 414 stores the person information in the image database 424 in association with the corresponding image. In this way, the user can input and change personal information on the image providing site. In addition to the person information, the management server 4 may accept a change of the person attribute “person / friend”. The person attribute may be set in more detail (in multiple stages). The management server 4 may provide a tab in which images are classified according to person attributes on the image list page.

(Importing personal information)
The management server 4 may import a list of personal information (friend information) from the user's mobile terminal 6. The list of person information is, for example, a list including nicknames (or names) and mail addresses of a plurality of persons who are friends of the user. Such a list of personal information can be generated, for example, by exporting a user's friend list from another SNS (Social Networking Service). The image management unit 414 of the management server 4 compares the list of person information imported from the portable terminal 6 of the user A with the person information (nickname or mail address) stored in the image database 424 for the user A. If both match, the image management unit 414 may change the person attribute to “good friend”, “SNS friend”, or the like. For example, when one piece of person information (email address) matches, the other person information (nickname) may be changed to a list of imported person information.

(Publishing images)
The management server 4 can recognize friend relationships among a plurality of users from the person information. The user can set in advance whether or not to open his / her image list stored in the image database 424 to specific friends or all friends. When the user A permits the friend B to open, the friend B can browse the image list of the user A. However, the management server 4 causes the friend B to view the image after performing face concealment processing on the face of the person G who is not in a friendship relationship with the friend B among the persons included in the image. In the image of user A, friend B can see the face of a person (such as user A) who has a friend relationship with friend B himself. In addition, when person G includes his / her face in an image published by a friend (user A), is it possible to display his / her face without hiding his / her face on friend (friend A) of friend (user A)? It can be set in advance on the image providing site.

  In addition, the user A can also make a setting so that the image list of the user A is made public to all users (including those other than friends) on the image providing site. Further, the user A can set whether or not to display the face of the user A himself / herself included in the image of another person without hiding his face for all users (including those other than friends) on the image providing site. .

  Based on the person information, the management server 4 determines whether each person included in the image is the viewer himself / herself, the friend of the viewer, or other unrelated others. The management server 4 specifies whether or not each person wants to hide his face based on the above public setting and the above determination result. Then, the management server 4 performs face concealment processing on the face of the person who wishes to conceal the face included in the image, and provides it to the viewer. Note that when the user changes the public setting on the image providing site, whether or not other viewers can see the user's face is changed.

[Example of software implementation]
The control blocks (especially the control unit 201, the control unit 401, and the control unit 601) of the photo sticker creating apparatus 1, the management server 4, and the portable terminal 6 are logic circuits (hardware) formed in an integrated circuit (IC chip) or the like. It may be realized by software, or may be realized by software using a CPU (Central Processing Unit).

  In the latter case, the photo sticker creating apparatus 1, the management server 4, and the portable terminal 6 can read a program instruction that is software for realizing each function, and the program and various data can be read by a computer (or CPU). ROM (Read Only Memory) or a storage device (these are referred to as “recording media”), a RAM (Random Access Memory) for expanding the program, and the like. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, the program may be supplied to the computer via any transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  INDUSTRIAL APPLICABILITY The present invention can be widely used as a photographed image management system that stores a photographed image of a user created in a photo sticker creating apparatus in a management server.

1 Photo sticker creation device 4 Management server 6 Mobile terminal (user terminal)
DESCRIPTION OF SYMBOLS 11 Shooting unit 12 Editing part 13 Subsequent customer service part 21 Imaging part 201 Control part 311 Communication control part (2nd transmission means)
312 Shooting control unit (shot image generation means)
313 Edit control unit 313a Feature amount specifying unit (feature amount specifying means)
313b Image generator (corrected image generator)
314 Print control unit 315 Post-service customer control unit 315a Face image extraction unit (face image extraction means)
315b Face image presentation unit (face image presentation means)
315c Person information input unit (person information input means)
401 control unit 402 storage unit 403 communication unit 411 communication control unit (feature amount acquisition unit, corrected image acquisition unit, person information acquisition unit, first transmission unit)
412 Mail creation unit 413 Web page creation unit (image extraction means)
414 Image management unit (image management means, person determination means)
415 Image processing unit (image processing means)
422 Member database (user information management means)
424 Image Database (Database)
601 Control unit 602 Storage unit 603 Communication unit 604 Input unit 605 Display unit 611 Communication control unit 612 Input control unit 613 Display control unit

Claims (14)

A management server connected to the photo sticker creation device via a network,
A feature amount acquisition means for acquiring a feature amount of a person's face included in a photographed image photographed by the photo sticker creating apparatus;
Corrected image acquisition means for acquiring a corrected image generated by correcting the face included in the photographed image in the photo sticker creating apparatus;
Image management means for associating the corrected image with the feature quantity and storing them in a database ;
Personal information acquisition means for acquiring personal information about the person,
The management server characterized in that the image management means stores the person information in association with the feature quantity in a database . The image management means stores the corrected image in the database in association with the user who is the registrant of the corrected image,
User information management means for managing user information related to the user;
Person determination means for determining whether or not the feature amount is a feature amount of the face of the user who is the registrant of the corrected image from the person information associated with the feature amount and the user information. Prepared,
It said image management means, the management server according to claim 1, characterized in that the determination result of the human being determining means, to be stored in the database in association with the feature value. The management server according to claim 2 , further comprising: an image extraction unit that extracts the corrected image including a specific person who is not the user from the plurality of corrected images associated with the user. The image extraction unit compares the feature amount determined not to be the feature amount of the user's face with the feature amount of the face included in the corrected image, so that the specific person is included in the corrected image. The management server according to claim 3 , wherein it is determined whether or not it is included. In the corrected image, the position of the face corresponding to the feature amount is specified using the feature amount associated with the corrected image, and predetermined image processing is performed on the face specified in the corrected image. the management server according to claim 1, any one of 4, characterized in that it comprises an image processing means for performing. The management server according to claim 5 , wherein the image processing unit performs the image processing for synthesizing the person information associated with the feature amount at a position corresponding to the face. The management server according to any one of claims 1 to 6 , further comprising first transmission means for transmitting the corrected image and the feature amount to a user terminal. The person information acquisition means, the management server users according to the personal information input to the photo sticker creation device, to any one of claims 1 to 4, characterized in that to get from the photo sticker creation device . The management server according to any one of claims 1 to 4 , wherein the person information acquisition unit acquires the person information from a user terminal of the user. The feature amount is at least one of the position of the face, the size of the face, the shape of the face, the position of the face part, the size of the part, and the shape of the part. The management server according to any one of claims 1 to 9 . The control program for functioning a computer as a management server as described in any one of Claims 1-10 , Comprising: The control program for functioning a computer as said each means. The computer-readable recording medium which recorded the control program of Claim 11 . Photographing image generation means for generating a photographed image of a person by photographing;
A feature amount specifying means for specifying a feature amount of a person's face included in the photographed image;
Corrected image generation means for generating a corrected image by correcting the face included in the captured image;
Second transmission means for associating the corrected image and the feature quantity with each other and transmitting the database to the database server ;
Face image extraction means for extracting a face image included in the photographed image or the corrected image;
Face image presenting means for presenting the face image to the user;
A photo sticker creating apparatus comprising: personal information input means for receiving input of personal information related to a person of the face image from the user . A feature amount identifying step for identifying a feature amount of a person's face included in a photographed image photographed by the photo sticker creating device;
A corrected image generating step of generating a corrected image by correcting a face included in the photographed image in the photo sticker creating apparatus;
A person information acquisition step of acquiring person information related to the person;
An image management method comprising: an image management step of storing the corrected image, the feature amount, and the person information in a database in association with each other.

Images