JP2012094048A - Clothing simulation system, clothing simulation method and clothing simulation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To execute processing, using a common engine program, corresponding to each of utilization situations of a Web application executed on Web or stand-alone software provided in a shop or the like.SOLUTION: A clothing simulation system is provided in which clothing is virtually fit on a screen. A user image Y photographing a user is captured, a sample image S of a person to become a sample is captured, and a class of an acquisition route of the user image Y is set. In accordance with the setting, either the user image Y or the sample image S is selected as a reference image and in a reference color extraction section 113, a reference color is extracted from the selected reference image. In an image combination section 114, an image which is not selected as the reference image is defined as an image to be processed, the image to be processed is processed on the basis of the reference color, and the processed image to be processed is combined with the reference image, thereby producing a combination image.

Description

  The present invention relates to a costume simulation system, a costume simulation method, and a costume simulation program used for a costume simulation in which a costume such as a wedding dress is virtually tried on a screen.

  Traditionally, brides and grooms planning weddings need to decide on a ceremony venue, date of the ceremony, costumes, cooking, and so on. Therefore, the bride and groom went to the ceremony hall many times to make a decision, and had a long meeting with the planner, which required a lot of time and effort. In particular, wedding dresses and other costumes to be worn on the wedding day have to meet a wide variety of customer preferences, and the number of costumes has increased significantly. Had the problem of taking a lot of time.

  Therefore, a wedding dress try-on simulation system using image processing has been proposed that displays the atmosphere of a try-on as an image even if the customer does not actually try on the costume (for example, Non-Patent Document 1). In this system, a dress is selected based on the user's body shape data, the current user's face is incorporated into the dress image, and the user's dress appearance is displayed on the screen, so that a wedding dress suitable for the user is selected. can do.

  However, although the technique disclosed in Patent Document 1 can select a wedding dress that matches the user's body shape, it is a technique that simply synthesizes a face photo with the face part of the dress image, so the user's face is synthesized. When viewing the recorded image data, the skin color of the user's face differs from the skin color of the person who became the model of the dress image part, and there is a problem that the atmosphere as if the user actually wore a dress could not be produced there were.

  In order to solve such a problem, there is a technique of processing so as not to feel uncomfortable when combining images (for example, Patent Document 2). In the technique disclosed in Patent Document 2, when a face photograph and an image of a wig are combined, the vicinity of the hairline is corrected in accordance with the user's skin color so that the composite photograph does not feel uncomfortable.

JP 2003-147619 A JP 2003-8881 A

  However, since the image synthesized by the technique disclosed in Patent Document 2 is always corrected on the basis of the user's skin color, it cannot cope with the user's needs that change depending on the usage situation. was there. In other words, situations where the user uses the fitting simulation system include, for example, when selecting a ceremony hall or a costume company to check in advance what services, facilities, and costume variations exist, Etc., and may be used to select costumes in earnest.

  In the former case, a service for searching for information on the Web from the personal computer at home through the Internet and performing a trial fitting simulation of a costume provided on a Web page may be used. In this try-on simulation on the Web, a dedicated Web application is provided from a Web server, and when using this Web application, the user captures and uploads himself / herself because he is at home, and uses the image for the simulation. At this time, since the user himself / herself takes a picture, image data with poor shooting conditions is often used.

  By the way, in this Web application, when using image data prepared by the user himself / herself at home, if a synthesis method that matches the skin color of the user's face is used, there is a gap between the user's imagination and the atmosphere in his / her wedding dress. There was a problem that accurate simulation could not be performed.

  On the other hand, in the latter case, a try-on simulation is executed in a store using a stand-alone type application together with a coordinator. In this case, since professional photographers take pictures with dedicated shooting equipment etc., the photos used are also good, and by actually correcting the composite photo according to your body shape and skin color, The atmosphere when worn can be expressed, and total coordination of costumes and makeup according to the user's request becomes possible.

  Therefore, the present invention solves the above-described problems, and in a costume simulation in which a costume such as a wedding dress is virtually tried on a screen, a web application executed on the web, a store, etc. It is possible to provide a costume simulation system, a costume simulation method, and a costume simulation program that can execute processing corresponding to each usage situation, such as the stand-alone software provided in, using a common engine program. Objective.

  In order to solve the above problems, the present invention provides a costume simulation system for virtually trying on a costume on a screen for a user, a user image acquisition unit that acquires a user image obtained by photographing the user, and a sample A sample image storage unit that stores a plurality of sample images of a person, a sample image acquisition unit that acquires an image of an arbitrary sample from the sample image storage unit, and an acquisition path setting unit that sets the type of acquisition path of a user image A reference color extraction unit that selects either a user image or a sample image as a reference image and extracts a reference color from the selected reference image according to the setting in the acquisition path setting unit, and an image that is not selected as the reference image Is processed image, the processed image is processed based on the reference color, and the processed image is processed as the reference image Synthesized, it has an image synthesizing unit for generating a composite image, based on information about the user, and an image processing unit for processing the composite image.

Another invention is a costume simulation method that allows a user to virtually try on a costume on a screen,
(1) An image acquisition step of acquiring a user image of a user and acquiring a sample image of a person as a sample;
(2) A reference color extraction step of setting a type of user image acquisition path, selecting either a user image or a sample image as a reference image according to this setting, and extracting a reference color from the selected reference image;
(3) An image composition that generates an image that has not been selected as a reference image as a processed image, processes the processed image based on the reference color, and combines the processed image with the reference image. Steps,
(4) An image processing step for processing the composite image based on information on the user.

  According to such an invention, either the user image or the sample image is selected as the reference image, and the unselected image is processed based on the reference color of the reference image. Since the image is combined with the image, for example, the combined image can generate a combined image with a uniform skin color and no sense of incongruity.

  In particular, in the present invention, the selection of the reference image can be determined according to the acquisition path of the user image. For example, when the image is uploaded from the user's personal computer through the Internet, the reference image is used as the sample image. By generating a composite image, even a user image with poor shooting conditions can be combined with a wedding image with a good skin color, and a fitting simulation can be performed without being affected by the shooting environment of the user image Can do. On the other hand, in the case of an image taken and acquired in a store, a composite image can be generated in accordance with the user's own skin color by generating a composite image using the reference image as a user image. As a result, it is possible to express the atmosphere when it is actually worn, and it is possible to make a total coordination of more specific costumes and makeup.

  Further, according to the present invention, it is possible to perform a simulation according to the use environment while using a common module program only by switching a reference image selection method according to a user image acquisition path. Development costs can be significantly reduced.

  In the above invention, the image processing unit further includes a function of adding a partial image as an additional image to the synthesized image and synthesizing the image, and performing a deformation process including any of the size, aspect ratio, and shape of the image. It is preferable to perform the process on the composite image and deform the additional image according to the parameters used in the deformation process, and combine the additional image with the composite image. In this case, since it has a function of adding small items such as necklaces, tiaras, and gloves used for the wedding ceremony to the composite photo, various wedding dress appearances can be simulated. In particular, in the present invention, when an additional image is added, a deformation process including any of the size, aspect ratio, and shape of the image can be performed. An image that looks like a try-on can be generated.

  In the above invention, the image processing unit may further include a function of performing makeup processing including color change processing of any of coloring, color tone change, lightness change, or image synthesis on a part of the composite image. preferable. In this case, even if the composite image is the reference color of either the sample image or the user image, the makeup processing parameters can be set in accordance with the skin color. The makeup process can be made possible by following different skin colors depending on the use of the stand-alone type or the use on the Web.

Furthermore, another invention is a costume simulation program that allows a user to virtually try on a costume on a screen, and
(1) An image acquisition step of acquiring a user image of a user and acquiring a sample image of a person as a sample;
(2) A reference color extraction step of setting a type of user image acquisition path, selecting either a user image or a sample image as a reference image according to this setting, and extracting a reference color from the selected reference image;
An image that is not selected as a reference image is set as a processed image, and based on the reference color, (3) the processed image is processed, and the processed processed image is combined with the reference image to generate a combined image. Steps,
(4) Based on information about the user, a process including an image processing step for processing the composite image is executed.

  According to such an invention, by using a costume simulation program, a special processing device is not required, and the various information processing devices are the above-described costume simulation devices, and the costume simulation method is realized in various information processing devices. can do. That is, by installing this costume simulation program on a general-purpose computer or IC chip and executing it on the CPU, a costume simulation apparatus having the above-described functions can be configured, and a costume simulation method can be easily constructed. it can.

  This program can be distributed through a communication line, for example, and can be transferred as a package application that operates on a stand-alone computer. Such a costume simulation program can be recorded on a computer-readable recording medium, and the above-described system and method can be implemented using a general-purpose computer or a dedicated computer. Can be easily stored, transported and installed.

  As described above, according to the present invention, when providing a costume simulation service that allows a user to virtually try on a costume such as a wedding dress on the screen, a web application executed on the web, Processing corresponding to each use situation, such as stand-alone software provided in a store or the like, can be executed using a common engine program. As a result, according to the present invention, it is possible to perform a simulation according to the usage environment while using a common module program by simply switching the reference image selection method according to the user image acquisition path. Development costs can be greatly reduced.

It is a conceptual diagram which shows the whole structure of the costume simulation system which concerns on embodiment. (A) is a block diagram showing an internal structure of each device constituting the costume simulation system that operates as a stand-alone information processing terminal according to the present embodiment, and (b) is a Web according to the present embodiment. It is a block diagram which shows the internal structure of each apparatus which comprises a costume simulation system which operate | moves as a server. It is a block diagram which shows the internal structure of the information processing terminal which concerns on embodiment. It is explanatory drawing which shows typically operation | movement of the image synthesis | combination of the costume simulation system which concerns on embodiment. It is explanatory drawing which shows typically the operation | movement of the color correction of the costume simulation system which concerns on embodiment. It is explanatory drawing which shows the image processing screen displayed on the window which concerns on embodiment. It is explanatory drawing which shows the additional image addition screen displayed on the window which concerns on embodiment. It is explanatory drawing which shows the makeup process screen displayed on the window which concerns on embodiment. It is a flowchart figure which shows the costume simulation method which concerns on embodiment. It is explanatory drawing which shows the image composition screen displayed on the window which concerns on embodiment. It is a perspective view which shows the structure of the skin color measuring sensor which concerns on embodiment.

  Hereinafter, an embodiment of a costume simulation system according to the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a conceptual diagram showing the overall configuration of a costume simulation system according to this embodiment.

(Overall system configuration)
As shown in FIG. 1, the costume simulation system according to the present embodiment is a system that can be used in a simulation service that allows a user 7 to virtually try on a costume such as a wedding dress. In this embodiment, by installing the costume simulation program of the present invention created in a predetermined language on a computer that is various information processing apparatuses 10, various information processing apparatuses 10 are operated as costume simulation apparatuses, and costumes are performed. A simulation method can be realized.

  Specifically, in this embodiment, a costume simulation program having a single engine program is installed in each of various information processing apparatuses 10 (10a and 10b) operating in different usage environments, and users in the usage environments are installed. By switching the setting of the image acquisition route, it is possible to construct respective costume simulation systems corresponding to different usage patterns.

  More specifically, when the costume simulation program of the present invention is installed in the store-side apparatus 10a, which is a general-purpose computer installed in the store 9, as shown in FIG. Works standalone, stand-alone and used as a costume simulation application for stores. In this store 9, a user 7a who has come to receive wedding coordination is photographed with a photographing device in the store, and the photographed image data is combined with a sample image and displayed on the display unit 2a.

  On the other hand, when the costume simulation program of the present invention is installed in the information processing terminal 10b connected to the communication network 5, the costume simulation program operates on the server as a web application, and the virtual costume simulation service on the web Used for use. In this virtual costume simulation service, access from the user terminal 6 used by a plurality of users 7b connected to the communication network 5 is accepted, and an image prepared by the user is uploaded, and the uploaded image data is stored. The sample image is synthesized and displayed on the display unit 2b of the user.

(Each device and its internal structure)
Next, each device constituting these costume simulation systems and the internal structure of each device will be described. FIG. 2A is a block diagram showing an internal structure of a costume simulation system that operates as a stand-alone information processing terminal according to the present embodiment, and FIG. 2B operates as a Web server according to the present embodiment. It is a block diagram which shows the internal structure of a costume simulation system. FIG. 11 is a perspective view showing the configuration of the skin color measurement sensor according to this embodiment. The “module” used in the description refers to a functional unit that is configured by hardware such as an apparatus or a device, software having the function, or a combination thereof, and achieves a predetermined operation. .

(1) Stand-alone type
First, a costume simulation system in a stand-alone information processing terminal will be described. In this case, as shown to Fig.2 (a), the store side apparatus 10a which is the information processing apparatus installed in the shop 9 and the camera 3a are provided.

  The store 9 is a wedding hall that provides facilities for weddings such as weddings and receptions, a store that sells or rents outfits such as wedding dresses, and includes facilities for photographing the user 7a that visits the store. . The camera 3a is an imaging device that captures a still image or a moving image. The camera 3a includes a lens and a fixed imaging device such as a CCD or a CMOS, and incident light representing an object image is formed on a light receiving surface of the CCD. Then, the image data is input to the store side apparatus 10a via the camera signal processing circuit and the A / D converter.

  The store side device 10a used in the store 9 is provided with a skin color measuring sensor 4 that directly measures the user's skin color. The skin color measuring sensor 4 is, for example, a spectrocolorimeter that makes light of a specific wavelength incident on the skin surface, receives the reflected light, and measures the reflectance of the skin at the specific wavelength. Specifically, as shown in FIG. 11, the skin color measurement sensor 4 is a hand-held spectrocolorimeter, and includes a lens 41 provided at the tip, a white LED provided in the main body, and an RGB color sensor. And have. Then, the skin color measurement sensor 4 uses the white LED as a light source, irradiates the skin with white light, and receives the reflected light from the skin surface via the lens 41. The received reflected light is measured by an RGB detection color sensor inside the main body, and is obtained as skin color data by color conversion. In addition, the skin color sensor 4 may be added with a function of measuring the melanin content, the hemoglobin content, and the like in addition to the spectral reflectance data obtained by the spectrocolorimeter and acquiring the skin color data.

  The store side device 10a is a user terminal having a calculation processing function by a CPU and a communication processing function by a communication interface, and can be realized by a general-purpose computer such as a personal computer or a dedicated device specialized in function. The store-side end device 10a includes a storage unit 120, an application execution unit 110, a display unit 2a, and an input / output interface 101.

  The display unit 2a is, for example, a display device such as a liquid crystal display, and includes a display device such as a liquid crystal display and a display drive circuit for the display, and displays images, characters, and the like on the display.

  The storage unit 120 is a memory device or the like that stores various data, and stores application data, image data displayed in each functional module, personal information such as a user name and address, and the like. The storage unit 120 includes a sample image storage unit 120a.

  The sample image storage unit 120a is a module that stores a plurality of sample images of a person to be a sample and a plurality of sample images of accessories such as wedding dresses, gloves, and necklaces. In the present embodiment, the sample image of a person has a plurality of sample images having different face shapes such as a round face, a long face, or a square face. This sample image also includes variations with different skin color types, such as skin color brightness and hue, for each face shape, and the face shape pattern and the skin color type are stored in association with each other. Has been.

  The application execution unit 110 is a module that executes an application such as browser software or an image reproduction application on the OS. In the present embodiment, the application execution unit 110 has a function of synthesizing the acquired user image and sample image.

 The input interface 101a is a device that inputs user operations such as operation buttons, a touch panel, and a jog dial, and the output interface 101b is a device that outputs video and sound, such as a display and a speaker. In this embodiment, the camera 3a and the skin color measurement sensor 4 are connected via the input interface 101a, and the image data photographed by the camera 3a and the skin color data measured by the skin color measurement sensor 4 are stored on the store side. Input to the device 10a.

(2) Web-type costume simulation system On the other hand, the costume simulation system used on the Web has a server device 10b connected to the communication network 5 and a user terminal 6 as shown in FIG. The user terminal 6 is equipped with a camera 3b.

  The communication network 5 is a distributed type constructed by mutually connecting various communication lines (optical lines such as FTTH, public lines such as ADSL lines, dedicated lines, and wireless communication networks) using the communication protocol TCP / IP. This IP network includes LANs such as intranets (intra-company networks) and home networks based on 10BASE-T, 100BASE-TX, and the like.

  The user terminal 6 is a user terminal provided with a calculation processing function by a CPU and a communication processing function by a communication interface. The user terminal 6 can be realized by a general-purpose computer such as a personal computer or a dedicated device specialized in function. Computers, PDAs (Personal Digital Assistance), mobile phones, etc. are included.

  The user terminal 6 includes a communication interface 62, an input interface 61a, an output interface 61b, a browser unit 63, and a display unit 2b.

  The browser unit 63 is software that accesses the Internet, which is the communication network 5, and transmits and receives data. The browser unit 63 is application software for browsing Web pages, downloads HTML (HyperText Markup Language) files, image files, music files, and the like from the Internet, analyzes the layout, and displays / reproduces them. It is also possible for a user to transmit data to a Web server using a form, or to operate application software described in JavaScript, Flash, Java (registered trademark), or the like.

  The browser unit 63 can output video and audio through the output interface 61b, and an operation signal from an operation device such as a mouse or a keyboard is input as a user operation through the input interface 61a. The browser unit 63 displays a Web page and enables user operations such as clicking on the Web page and inputting characters.

  The communication interface 62 is a communication interface for performing data communication, and has a function of performing non-contact communication by wireless or the like, and contact (wired) communication by a cable, an adapter unit, or the like. The input interface 61a is a device that inputs user operations such as operation buttons, a touch panel, and a jog dial, and the output interface 61b is a device that outputs video and sound, such as a display and a speaker. In the present embodiment, the camera 3b is connected via the input interface 61a, and image data captured by the camera 3b is input to the user terminal 6.

  The display unit 2b is, for example, a display device such as a liquid crystal display, and includes a display device such as a liquid crystal display and a display drive circuit for the display, and displays images, characters, and the like on the display. The camera 3b is an imaging device that captures a still image or a moving image. The camera 3b includes a lens and a fixed imaging element such as a CCD or a CMOS, and incident light representing an object image is formed on a light receiving surface of the CCD. Then, the image data is transmitted to the user terminal 6 via the camera signal processing circuit and the A / D converter.

  On the other hand, the server device 10b is an information processing terminal having an arithmetic processing function by a CPU and a communication processing function by a communication interface, and can be realized by a general-purpose computer such as a personal computer or a dedicated device with specialized functions. it can. In particular, in the present embodiment, the server apparatus 10b is a server computer that transmits information such as an HTML (HyperText Markup Language) file, an image file, or a music file in a document system such as WWW (World Wide Web) or a function thereof. It has a Web server function realized by software having a network, stores information such as HTML documents and images, and in response to a request from client software such as a Web browser, these are transmitted through an IP network such as the Internet. Send information.

  In the present embodiment, the server device 10 b includes a storage unit 120, an application execution unit 110, and a communication interface 102.

  The storage unit 120 is a memory device or the like that stores various data, and stores application data, image data displayed in each functional module, personal information such as a user name and address, and the like. The storage unit 120 includes a sample image storage unit 120a.

  The sample image storage unit 120a stores a plurality of sample images of a person as a sample and sample images of accessories such as wedding dresses, gloves, and necklaces. Further, in the present embodiment, the sample image of a person includes a plurality of sample images having different face shapes and skin color types such as a round face, a long face, or a square face. That is, this sample image includes variations with different skin color types such as skin color brightness and hue for each face shape, and the face shape pattern and the skin color type are stored in association with each other. Has been.

  The application execution unit 110 is a module that executes applications such as browser software and image reproduction applications on the OS. In the present embodiment, the application execution unit 110 executes image data selection processing and composition / processing. The communication interface 102 is a communication interface for performing data communication, and has a function of performing non-contact communication by wireless or the like, and contact (wired) communication by a cable, an adapter unit, or the like.

(Internal structure of application execution unit 110)
Next, the internal structure of the application execution unit 110 in the store side device 10a and the server device 10b will be described. FIG. 3 is a block diagram showing the internal structure of the store side device 10a and the server device 10b, and FIG. 4 is an explanatory view schematically showing the image composition operation of the costume simulation system according to the present embodiment. FIG. 5 is an explanatory diagram schematically showing the color correction operation of the costume simulation system. FIG. 6 is an explanatory view showing an image processing screen displayed on the window according to the present embodiment, and FIG. 7 is an explanatory view showing an additional image addition screen displayed on the window according to the present embodiment. FIG. 8 is an explanatory diagram showing a makeup processing screen displayed on the window according to the present embodiment.

  Specifically, the application execution unit 110 is an arithmetic processing unit such as a CPU, and other than the CPU, a processor such as a DSP (Digital Signal Processor), a memory, and other hardware such as an electronic circuit, or a function thereof. It is an arithmetic module configured by software such as a program or a combination thereof. In the present embodiment, the application execution unit 110 reads and executes a costume simulation program to virtually construct various functional modules on the CPU, and with each constructed functional module, processing related to image data, Various processes are performed for operation control of each unit and user operation.

  The application execution unit 110 includes an operation signal acquisition unit 103, a user image acquisition unit 111, and a sample image acquisition unit 115 as modules related to selection of image data.

  The operation signal acquisition unit 103 is a module that receives an operation signal from a user, analyzes the received operation signal, and causes another module to execute processing according to the user operation. In particular, in the present embodiment, the operation signal acquisition unit 103 receives signals related to user operations such as image composition processing and acquisition path type setting, and causes each module of the application execution unit 110 to execute.

  The user image acquisition unit 111 is a module that acquires a user image obtained by photographing a user. In the present embodiment, the user image acquisition unit 111 acquires a user image via the input interface 101a or the communication interface 102. When the user's skin color is measured by the skin color measuring sensor 4, the measured user's skin color data is also input.

  The sample image acquisition unit 115 is a module that acquires an image of an arbitrary sample from the sample image storage unit 120a. In the present embodiment, the sample image acquisition unit 115 analyzes a face outline portion in the acquired user image using a face recognition technique, and determines whether the shape of the user's face is a round face, a long face, or a square face. And so on. Then, sample images belonging to the determined category are extracted from the sample image storage unit 120a. Here, the face recognition technology is a technology that extracts the contour part of the face as a feature point, and is based on eigenface using principal component analysis, linear discriminant analysis, elastic bunch graph matching, hidden Markov model, neuron motivation Various techniques such as dynamic link matching can be used. At this time, the sample image acquisition unit 115 also analyzes the skin color of the user image and acquires a sample image having a skin color that approximates the skin color of the user image from the determined face shape category.

  The application execution unit 110 also includes an acquisition path setting unit 112, a reference color extraction unit 113, an image synthesis unit 114, an image processing unit 116, and an output signal generation as modules related to image data synthesis / processing. Part 104.

  The acquisition path setting unit 112 is a module for setting the type of acquisition path of the user image, and is set manually or automatically with respect to the costume simulation program, so that the reference image when the images are combined is set as the user image. Or whether to use a sample image. Here, in this embodiment, when the computer is used as the store side device 10a, the acquisition route is set as the peripheral connection device route, and when the computer is used as the server device 10b on the Web, the acquisition route is set. Set as Web path. Note that the acquisition route type setting in the acquisition route setting unit 112 is set manually by, for example, requesting the user to select when installing the costume simulation program, or by an arbitrary selection operation by the user after installation. Alternatively, the function of the information processing apparatus 10 to be installed (such as OS, driver, firmware type, presence of external device or plug-in) is automatically detected and detected. You may make it set automatically according to a function.

  The reference color extraction unit 113 is a module that selects either a user image or a sample image as a reference image according to the setting in the acquisition path setting unit 112 and extracts a reference color from the selected reference image. In this embodiment, the reference color extraction unit 113 uses the user image as the reference image and the skin color of the user image as the reference color when the setting in the acquisition route setting unit 112 is set as the peripheral connection device route. Or the skin color data measured by the skin color measuring sensor 4 is extracted as a reference color. On the other hand, the reference color extraction unit 113 extracts the sample image as the reference image and extracts the skin color of the sample image as the reference color when the setting by the acquisition route setting unit 112 is set as the Web route.

  The image synthesizing unit 114 is a module that uses an image that is not selected as a reference image as a processed image, processes the processed image based on the reference color, combines the processed image with the reference image, and generates a combined image. is there. Specifically, in the process of synthesizing the image, as shown in FIG. 4, the face portion Y1 of the user image is analyzed for eyes, nose, mouth, and contour using face recognition technology, and the eyes, nose, The center part P of the face including the mouth is determined. Then, according to the face portion S1 of the sample image to be synthesized, the user image Y is enlarged / reduced, the inclination of the user image Y is corrected, and the face portion Y1 to be synthesized is cut out. Then, the face part Y1 of the cut-out user image is fitted into the face part S1 of the sample image.

  Further, the image composition unit 114 executes processing for correcting the color of the image. In this embodiment, the color correction of the composite image is performed by determining whether the reference image is a user image or a sample image, using the image not selected as the reference image as a processed image, and processing the processed image based on the reference color. . Specifically, for example, when the reference image is a user image, the image composition unit 114 synthesizes the face portion Y1 of the user image with the face portion S1 of the sample image as shown in FIG. At the same time, the peripheral portion S2 of the face of the sample image and the upper body portion S3 are corrected to the skin color of the face portion Y1 of the user image.

  On the other hand, when the reference image matches the sample image, as shown in FIG. 5B, the face portion Y1 of the user image is combined with the face portion S1 of the sample image, and the face portion Y1 of the user image is sampled. The skin color of the peripheral portion S2 of the face of the image and the upper body portion S3 is corrected. In this embodiment, the color correction is processed after the reference image and the processed image are combined. However, the present invention is not limited to this, and the reference image and the processed image are corrected after correcting the color of the processed image. The composition which synthesizes with an image may be sufficient.

  The image processing unit 116 is a module that processes a composite image, and has a function of processing the composite image based on information about the user. Specifically, as shown in FIG. 6 (a), a parameter screen 240 on which each item 243 of information on the body shape and the like is displayed is displayed, and the user adjusts the slider 241 of each item 243 or input value column By inputting a numerical value in 242, the transformation processing including any of the image size, aspect ratio, and shape is performed on the composite image, and the user displayed on the entire screen 210 as shown in FIG. 6B. The overall figure 211 is changed.

  In addition, the image processing unit 116 has a function of adding a partial image as an additional image to the combined image and combining it. Specifically, as shown in FIG. 7, on the parts screen 220 displayed in the window 200, an icon image 221 such as a hairstyle, a dress, and an accessory is displayed, and any one of the icon images 221 is selected. In addition, an icon image 221 of a dress or accessory is added to the overall appearance 211 of the user displayed on the entire screen 210 to be synthesized. At this time, the image processing unit 116 has a function of deforming the additional image in accordance with the parameters used for the deformation process and combining the additional image with the composite image. That is, for example, when information about the user is changed and the size of the user's entire figure is changed, the dress image and the accessory image that are additional images are also changed and combined at the same ratio. It has become.

  Furthermore, the image processing unit 116 has a function of performing makeup processing including color change processing of any of coloring, color tone change, lightness change, or image synthesis on a part of the composite image. Specifically, in this makeup process, as shown in FIG. 8, a makeup process screen 250 and an enlarged face screen 260 are displayed in the window 200, and each slider displayed on the makeup process screen 250 is displayed. By changing 251 or selecting an icon image 252 such as false eyelashes, the skin color of the enlarged screen 260 of the face portion is corrected by RGB, or an icon image is added.

  At this time, the image processing unit 116 sets parameters related to the color change processing based on the type of acquisition path, information on the reference color, and information on processing on the processed image. The makeup data that has been made up can be stored in the storage unit 120.

  The output signal generation unit 104 converts the user image acquired by the user image acquisition unit 111, the image synthesis process executed by the application execution unit 110, and the execution result of the processing process into a video signal and an audio signal, and a display unit This is a module that generates signals that can be output by 2a and 2b and speakers. In this embodiment, not only the composite image of the sample image and the user image is displayed, but the composite photo and the photo of the ceremony hall can be combined to generate and display a wedding landscape image. The atmosphere can be reproduced.

(Costume simulation program)
The costume simulation system and the costume simulation method according to the present embodiment can be realized by executing a program described in a predetermined language on a computer. That is, by installing this program on a computer such as a user terminal or a Web server or an IC chip and executing it on the CPU, a system having the above-described functions can be easily constructed. This program can be distributed through a communication line, for example, and can be transferred as a package application that operates on a stand-alone computer.

  Such a program can be recorded on a recording medium readable by a general-purpose computer. Specifically, it can be recorded on various recording media such as a RAM card in addition to a magnetic recording medium such as a flexible disk or a cassette tape, or an optical disk such as a CD-ROM or DVD-ROM. The present embodiment is characterized in that a link can be provided for content in a non-writable CD-ROM or DVD-ROM.

  According to the computer-readable recording medium in which the program is recorded, the above-described system and method can be implemented using a general-purpose computer or a dedicated computer, and the program can be stored, transported, and installed. It can be done easily.

  According to such an invention, by using a costume simulation program, a special processing device is not required, and the various information processing devices are the above-described costume simulation devices, and the costume simulation method is realized in various information processing devices. can do.

(Costume simulation method)
The costume simulation method of the present invention can be implemented by operating the costume simulation system or the costume simulation program having the above-described configuration. FIG. 9 is a flowchart illustrating the operation of the costume simulation system according to the present embodiment, and FIG. 10 is an explanatory diagram illustrating an image composition screen displayed on the window according to the present embodiment.

  First, the type of user image acquisition path is set (S101). In the present embodiment, when the costume simulation program is installed in the store side device 10a, the setting is set as a peripheral connection device route, and when the costume simulation program is installed in the server device 10b, the setting is set as a Web route. Set. Next, the user is photographed by the cameras 3a and 3b, and the user image data is transmitted to the store side device 10a or the server device 10b via the input interface 101a or the communication interface 102.

  The store-side device 10a or the server device 10b acquires a user image obtained by photographing the user (S102) and also acquires a sample image of a person as a sample (S103). At this time, the sample image acquisition unit 115 analyzes the face contour portion in the acquired user image using a face recognition technique, and determines whether the shape of the user's face is a round face, a long face, a square face, or the like. To belong to one of the categories. Furthermore, the sample image acquisition unit 115 also analyzes the skin color of the user image, and determines a sample image having a skin color that approximates the skin color of the user image from the determined face shape category. Then, sample images of face shapes belonging to the determined category are extracted from the sample image storage unit 120a, and sample images that approximate the shape of the user's face and the type of skin color are displayed as shown in FIG. .

  Further, the store side device 10a or the server device 10b selects either the user image or the sample image as the reference image according to the type setting (S104), and extracts the reference color from the selected reference image (S105). Specifically, in the store-side apparatus 10a, the setting in the acquisition route setting unit 112 is set as the peripheral connection device route, and the reference color extraction unit 113 uses the user image as a reference image and sets the skin color of the user image. Extracted as a reference color or skin color data measured by the skin color measurement sensor 4 is extracted as a reference color. On the other hand, in the server device 10b, the setting in the acquisition route setting unit 112 is set as the Web route, and the reference color extraction unit 113 extracts the sample image as the reference image and extracts the skin color of the sample image as the reference color. .

  As shown in FIG. 10, when the sample image 231 and the user image 232 are displayed on the face synthesis screen 230 displayed in the window 200 and the user clicks the MIX button 235, the image synthesis unit 114 is displayed. Uses the image not selected as the reference image as a processed image, processes the processed image based on the reference color, and combines the processed image with the reference image to generate a combined image (S106). .

  More specifically, first, the image composition unit 114 analyzes the face portion Y1 of the user image using the face recognition technique and the feature point extraction technique, and as shown in FIG. 4, the center of the face including the eyes, nose, and mouth. Part P is determined. Then, according to the size of the face of the sample image to be synthesized, the user image is enlarged / reduced, the inclination of the user image Y is corrected, and the face portion Y1 to be synthesized is cut out. Then, the face part Y1 of the cut-out user image is fitted into the face part S1 of the sample image.

  Next, the color of the image is corrected. The image composition unit 114 first determines the reference image selected by the reference color extraction unit 113 and determines the color of the skin portion of the reference image. Then, the image composition unit 114 corrects the processed image to the skin color of the reference image. Specifically, if the reference image is a user image, as shown in FIG. 5A, the face portion Y1 of the user image is synthesized with the face portion S1 of the sample image, and the periphery of the face of the sample image The part S2 and the upper body part S3 are corrected to the skin color of the face part Y1 of the user image.

  On the other hand, when the reference image matches the sample image, as shown in FIG. 5B, the face portion Y1 of the user image is combined with the face portion S1 of the sample image, and the face portion Y1 of the user image is sampled. The skin color of the peripheral portion S2 of the face of the image and the upper body portion S3 is corrected. Then, by clicking the “use this image” button in the window 200 shown in FIG. 10, the entire figure 211 on which the user's face is displayed is displayed on the entire screen 210.

  Further, the user processes the composite image based on the information about the user (S107). In the processing of the composite image in the present embodiment, as shown in FIG. 9B, first, deformation processing including any of the image size, aspect ratio, and shape is performed on the composite photograph (S201). Specifically, as shown in FIG. 6A, the user adjusts the slider 241 of each item 243 of the parameter screen 240 displayed on the window 200 or inputs a numerical value in the input value column 242. Then, in response to the adjustment or numerical input, the image processing unit 116 performs a deformation process including any of the image size, aspect ratio, and shape on the composite image, and as illustrated in FIG. The overall appearance 211 of the user displayed on the overall screen 210 is changed.

  Next, the image processing unit 116 adds a partial image as an additional image to the combined image and combines (S202). Specifically, as shown in FIG. 7, when the user selects the icon image 221 displayed on the parts screen 220, the icon image 221 is added to the overall appearance 211 of the user displayed on the entire screen 210 portion. Synthesized and displayed. At this time, the image processing unit 116 deforms the additional image at the same ratio according to the parameters used in the deformation process, combines the additional image with the composite image, and displays the composite image.

  In addition, the user performs a makeup process including a color change process of any one of coloring, color tone change, brightness change, and image synthesis on a part of the composite image (S203). At this time, as shown in FIG. 10, the image processing unit 116 sets parameters of the makeup processing screen 250 related to the color change processing based on the type of acquisition path, information on the reference color, and information on processing on the processed image. Set.

  In the present embodiment, the costume of the wedding dress is virtually tried on the screen using the costume simulation system, the costume simulation method, and the costume simulation program, but the present invention is not limited to this. The image of the costume and the image of the person and animal can be combined and virtually tried on the screen. For example, the costume for Shichigosan, the costume for the adult ceremony, the character costume, the pet costume, etc. Various costumes may be used.

(Action / Effect)
According to this embodiment, either one of the user image or the sample image is selected as a reference image, and the unselected image is processed based on the reference color of the reference image. Since the sample image is combined with the sample image, for example, the combined image can generate a combined image with a uniform skin color and no sense of incongruity.

  In particular, in the present embodiment, the selection of the reference image can be determined according to the acquisition path of the user image Y. For example, in the case of an image uploaded from the user's personal computer through the Internet, the reference image is selected. By generating a composite image using the image as the sample image S, even if the user image Y has poor shooting conditions, the composite wedding image can be made a good image. Encourage interest in the company's wedding dress offering.

  On the other hand, in the case of an image that is transferred from a photographing device and acquired through a peripheral device connection cable such as a USB in a store, a composite image is generated by using the reference image as the user image Y, so A composite image can be generated in accordance with this, and the atmosphere when actually worn can be expressed.

  In the present embodiment, the image processing unit 116 further includes a function of adding a partial image as an additional image to the synthesized image and synthesizing the image, and includes a deformation including any of the image size, aspect ratio, and shape. The processing is performed on the composite image, and the additional image is deformed according to the parameters used for the deformation process, and the additional image is combined with the composite image, so accessories such as necklaces, tiaras, and gloves used during the wedding ceremony Since the icon image 221 is added to the overall appearance 211 of the overall screen 210, various wedding dress appearances can be simulated. In particular, in the present embodiment, when adding an additional image, it is possible to perform deformation processing including any of the size, aspect ratio, and shape of the image, so that the wedding dress figure corresponding to the body shape of the person can be displayed. It is possible to generate an image that can be actually tried on.

  Further, in the present embodiment, the image processing unit 116 has a function of performing makeup processing including color change processing of any of coloring, color change, brightness change, or image composition on a part of the composite image. Then, makeup can be made on the face portion of the wedding dress in which the self-face is synthesized, and the atmosphere when the wedding dress is actually worn can be provided. In particular, in the present embodiment, the makeup processing function sets parameters related to the color change processing based on the type of acquisition path, information on the reference color, and information on processing on the processed image. Even when the image S or the user image Y is used as a reference color, a parameter that allows makeup processing according to the skin color can be set. By following the use of the above, different skin colors can be followed to make up the makeup.

P ... Center part S ... Sample image S1 ... Face part of sample image S2 ... Peripheral part of face of sample image S3 ... Upper body part of sample image Y ... User image Y1 ... Face part 2 of user image (2a, 2b) ... Display Part 3 (3a, 3b) ... Camera 4 ... Skin color measuring sensor 5 ... Communication network 6 ... User terminal 7 (7a, 7b) ... User 9 ... Store 10 ... Information processing device 10a ... Store side device 10b ... Server device 41 ... Tip Unit 61a ... Input interface 61b ... Output interface 62 ... Communication interface 63 ... Browser unit 101 ... Input / output interface 101a ... Input interface 102 ... Communication interface 103 ... Operation signal acquisition unit 104 ... Output signal generation unit 110 ... Application execution unit 111 ... User Image acquisition Unit 112 ... acquisition path setting unit 113 ... reference color extraction unit 114 ... image synthesis unit 115 ... sample image acquisition unit 116 ... image processing unit 120 ... storage unit 120a ... sample image storage unit 200 ... window 210 ... whole screen 210 ... composite photo 211 ... Overall view 220 ... Parts screen 221 ... Icon image 230 ... Face composition screen 231 ... Sample image 232 ... User image 235 ... MIX button 240 ... Parameter screen 241 ... Slider 242 ... Input value field 243 ... Item 250 ... Make-up processing Screen 251 ... Slider 252 ... Icon image 260 ... Enlarged screen of the face part

Claims (9)

A costume simulation system that allows users to virtually try on costumes on the screen,
A user image acquisition unit for acquiring a user image of the user,
A sample image storage unit for storing a plurality of sample images of a person as a sample;
A sample image acquisition unit for acquiring an image of an arbitrary sample from the sample image storage unit;
An acquisition route setting unit for setting the type of acquisition route of the user image;
A reference color extraction unit that selects either the user image or the sample image as a reference image according to the setting in the acquisition path setting unit, and extracts a reference color from the selected reference image;
An image that is not selected as the reference image is set as a processed image, the processed image is processed based on the reference color, and the processed image is combined with the reference image to generate a combined image. A synthesis unit;
A costume simulation system comprising: an image processing unit that processes the composite image based on information about the user. The image processing unit further includes a function of adding a partial image as an additional image to the combined image and combining it.
A deformation process including any of image size, aspect ratio, and shape is performed on the composite image, and the additional image is deformed according to the parameters used in the deformation process, and the additional image is combined with the composite image. The costume simulation system according to claim 1, wherein the costume simulation system is combined with an image. The image processing unit further includes a function of performing makeup processing including color change processing of any one of coloring, color change, brightness change, or image synthesis on part of the composite image,
The costume simulation system according to claim 1, wherein a parameter relating to the color changing process is set based on the type of the acquisition path, information relating to the reference color, and information relating to processing on the image to be processed. A costume simulation method that allows a user to virtually try on a costume on the screen,
An image acquisition step of acquiring a user image of a user and acquiring a sample image of a person as a sample;
A reference color extraction step of setting a type of acquisition path of the user image, selecting either the user image or the sample image as a reference image according to the setting, and extracting a reference color from the selected reference image;
An image that is not selected as the reference image is set as a processed image, the processed image is processed based on the reference color, and the processed image is combined with the reference image to generate a combined image. A synthesis step;
A costume simulation method comprising: an image processing step of processing the composite image based on information about the user. In the image processing step,
Add a partial image as an additional image to the synthesized image and synthesize it,
A deformation process including any of image size, aspect ratio, and shape is performed on the composite image, and the additional image is deformed according to the parameters used in the deformation process, and the additional image is combined with the composite image. The costume simulation method according to claim 4, wherein the costume simulation method is combined with an image. In the image processing step,
A part of the composite image is subjected to makeup processing including color change processing of any one of coloring, color tone change, brightness change, or image synthesis,
The costume simulation method according to claim 4, wherein a parameter relating to the color change processing is set based on the type of the acquisition path, information on the reference color, and information on processing on the processed image. A costume simulation program that allows users to virtually try on costumes on the screen.
An image acquisition step of acquiring a user image of a user and acquiring a sample image of a person as a sample;
A reference color extraction step of setting a type of acquisition path of the user image, selecting either the user image or the sample image as a reference image according to the setting, and extracting a reference color from the selected reference image;
An image that is not selected as the reference image is set as a processed image, the processed image is processed based on the reference color, and the processed image is combined with the reference image to generate a combined image. A synthesis step;
A costume simulation program for executing a process including an image processing step of processing the composite image based on information on the user. In the image processing step,
Add a partial image as an additional image to the synthesized image and synthesize it,
A deformation process including any of image size, aspect ratio, and shape is performed on the composite image, and the additional image is deformed according to the parameters used in the deformation process, and the additional image is combined with the composite image. The costume simulation program according to claim 7, wherein the costume simulation program is synthesized with an image. In the image processing step,
A part of the composite image is subjected to makeup processing including color change processing of any one of coloring, color tone change, brightness change, or image synthesis,
8. The costume simulation program according to claim 7, wherein a parameter relating to the color changing process is set based on the type of the acquisition path, information relating to the reference color, and information relating to processing on the processed image.