JP7249015B2 - Product fitting system, terminal device, management server, computer program, and product sales method - Google Patents

Description

The present disclosure relates to a product fitting system, a terminal device, a management server, a computer program, and a product sales method.

When purchasing a furnishings such as clothes, it is common practice for a user to actually wear the clothing and check the wearing state before making a purchase. However, in some cases, it may feel troublesome to actually wear the furnishings that the user is considering purchasing. In particular, when there are a large number of clothing accessories to be considered, it is troublesome to wear them one by one. In this respect, for example, Japanese Patent Application Laid-Open No. 2015-079362 (Patent Document 1) discloses a try-on assisting device that confirms the wearing state of furnishings on a user by synthesizing the image data of the user and the image data of the furnishings. is proposing. By using this technology, it is possible to check the wearing state on the screen without actually wearing the furnishings, reducing the trouble of wearing them.

JP 2015-079362 A

On the other hand, if the fitting support device is installed in a store that sells furnishings, the user needs to go to the store each time he/she considers purchasing the furnishings and perform an operation to create his own image data. Become. Therefore, first of all, it takes time and effort to create the image data. In addition, even though some users want to freely see and enjoy wearing the furnishings at any place other than the store, such as their own home, it is troublesome to go to the store.

Taking these into consideration, the inventor of the present invention generated the user's image data in advance and stored it in the memory, and at the timing desired by the user, combined it with the image of the desired clothing and accessories and displayed it. Obtained. This saves the trouble of creating image data each time a purchase is considered, and allows the user to wear the furnishings at a desired location and timing without being restricted by a specific location or timing such as a store. can be confirmed.

Since the image data used for synthesizing with the image of the furnishings is used for confirming the wearing state of the furnishings, the more accurately the user's body shape is represented, the more accurately the wearing state can be obtained. can be confirmed. In particular, when the furnishings are clothes, the image data used for synthesis is better if the user is naked or wearing thin clothes such as underwear that is close to naked. appear. Therefore, it is desirable to use data representing the body shape of the user in the process of synthesizing the image of the furnishings.

On the other hand, since such image data may affect the privacy of users, some users may not store such image data in memory or use it on devices other than user terminals. may not wish to Therefore, in the process of synthesizing the image data of the user with the image of the furnishings, it is desired to achieve both the use of data that reproduces the body shape of the user and the assurance of security.

According to one embodiment, a product try-on system includes a terminal device, a management server, and a registered user image of a registered user who is a user of the terminal device and who is accessible from the terminal device and inaccessible from the management server. and a memory area for storing personal data including personal data, wherein the terminal device transmits at least the personal data excluding the registered user image to the management server, and the management server uses the personal data from the terminal device to perform fitting. Management processing is executed, and the fitting management processing includes transmitting product data for displaying product images to the terminal device, and the terminal device uses the product data to display the product images and registered user images. Display the synthesized image.

According to another embodiment, the terminal device is a terminal device that can communicate with the management server and has a processor that can access a memory area that does not allow access to the management server, and the memory area is a registered user's registered user The personal data including the image is stored, and the processor transmits the personal data excluding at least the registered user image to the management server, and uses the product data received from the management server for displaying the image of the product. , an image obtained by synthesizing the product image and the registered user image is displayed.

According to another embodiment, the management server comprises a communication device capable of communicating with the terminal device and a processor, the communication device receiving personal data of the user of the terminal device from the terminal device, the processor comprising: A try-on management process is executed using the personal data, and the try-on management process includes transmitting product data for displaying an image of the product to the terminal device.

According to another embodiment, the computer program is a computer program that causes the computer to function as a terminal device capable of communicating with the management server, the personal data of the registered user being stored in the computer to the management server, at least a step of transmitting personal data excluding the registered user image; and a step of displaying an image obtained by synthesizing the product image and the registered user image using the product data for displaying the product image received from the management server. and let it run.

According to another embodiment, the computer program is a computer program that causes the computer to function as a communication device capable of communicating with the terminal device, and the personal data of the user of the terminal device received from the terminal device is used by the computer to try on the clothes. , and the try-on management process includes transmitting product data for displaying an image of the product to the terminal device.

According to another embodiment, a product sales method is a product sales method in a product try-on system including a user terminal device, a management server, and a sales company server on the product sales side, wherein the product try-on system includes: , having a memory for storing personal data including a registered user image of a registered user who is a user of the terminal device, which is accessible from the terminal device and inaccessible from the management server and the sales company server, and is accessible from the terminal device to the management server; a step of transmitting the registered user's personal data excluding the registered user image; a step of transmitting product data for displaying the product image from the management server to the terminal device; a step of synthesizing an image of a product with a registered user image stored in a memory and displaying the synthesized image; a step of transmitting product order data from the terminal device to the management server; and a step of transmitting the order data and data specifying the registered user from the management server to the sales company server.

FIG. 1 is a diagram showing an example of the configuration of a product fitting system (hereinafter referred to as system) according to an embodiment. FIG. 2 is a block diagram showing the configuration of a terminal device included in the system. FIG. 3 is a diagram schematically showing the contents of personal data stored in the memory of the terminal device. FIG. 4 is a block diagram showing the configuration of a management server included in the system. FIG. 5 is a diagram schematically showing the content of clothing data stored in the memory of the management server. FIG. 6A is a flow chart showing an overview of the operation of the system, and FIG. 6B is a diagram showing transition of operation modes of the terminal device. FIG. 7 is a flow chart showing the processing flow of the terminal device when the operation mode is the mirror mode. FIG. 8 is a diagram for explaining the contents of user registration processing and personal data update processing. FIG. 9 is a diagram showing the processing flow of the terminal device and the management server when the operation mode is the try-on mode. FIG. 10 is a diagram showing an example of the try-on screen. FIG. 11 is a diagram showing an example of a try-on screen. FIG. 12 is a diagram showing the flow of sales operations in the system. FIG. 13 is a diagram showing the flow of another example of sales operations in the system.

[1. Overview of product fitting system, terminal device, management server, computer program, and product sales method]

(1) A product fitting system included in the present embodiment includes a terminal device, a management server, and registration of a registered user who is a terminal device user who is accessible from the terminal device and inaccessible from the management server. a memory area for storing personal data including a user image, wherein the terminal device transmits at least the personal data excluding the registered user image to the management server, and the management server uses the personal data from the terminal device The try-on management process includes transmitting product data for displaying an image of the product to the terminal device, and the terminal device uses the product data to display the image of the product and the registered user. Display the image synthesized with the image.

The terminal device is, for example, a device that has a life-size display of the user and a camera that captures the user standing in front of the display. The memory area may be possessed by the terminal device, or may be possessed by another device included in the product try-on system.

The registered user image is, for example, a registered user CG (computer graphics) or a photograph as described later. Since the registered user images are stored in a memory area accessible from the terminal device, when the terminal device receives the image of the product from the management server, the registered user image can be synthesized without newly preparing (creating, etc.). can be displayed as Therefore, it is possible to reduce the troublesomeness of operations such as photographing for creating a registered user image each time it is displayed, and to quickly display a synthesized image. Although the registered user image is stored in the memory area accessible from the terminal device, it is not transmitted to the management server. Therefore, it is possible to provide data that reproduces the body shape of the user, which is effective for management processing in the management server, while ensuring the security of the registered user image.

(2) Preferably, the product image is a CG of the product. As a result, an image in which the CG of the product and the image of the registered user are combined can be displayed.

(3) Preferably, the registered user image is the registered user's registered user CG. As a result, it is possible to display an image in which the product image and the registered user's CG are combined.

(4) Preferably, the terminal device creates a registered user CG using a plurality of photographed images from a plurality of directions including the registered user's face, and stores it in the memory area as personal data. The multiple directions include, for example, the front, side, rear, and upper directions. As a result, a highly accurate registered user CG can be created.

(5) Preferably, the terminal device includes a camera and a display, and when the image of the registered user is detected from the image taken by the camera, the terminal device acquires the body shape data of the registered user from the taken image, Further used for creating CG. By extracting the body type data of the registered user from the photographed image, the body type data of the registered user can be automatically extracted only by standing in front of the terminal device and being photographed without performing a special input operation. Therefore, a highly accurate registered user CG is created without forcing the user to perform operations.

(6) Preferably, the terminal device updates the registered user CG stored in the memory area using the body shape data after acquiring the body shape data of the registered user from the captured image. As a result, the registered user CG can automatically reproduce the latest appearance of the user without forcing the user to perform any special operation.

(7) Preferably, the terminal device acquires body shape data of the registered user from the predefined image when the recorded image includes the predefined image of the registered user. The predefined images are, for example, front facing images, hand raised images, and the like. As a result, accurate body type data of the registered user can be obtained.

(8) Preferably, the terminal device further acquires information about the appearance of the registered user from the captured image, and stores it in the memory area as personal data. By storing information about the registered user's appearance in the memory area, it is possible to create a CG that is closer to the registered user's appearance. As a result, when combined with the CG of furnishings such as clothes, it is possible to make it look closer to the state of being tried on. Also, if the information on the appearance of the registered user is information on the furnishings worn by the registered user, the CG of the furnishings can also be created. As a result, by synthesizing the CG of the registered user and the CG of the product, and further synthesizing the CG of the furnishings, it is possible to confirm the compatibility of the product with the furnishings in hand from the synthesized image.

(9) Preferably, the terminal device has a mirror mode and a try-on mode as operation modes. In the mirror mode, the image captured by the camera is displayed on the display. CG is synthesized and displayed. The product is clothing and other accessories, and by synthesizing these CG images, an image of the registered user trying on the product can be obtained. As a result, the user can see an image as if the user were trying on clothes without actually trying them on.

(10) Preferably, in the terminal device, of the display used for display in the fitting mode and the display used for display in the mirror mode, at least the display used for display in the mirror mode is such that the entire body of the registered user is substantially It is the size displayed in life-size. Therefore, the user is displayed in approximately life size on the display. Therefore, the user can use the terminal device like a full-length mirror, which increases convenience. In addition, the user is more likely to stand in front of the terminal device at natural timings, such as when changing clothes or before going out. As a result, the user is photographed in a natural daily action, rather than intentionally.

(11) Preferably, the terminal device reads user attribute information other than the registered user's body shape data from another device, and further uses it for creating the registered user CG. User attribute information other than body type data is, for example, sex, age group, and the like. As a result, it is possible to eliminate or reduce the user's input operation, and obtain more detailed user attributes in some cases.

(12) Preferably, the terminal device has, as an operation mode, a mirror mode for displaying an image captured by the camera on the display, and the terminal device further changes the operation mode from the mirror mode to a mode other than the image captured by the camera. It is possible to switch to other modes of displaying images.

(13) Preferably, the other mode of operation includes displaying an image captured by a camera other than the camera. Another camera is, for example, a camera installed at a location different from the terminal device. In this case, the user can view captured images of different locations at the location where the terminal device is installed.

(14) Preferably, the product try-on system further includes a size selection unit that selects a product size based on the personal data of the registered user, and the terminal device displays the product image and the registered user image corresponding to the selected size. display an image that combines the As a result, the user can see an image as if the user were trying on clothes without actually trying them on.

(15) Preferably, the personal data includes a plurality of registered user images, and the plurality of registered user images are images representing a plurality of different poses of the registered user and/or chronologically continuous images forming a moving image. is. As a result, it is possible to display an image synthesized with the image of the product by changing the pose of the registered user. Therefore, the user can see an image that is similar to when the user actually tries on the clothes and takes various poses, and can obtain a more accurate image of when the user is wearing the clothes. In addition, an image synthesized with an image of a product can be displayed as a moving image. Therefore, the user can see an image similar to when the user actually tries on the clothes and moves, and can obtain a more accurate image of when the user is wearing the clothes.

(16) Preferably, the terminal device transmits personal data to the management server, excluding at least the registered user image, the registered user name, and the photographed image. As a result, it is possible to ensure both security and try-on management processing in the management server.

(17) The terminal device included in the present embodiment is a terminal device included in the product fitting system described in (1) to (16), and is capable of communicating with the management server. a processor capable of accessing a non-accessible memory area, the memory area storing personal data including a registered user image of a registered user, the processor providing at least the personal data excluding the registered user image to the management server; and displays an image obtained by synthesizing the image of the product and the registered user image using the product data for displaying the image of the product received from the management server. This terminal device has the same effects as the terminal devices described in (1) to (16).

(18) The management server included in this embodiment includes a communication device capable of communicating with a terminal device, and a processor. Upon receipt, the processor performs fitting management processing using the personal data, and the fitting management processing includes transmitting product data for displaying an image of the product to the terminal device. As a result, the terminal device can synthesize and display the stored image of the user and the image of the product.

(19) A computer program included in the present embodiment is a computer program that causes a computer to function as a terminal device capable of communicating with a management server, and the personal data of a registered user is transmitted to the computer to the management server. a step of transmitting personal data excluding at least the registered user image; and using the product data for displaying the product image received from the management server, displaying an image obtained by synthesizing the product image and the registered user image. and causing to be performed. Since this computer program causes a computer to function as the terminal device according to any one of (1) to (16), it has the same effects as those terminal devices.

(20) A computer program included in the present embodiment is a computer program that causes a computer to function as a communication device capable of communicating with a terminal device. The try-on management process includes transmitting product data for displaying an image of the product to the terminal device. This computer program causes the computer to function as the management server described in (18). Therefore, the computer program has the same effect as this management server.

(21) A product sales method included in the present embodiment is a product sales method in a product try-on system including a user terminal device, a management server, and a sales company server on the product sales side. The system has a memory that stores personal data including a registered user image of a registered user who is a user of the terminal device and that is accessible from the terminal device and inaccessible from the management server and the sales company server. a step of transmitting personal data excluding a registered user image of a registered user to a management server; a step of transmitting product data for displaying an image of a product from the management server to a terminal device; a step of synthesizing an image of a product and a registered user image stored in a memory using a , and displaying the synthesized image; a step of receiving, a step of transmitting product order data from the terminal device to the management server, and a step of transmitting the order data and data specifying the registered user from the management server to the sales company server.

Since this sales method includes the processes in the product try-on system described in (1) to (16), it has the same effect as the product try-on system described in (1) to (16). Furthermore, in this sales method, exchange of data relating to product sales between the terminal device and the sales company server is mediated by the management server, so highly reliable transactions can be carried out.

Preferably, the sales method further comprises the steps of: transmitting a shipping notification indicating completion of shipping of the product to the registered user from the sales company server to the management server; and transferring the shipping notification from the management server to the terminal device. .

[2. Examples of product fitting system, terminal device, management server, computer program, and product sales method]

[First embodiment]
<System configuration>
As shown in FIG. 1, a product fitting system (hereinafter abbreviated as system) 100 according to the present embodiment includes terminal device 1 . The terminal device 1 has a substantially life-size display 13 and a camera 14 of a user. The terminal device 1 also has personal data B111, which will be described later.

System 100 includes management server 5 . The management server 5 has a communication function to communicate with other devices via the Internet 9 . Massage chair 3 may be included in system 100 as another device with which management server 5 can communicate via Internet 9 . The massage chair 3 is connected to the Internet 9 via, for example, a wireless LAN (Local Area Network) such as WiFi (Wireless Fidelity), a communication line for mobile phones such as LTE (Long Term Evolution), or the like. 9 to communicate with the management server 5 . The terminal device 1 has a function of performing wireless communication 7 with another device. The wireless communication 7 is, for example, short-range wireless communication such as Bluetooth (registered trademark) with the massage chair 3 . In this case, the terminal device 1 can connect to the Internet 9 via the massage chair 3 and communicate with the management server 5 mutually. Note that the terminal device 1 may perform wireless communication 7A, which is communication via a wireless LAN or a communication line for mobile phones. In this case, the terminal device 1 may be connected to the Internet 9 via a wireless LAN, a communication line for mobile phones, or the like, and may directly communicate with the management server 5 via the Internet 9, or may be connected to the Internet 9. may be

<Configuration of terminal device>
As shown in FIG. 2, the terminal device 1 includes a computer with a processor 10 and a memory 11 in addition to a display 13 and a camera 14 . The processor 10 is, for example, a CPU (Central Processing Unit).

The camera 14 is installed, for example, above the display 13 as shown in FIG. Thereby, the whole body of the user standing in front of the display 13 is photographed. In addition, when the display 13 can transmit from the rear to the front like a half mirror, the camera 14 may be installed behind the display 13 . As a result, the user standing in front of the display 13 can be photographed from a height close to the front. The camera 14 performs a photographing operation according to the control signal from the processor 10 and inputs the photographed image to the processor 10 .

The display 13 is approximately the same size as the user, as described above. The display 13 displays an image based on the image data passed from the processor 10 according to control signals from the processor 10 .

The memory 11 includes flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), ROM, RAM (Random Access Memory), and the like. Memory 11 includes a personal data database (DB) 111 . The personal data DB 111 can store one or a plurality of pieces of personal data PD1, PD2, PD3, . At least the personal data DB 111 in the memory 11 is a memory area that does not allow access from other devices including the management server 5 . The personal data DB 111 may be stored in an external device such as a server that is different from the terminal device 1 . Even in that case, the personal data DB 111 is permitted to be accessed from the terminal device 1, and is not permitted to be accessed from other devices.

As shown in FIG. 3, the personal data PD includes data D11 of an image captured by the camera 14 of a registered user (hereinafter also referred to as a registered user) and identification data D12 of the registered user. The registered user identification data D12 is data used for authenticating the registered user from the photographed image, and is, for example, image data such as the registered user's face image, fingerprint image, or palm print image.

Personal data PD further includes registered user CG data D13. A registered user CG is an example of an image of a registered user (hereinafter referred to as a registered user image), and refers to computer graphics (CG) created for a registered user and reproducing the body shape of the registered user. In the following description, it is assumed that the registered user image is the registered user CG. In this case, the terminal device 1 displays a composite image of the registered user's CG and the image of the furnishings (for example, clothes) on the display 13 by the processing described later.

Personal data PD further includes registered user attribute data D14 and figure data D15. The attribute data D14 is, for example, data indicating user attributes such as gender and age (or generation), and is input from the input device 15 as an example. As another example, as shown in FIG. 1, communication device 12 may read from server 4 if other servers 4 are included in the system. The server 4 is, for example, a server or the like that manages users of the massage chair 3 . As a result, it is possible to eliminate or reduce the user's input operation, and obtain more detailed user attributes in some cases. The figure data D15 includes data indicating the size of each part of the registered user's body. In particular, it includes data such as height, shoulder width, chest circumference, etc., which are used to specify the size of clothing and other accessories.

The memory 11 also stores a program 112 consisting of one or more programs. The program 112 includes a program M11 for causing the processor 10 to perform operations in the mirror mode, a program M12 for performing operations in the try-on mode, and an example of an operation mode other than the mirror mode and the try-on mode. and a program M13 for executing the operation in the viewing mode. Operations in each operation mode will be described later. These programs M11 to M13 may be independent programs, or may be program modules branched and called from the main program according to the operation mode.

The processor 10 reads the program 112 and executes various processes. The program 112 can be transferred in a state recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory) or DVD-ROM (Digital Versatile Disk ROM), or can be downloaded from a computer device such as a server computer. can also be transferred by The program 112 may be a so-called web application that runs on a web browser, or a so-called dedicated application that runs only on the processor 10 .

The terminal device 1 further has a communication device 12 . The communication device 12 has a function of performing wireless communication 7 and is, for example, a communication module for performing wireless communication 7 .

The terminal device 1 also has an input device 15 . The input device 15 is a button, a keyboard, or the like, and inputs an operation signal indicating a user's operation to the processor 10 . The input device 15 may be a touch panel integrated with the display 13. In this case, the input device 15 inputs to the processor 10 an operation signal indicating a position (coordinates) on the touch panel. Also, the input device 15 may be a microphone for voice input, and in this case, the terminal device 1 can receive an instruction input by voice.

<Management server configuration>
As shown in FIG. 4, the management server 5 is configured by a computer having a processor 50 and a memory 51. FIG. Processor 50 is, for example, a CPU.

The memory 51 includes flash memory, EEPROM, ROM, RAM, and the like. Memory 51 includes clothing data DB 511 . The clothing data DB 511 is a database that stores one or more pieces of clothing data CD1, CD2, CD3, . . . (also referred to as clothing data CD).

The clothing data CD is data for displaying the image of the registered clothing on the terminal device 1, and as shown in FIG. It includes size data D21 for each size (for example, S, M, L) and CG data D22 to D24 for clothes of each size. The size data D21 is data that defines the condition of the user's appearance suitable for clothing of that size, and is, for example, body type data representing conditions (range) of suitable sizes, or conditions of suitable gender and age. including attribute data, etc.

Note that the clothing data CD may further include recommendation data, which is data defining conditions such as attributes for selecting clothing to be recommended to registered users. The recommended data includes, for example, data indicating the registered user's sex, age, body type range, makeup color, clothing color, hairstyle, hair length, and the like.

CG of clothes is an example of an image of clothes, and refers to computer graphics created for clothes. In the following description, it is assumed that the clothing image is CG. In this case, the terminal device 1 displays a composite image of the image of the registered user and the CG of the clothes on the display 13 by the processing described later.

The CG data D22 to D24 may be created by a manufacturer or the like that sells the target clothing and provided to the management server 5. FIG. Alternatively, data such as the shape (size of each part), color, material, etc. necessary for creating a CG is obtained from a manufacturer that sells the target clothing, and the processor 50 of the management server 5 creates the CG of the clothing based on the data. may be created and stored in the DB 511 as the clothing data CD.

The memory 51 also includes a transmission data DB 512 . The transmission data DB 512 is a database that stores one or a plurality of transmission data TD1, TD2, TD3, .

The transmission data TD is data related to clothing transmitted from the management server 5 to the terminal device 1, and as shown in FIG. It includes user identification data D32, which is data to be specified, and charging data D33. Preferably, the user identification data D32 is less confidential than the registered user's face, name, address, telephone number, etc., such as an identification number assigned to the registered user by the management server 5 or the manufacturer that sells clothes. . The billing data D33 is billing data for allowing the terminal device 1 to use the CG data of the clothes transmitted from the management server 5 to the terminal device 1 for a certain period of time (hereinafter referred to as a keep state). For example, it is data indicating whether or not a predetermined amount of money has been paid, the date and time of payment, the amount of money paid, and the duration of the keep state corresponding to the amount of money paid.

The memory 51 also stores a program 513 consisting of one or more programs. The program 513 includes a try-on mode program M21 for causing the processor 50 to execute a try-on mode operation, and a purchase mode program M22 for causing the purchase mode operation to be executed. These programs M21 and M22 may be independent programs, or may be program modules branched and called from the main program according to the operation mode.

<Outline of operation in the system>
The terminal device 1 is assumed to be installed and used in a home or the like. A user is registered in the terminal device 1 (registered user), and a registered user CG is created. The registered user CG may be a two-dimensional image or a three-dimensional image.

The registered user's personal data PD including the registered user CG is stored in the terminal device 1 . As shown in FIG. 6A, the terminal device 1 transmits at least the personal data excluding the registered user CG to the management server 5 (step S10).

The management server 5 uses the personal data from the terminal device 1 to execute a management process for trying on furnishings (step S20). The try-on management process includes transmitting to the terminal device 1 clothing-related data (merchandise data) including CG data for displaying CG of clothing (merchandise) as an example of clothing. As another example, it may include transmitting to the terminal device 1 image data for displaying an image of the furnishings to be recommended to the user.

The terminal device 1 synthesizes the CG of the clothes provided by the management server 5 and the stored registered user CG, and displays the synthesized image as a fitting image on the display 13 (step S30). In the system 100, the management server 5 provides the CG of clothes, and the terminal device 1 synthesizes the CG of the clothes and the stored registered user CG and displays the synthesized image on the display 13 as a fitting image. A series of actions is also called a try-on action. Although the registered user has not actually tried on the clothes, he or she can confirm the image of the CG wearing the clothes on the display 13 as if they had tried on the clothes.

<Functional configuration of terminal device>
As shown in FIG. 2, the processor 10 of the terminal device 1 functions as an extraction processing unit 107 that executes extraction processing by reading out the program 112 from the memory 11 and executing it. Extraction processing is processing for extracting a registered user from an image captured by the camera 14 . The extraction process includes, for example, matching the photographed image with the face image, which is an example of the authentication data of the registered user. As another example, the extraction process may include a process of extracting a face image from a captured image by edge extraction, and performing user authentication based on the correlation between the extracted face image and the registered user's face image. .

Preferably, the extraction process further includes a process of extracting body data of the registered user from the image of the registered user extracted from the captured image. More preferably, the extraction process is performed when the image of the registered user extracted from the captured image is a predefined image. The predefined images are, for example, front facing images, hand raised images, and the like. As a result, accurate body type data can be obtained.

The extracted body type data includes at least body part sizes used to select clothing sizes. Alternatively, in the process of extracting the body data, the body data of the registered user may be extracted from the registered user CG created from the image of the registered user extracted from the captured image by the CG creation process described later. As a result, the registered user can automatically take a picture and extract the body type data simply by standing in front of the terminal device 1 without performing any particular input operation.

Note that the extraction process includes a process of extracting information about the appearance of the registered user from the image of the registered user extracted from the captured image. Appearance information includes, for example, the image of clothing and other accessories worn, color, size, hair length, hairstyle, hair color, makeup color, combination of accessories, and the like. . The processor 10 may include data indicating these pieces of information in the registered user's personal data PD as the attribute data D14.

The processor 10 also functions as a CG creation unit 101 that executes CG creation processing. The CG creation process may be a general CG creation process. That is, the CG creation process is, for example, a process of generating a stereoscopic image using captured images including the registered user's face from four directions, i.e., the front, the side, the rear, and the top. As a result, a highly accurate registered user CG can be created. Note that the imaging directions are not limited to the front, side, rear, and upper directions, and may include other directions. For example, diagonal directions may be included. For this reason, the processor 10 may have a function of guiding the registered user to an appropriate position and direction when photographing for CG creation processing. The guidance includes, for example, displaying the preferred standing position on the display, announcing the preferred posture by voice, and the like. As a result, a photographed image necessary for CG creation can be obtained, and a more accurate registered user CG can be obtained.

Preferably, the CG creation unit 101 performs the CG creation process using other data included in the personal data in addition to the captured image of the registered user. The other data included in the personal data is, for example, information extracted by the extraction process, such as body type data of the registered user, information on appearance (for example, hairstyle, etc.), and the like. By further using information related to the appearance such as hairstyle, it is possible to create a CG that is closer to the appearance of the registered user. As a result, when combined with the CG of the clothes, it is possible to make the clothes closer to the state of trying them on. Further, by further using information about the appearance of the furnishings worn by the registered user, it is possible to synthesize the CG of the furnishings as well as the CG of the registered user and the CG of the product. As a result, it is possible to propose coordination in combination with clothing and accessories owned by the registered user, and it is also possible to propose clothing and accessories from the manufacturer side. In addition, it is possible to check the compatibility between the clothing, which is the product, and the accessories in hand by using the synthesized image.

Further, other data included in the personal data is, for example, attribute data (gender, etc.) of the registered user, such as attribute data input using the input device 15 or attribute data read from another device. and so on. By using these data in the CG creation process, a more accurate registered user CG can be obtained.

The processor 10 further includes a transmission processing unit 102 that executes a transmission process of transmitting size selection data, which is data necessary for the management server 5 to execute the management process of trying on furnishings, to the management server 5 . function as The size selection data includes, for example, data indicating selection results of clothes to be tried on, and registered user's personal data used for selecting a size according to the registered user. As a result, the management server 5 can select a clothing size suitable for the registered user, and can transmit CG data of clothing of a suitable size to the terminal device 1 . Further, in the management server 5, by storing the data for size selection in association with the identification information of the registered user, when the user corresponding to the identification information requests the CG of clothes later, the stored size can be displayed. Based on the data for selection, the CG of the clothes of the size suitable for the size of the registered user can be transmitted to the terminal device 1 . In addition, the stored size selection data can be provided for processing such as sales promotion to registered users and/or other users related to registered users.

It should be noted that the transmission processing unit 102 transmits to the management server 5 at least the personal data excluding the registered user CG. As a result, the registered user CG is not passed to the management server 5 side. Since the registered user CG is used for synthesizing with the clothing CG, it is desired that the registered user's physique is represented. For example, it is desired that the CG is a naked body of the registered user or wearing thin clothes such as underwear. Therefore, some users do not want such CGs to be sent to the management server 5, so the data is highly secure. In this respect, the transmission processing unit 102 transmits at least the personal data other than the registered user CG to the management server 5, thereby avoiding transmission of the registered user CG to the management server 5 and ensuring high security.

Preferably, as shown in FIG. 3, the transmission processing unit 102 transmits at least the registered user CG data D13, the registered user identification data D12 such as the registered user name, and the photographed image D11 from the personal data PD. Data (non-transmission data) excluding (transmission data) including attribute data D14 such as gender and age (or generation) and body type data D15 is transmitted to the management server 5 . As a result, it is possible to ensure both security and try-on management processing in the management server 5 .

The processor 10 also functions as an image synthesizing unit 103 that executes image synthesizing processing. The image synthesizing unit 103 uses the registered user CG data D13 stored in the personal data DB 111 and product data (for example, clothes data) acquired from the management server 5 to combine the CG of the clothes and the CG of the registered user. Execute the image synthesis process to be synthesized. As a result, the registered user can obtain an image of a state in which the user has tried on the clothes using his or her CG without actually trying on the clothes. In addition, since the registered user CG data D13 stored in the personal data DB 111 is used for the image synthesizing process, it is possible to easily and quickly obtain a tried-on image rather than creating a registered user CG each time a try-on is performed.

Note that the display control unit 104 and the purchase processing unit 105 shown in FIG. 2 will be described in later embodiments. Also, the size selection unit 106 indicated by a dotted line is not included in the processor 10 of the terminal device 1 according to the first embodiment, and is realized by the processor 10 of the terminal device 1 according to the later embodiments. shall be

<Functional configuration of management server>
As shown in FIG. 4, the processor 50 of the management server 5 functions as a size selector 501 by reading out and executing a program 513 from the memory 51 . The size selection unit 501 selects a size suitable for the registered user from among a plurality of sizes prepared for the designated clothes based on the size selection data of the registered user received from the terminal device 1. . At this time, the size selection unit 501 compares the registered user's personal data included in the size selection data with the size data D21 included in the clothes data CD for the selected clothes, and the personal data is the size data D21. Select a size that satisfies the conditions indicated by .

The processor 50 further functions as a transmission processing unit 502 that executes transmission processing for transmitting CG data of clothes. The transmission process includes reading CG data of the size selected by the size selection section 501 from the clothing data CD of the clothing selected by the registered user, passing it to the communication device 52 and transmitting it to the terminal device 1 .

The transmission process also includes generating the transmission data TD shown in FIG. 4 and storing it in the transmission data DB 512 of the memory 51 . That is, the transmission process includes a process of associating the transmitted clothing data D31 with the identification data D32 of the registered user who is the transmission destination and the billing data D33 to generate the transmission data TD.

Processor 50 further functions as a purchase processing unit 504 that executes purchase processing when a registered user's instruction to purchase the selected clothing item is received. Purchase processing will be described in a later embodiment.

It should be noted that the image synthesizing unit 505 indicated by the dotted line is not included in the processor 50 of the management server 5 according to this embodiment, and will be described in a later embodiment.

<Operating mode>
The terminal device 1 has multiple operation modes. The multiple modes of operation include a mirror mode, a try-on mode and, as another mode, a viewing mode, as shown in FIG. 6B. The mirror mode is an operation mode realized by the processor 10 of the terminal device 1 reading out the program 112 from the memory 11 and executing the mirror mode program M11. , or an operation mode in which the images are displayed substantially in real time. In mirror mode, personal data update operations are repeatedly performed. The personal data update operation is an operation for executing personal data update processing, which will be described later. Further, in the mirror mode, when a user operation instructing the start of the user registration operation is received with respect to the input device 15, the user registration operation described later is executed.

The try-on mode is an operation mode realized by the processor 10 executing the program M12 for the try-on mode. This is an operation mode in which a try-on operation is performed. In the try-on mode, a sales operation, which will be described later, is executed each time a user operation to instruct the purchase of clothes, which will be described later, is performed on the input device 15 . Note that the viewing mode is an operation mode in which an image other than the captured image from the camera 14 is displayed on the display 13 .

The processor 10 of the terminal device 1 switches the operation mode from the mirror mode to the try-on mode when receiving a user operation for instructing switching to the try-on mode on the input device 15 when the operation mode is the mirror mode. In the try-on mode, similarly, when a user operation instructing switching to the mirror mode is received on the input device 15, the operation mode is switched from the try-on mode to the mirror mode. It should be noted that switching from the try-on mode to the mirror mode may also be performed when a predetermined period of time elapses (timeout) without user operation being performed in the try-on mode.

Further, when the processor 10 of the terminal device 1 receives a user operation for instructing switching to the viewing mode with respect to the input device 15 when the operation mode is the mirror mode, the processor 10 switches the operation mode from the mirror mode to the viewing mode. In the viewing mode, similarly, when a user operation instructing switching to the mirror mode is received on the input device 15, the operation mode is switched from the viewing mode to the mirror mode.

Further, when the operation mode is the fitting mode, the processor 10 of the terminal device 1 switches the operation mode from the fitting mode to the viewing mode upon receiving a user operation for instructing switching to the viewing mode on the input device 15 . In the viewing mode, similarly, when a user operation instructing switching to the fitting mode is received on the input device 15, the operation mode is switched from the viewing mode to the fitting mode.

<Registration operation and update operation>
In the terminal device 1, in the mirror mode, as shown in FIGS. 7 and 8, user registration processing (step S1) is first executed, and then personal data update processing (step S1) is executed to update the registered personal data. Step S2) is executed. These processes are executed when the operation mode is the mirror mode.

The processor 10 of the terminal device 1 reads the program 112 from the memory 11 and executes the program M11 for the mirror mode. At this time, when the user registration processing is instructed from the input device 15, the user registration processing S1 is executed. do. The processor 10 repeatedly executes the personal data update process S2 while executing the mirror mode program M11.

In the user registration process S1, first, the processor 10 obtains the captured image IM1 from the camera 14 (step S101), and extracts the user A from it (step S103). In step S101, photography necessary for CG creation is performed. For example, photography is performed from four directions, front, side, back, and top. Further, in step S101, the processor 10 may guide the user by voice or display on the display 13 so that the position and direction are suitable for photographing.

Note that in step S101, in addition to photographing in a default posture such as facing the front, for example, the photographing is performed in one or a plurality of different poses defined in advance, such as a state in which the hand is raised, a state in which the user faces the back or the side, and the like. may be taken. As a result, it is possible to display the registered user with different poses in the display processing described later.

The processor 10 extracts the body type data SI1 of the user from the photographed images of the user A obtained in steps S101 and S103 (step S105). In step S105, the processor 10 first determines whether or not the photographed image of the user A is a front-facing image, which is an example of a predefined image. Whether or not the face is facing forward can be determined by, for example, whether or not the eyes and nose are extracted from the face image by edge extraction. If the image is a frontal image (YES in step S51), body shape data is extracted from the image (step S52). As a result, accurate body type data can be obtained.

The processor 10 also creates CG data CG1 of the user A from the captured image (step S107). Preferably, in step S107, the processor 10 may display the created CG of the user A on the display 13, and may accept the user's registration operation on the input device 15 or an operation instructing redo. If an operation instructing redo is accepted, the processor 10 may redo the shooting from step S101. As a result, the CG desired by the user can be registered.

The processor 10 also receives an input of attribute data from a signal indicating a user's operation from the input device 15 (step S109). In step S109, the processor 10 may read attribute data such as gender and age (or generation) of the user to be registered from another device such as the server 4 shown in FIG. Processor 10 then stores these data as personal data PD in personal data DB 111 of memory 11 (step S111).

The order of steps S105 to S109 is not limited to the order shown in FIG. That is, they may be performed in an order different from this order. Also, if the body type data is extracted in step S105 after the CG is created in step S107, the processor 10 may extract the body type data SI1 from the created CG as shown in FIG. For example, as shown in FIG. 8, the processor 10 may extract the figure data SI1 by measuring the lengths L1 to L8 (=XXX) of each defined part of the CG data CG1 of the user A. . In FIG. 8, the length of the part L1 is the shoulder width, the length of the part L2 is the chest circumference, the length of the part L3 is the length of the body, the length of the part L4 is the rise length, the length of the part L5 is the crotch length, and the part L6 The length of the part L7 represents the waist, and the length of the part L8 represents the hip circumference. The body type data may include more part lengths than these parts, less than these parts lengths, or not all of these part lengths.

As another example, the body shape data to be extracted may be CG data SI1-1 for body shape data obtained by excluding the shape of the part where user A is specified from the CG data CG1 created in step S107. The site where user A is specified includes at least user A's face. This facilitates the process of extracting the body shape data, and allows the management server 5 to select a more suitable size in the size selection process.

The processor 10 may perform the above user registration process S1 for different users. As a result, a plurality of pieces of personal data PD for each of a plurality of users are registered in the terminal device 1 as shown in FIG. For example, when the terminal device 1 is installed at home, each family member may be registered as a user. This allows multiple users to use the system 100 .

In the mirror mode, the processor 10 causes the camera 14 to shoot at all times or for a predetermined period (step S113), and displays the shot image on the display 13. FIG. Since the display 13 has a life-size size of the user, the user is displayed on the display 13 approximately life-size. Therefore, the user can use the terminal device 1 like a full-length mirror, which increases convenience. In addition, the user is more likely to stand in front of the terminal device 1 at natural timings, such as when changing clothes or before going out. As a result, the user is photographed in a natural daily action, and the user registration process S1 and the personal data update process S2 are performed.

The image capturing in step S113 may be constant image capturing, or may be image capturing only for a predetermined period. In the case of constant shooting, the captured image input from the camera 14 and displayed on the display 13 is a moving image. By allowing the camera to be constantly photographed, it is possible to eliminate the necessity of determining whether or not photographing is possible, and prevent omission of photographing.

The predetermined period is, for example, a period in which the camera 14 is caused to take images in a short period of time and there is a change from the previous captured image. For example, when the terminal device 1 is installed in the living room of a home, shooting and display are not performed when no one is in the living room. When the user enters the living room and enters the photographing range of the camera 14, photographing and display are continuously performed during the period in which the user enters. As a result, shooting during an unnecessary period can be suppressed, so that shooting can be performed efficiently, and the amount of data of the shot image can be suppressed, so that the pressure on the memory can be suppressed. In addition, display can be suppressed during unnecessary periods, resulting in power saving.

In the mirror mode, the processor 10 repeatedly executes personal data update processing S2. In the personal data update process S2, the processor 10 extracts the user from the captured image IM2 obtained in step S113, and authenticates whether or not the user is registered user A. User authentication includes comparing the user identification data D12 included in the personal data PD with identification data obtained from the captured image. The identification data is, for example, a face image, positions of feature points in the face image, and the like.

If registered user A is included in photographed image IM2 (YES in step S115), processor 10 creates CG data CG2 of registered user A from the photographed image of registered user A, and creates CG data CG2 of registered user A. The registered user CG data D13 included in the personal data PD is updated (step S117).

Further, the processor 10 extracts the body type data SI2 of the registered user A from the photographed image of the registered user A, and updates the body type data D15 included in the personal data PD of the registered user A with the extracted body type data SI2 (step S119). . In step S119, as shown in FIG. 8, body type data SI2 may be extracted from the created CG data CG2 of user A by measuring the length (=YYY) of each part L11 to L18. The body data SI2 to be extracted may be CG data SI2-1 for body data, as shown in FIG. In step S119, information about the appearance of registered user A, such as the image of clothing and accessories worn, color, size, hair length, hairstyle, etc., is extracted from the photographed image of registered user A. may be registered in the personal data PD.

Note that the order of steps S117 and S119 is not limited to the order shown in FIG. 7, and may be reversed. In this case, the processor 10 may extract body shape data of the registered user A from the created CG data CG2, as shown in FIG.

The processor 10 repeatedly executes steps S113 to S119 during execution of the mirror mode program M11. As a result, the captured image is displayed on the display 13 as if it were a mirror, at least while the user is nearby, and the registered user CG is updated every time the registered user is captured. Therefore, the registered user CG represents the latest appearance of the user without performing any special operation or photographing action. In the example of FIG. 8, user A has changed in body shape (became thin) after user registration. In this regard, by repeating steps S113 to S119, the initial body type data L1 to L8 (=XXX) are replaced by body type data L11 to L18 (=YYY) in the latest state (thin state). is updated to

<Try-on operation>
In the try-on mode, the system 100 operates as shown in FIG. 9, and a try-on operation is performed in the system 100. 10 and 11 are displayed on the display 13 of the terminal device 1. FIG.

10 is displayed on the display 13 of the terminal device 1 (step S300). The try-on screen 40 shown in FIG. 10 includes a region 41 for selectively displaying clothes that can be tried on, a button 43 for instructing synthesis with CG corresponding to the try-on, and a button 45 for instructing the end of the try-on mode. ,including.

In the try-on screen 40, when the user selects clothes that he or she wishes to try on from among the clothes displayed in the area 41 (step S201), the processor 10 of the terminal device 1 selects from the personal data PD about the registered user: The transmission data to be transmitted to the management server 5 is read (step S202). The transmission data includes attribute data D14 and figure data D15. The processor 10 adds an ID for identifying the registered user to the transmission data, and transmits the data to the management server 5 as a request for data necessary for image synthesis (step S203). The request includes the user's identification data, attribute data, and body type data, as well as the identification data of the selected garment. It should be noted that the user identification data transmitted at this time may be data that allows the management server 5 to identify the user rather than the unique data of the user.

The management server 5 that has received the request executes a try-on management process (step S204). The try-on management process in step S204 includes selecting a specified clothing size suitable for the user (step S41) and transmitting CG data of the clothing of the selected size to the terminal device 1 (step S42).

The processor 10 of the terminal device 1 that has received the CG data of the clothes synthesizes the received CG of the clothes and the stored CG of the registered user (step S205), and displays it on the display 13 (step S206). As a result, the display on the display 13 is switched from the try-on screen 40 to the try-on screen 60 shown in FIG.

As shown in FIG. 10, the try-on screen 40 may further include an area 42 for selectably displaying options for furnishings other than clothes. Clothing accessories other than clothes are, for example, bags, hats, and shoes. In this case, the memory 51 of the management server 5 includes a database storing data on each of these furnishings, similar to the clothing data DB 511 . Processor 50 may randomly select one of them to display in area 42 when the try-on mode is initiated.

Alternatively, based on the image, color, etc. of the registered user's furnishings shown in the attribute data D14, the furnishings suitable for the registered user or the selected clothing among the clothing displayed in the area 41 may be selected. Appropriate furnishings may be extracted from the database and displayed in area 42 in a selectable manner. In this case, the processor 50 compares the attribute data D14 of the registered user or the attribute of the clothing in the selected state with the data representing the attribute corresponding to the size data D21 included in the clothing data. You can choose clothes that suit you. As a result, the system 100 can suggest accessories such as bags to the user of the terminal device 1 in combination with clothes. As a result, the registered user can enjoy seeing the image of the user trying on other clothing items that match the clothes or other clothing items that match him/herself, and the possibility of promoting sales of these clothing items increases.

Also, the try-on screen 40 may further include a button 44 for instructing display of the image in the keep state, as shown in FIG. When the button 44 is used to instruct the display of the image in the kept state, the processor 10 of the terminal device 1 requests the management server 5 for the CG data of the clothes in the kept state. Then, the CG data of the clothes kept in the kept state is obtained from the management server 5, synthesized with the stored CG of the registered user (step S205), and displayed on the display 13 (step S206). As a result, the display on the display 13 is switched from the try-on screen 40 to the try-on screen 60 shown in FIG.

The try-on screen 60 of FIG. 11 includes an area 61 for displaying a composite image of the CG of the clothes and the CG of the registered user, and a button 66 for returning the display to the try-on screen 40 of FIG. As a result, the registered user can automatically see the synthesized image with his/her own CG on the try-on screen 60 simply by selecting the clothes that the registered user wants to try on on the terminal device 1 .

In the try-on screen 60, a composite image using CG of a plurality of registered users may be displayed. An example of display of a composite image using CG of a plurality of registered users is display of a composite image of CG of a plurality of registered users representing a plurality of different poses of the registered user and CG of clothes. In this case, the try-on screen 60 includes, as shown in FIG. 11, an area 62 for selectably displaying options for other poses of the registered user's CG, and a button 63 for instructing pose change. . If multiple poses are included in the personal data PD as the registered user's CG data, the area 62 displays options for other poses. In the example of FIG. 11, the registered user CG facing forward is used for the display in the area 61, and the registered user CG in backward and sideways poses are selectably shown in the area 62. FIG.

When another pose is selected from the area 62 and an instruction to change the pose is given by the button 63, the processor 10 of the terminal device 1 repeats steps S205 and S206 to perform the registration user CG corresponding to the selected pose. A composite image is displayed in area 61 . As a result, the registered user can see images on the display 13 of the terminal device 1 similar to those when actually trying on clothes and taking various poses, and obtains a more accurate image of wearing the clothes. be able to.

Another example of display of a composite image using CG of a plurality of registered users is display of a composite image of each of a plurality of registered user CGs that are continuous in time series and CG of clothes that constitute a moving image. In this case, the try-on screen 60 further includes a button 67 for instructing video display, as shown in FIG. When the button 67 is pressed, a composite image of each of the plurality of registered user CGs and the CG of the clothes which are consecutive in chronological order is continuously displayed in the area 61 . As a result, in the area 61, a composite image of the registered user CG and the clothing CG is displayed as a moving image. With this display, the registered user can see an image similar to that of actually trying on the clothes and moving, on the display 13 of the terminal device 1, and can obtain a more accurate image of wearing the clothes. can be done.

In the try-on screen 60, when displaying a composite image using CG of a plurality of registered users, a composite image of a plurality of registered users' CG representing a plurality of different poses of the registered users and a moving image are displayed. Only one or both of the composite images with each of the registered user CGs that are continuous in series may be displayed.

The try-on screen 60 may further include a button 65 for instructing to keep the CG of the clothes used for synthesis, that is, the CG of clothes of a size suitable for the registered user. When the button 65 is pressed to enter the keep state (step S207), the processor 10 requests the management server 5 to enter the keep state (step S208). Preferably, when instructed to enter the keep state, the processor 10 displays, for example, a payment screen (not shown) on the display 13, and accepts an operation to pay the amount required to enter the keep state. , requests the management server 5 to keep the state. Upon receiving the request, the processor 50 of the management server 5 stores the CG data of the clothes transmitted to the terminal device 1 in the transmission data DB 512 of the memory 51 as the transmission data TD (step S209).

When the CG data of the clothes transmitted to the terminal device 1 is stored in the transmission data DB 512, the management server 5 is instructed to display an image in which the button 44 is pressed on the try-on screen 40 of FIG. The previously transmitted CG data of clothes can be transmitted to the terminal device 1 again. At this time, the processor 50 of the management server 5 refers to the transmission data TD and determines whether or not the CG data of the clothes can be transmitted based on the fee paid and the period of keeping. Thereby, the registered user who has once displayed the composite image on the terminal device 1 can keep the clothes in a keep state if he/she does not purchase the clothes immediately, and can display the clothes on the display 13 again later. As a result, it is possible to sufficiently consider purchasing, and sales of clothes may be promoted.

The try-on screen 60 may further include a button 64 for instructing purchase of the selected garment. When the purchase is instructed by the button 64 (step S210), the processor 10 of the terminal device 1 may execute purchase processing (step S211). Then, the purchase request generated by the purchase process is transmitted to the management server 5 (step S212). Processor 50 of management server 5 that has received the request executes purchase processing (step S213). A purchase operation in the system 100 performed by the terminal device 1 and the management server 5 executing purchase processing will be described in a second embodiment.

[Second embodiment]
<System configuration>
The system 100 according to the second embodiment further includes a sales company server 6 owned by a clothing sales company, as shown in FIG. The sales company server 6 communicates with the terminal device 1 and the management server 5 via the Internet 9 .

<Sales action>
In the system 100 according to the second embodiment, which includes the sales company server 6, the sales operation using the sales method shown in FIG. 12 is performed. Referring to FIG. 12, when a purchase instruction is given by button 64 on try-on screen 60 of FIG. 11 (step S301), processor 10 of terminal device 1 executes transfer processing (step S302). The transfer process in step S302 is a process of paying expenses including the price of the clothes to be ordered to the management server 5, and may be a general electronic settlement process. The amount to be paid may include a handling fee and a shipping fee in addition to the product price.

When the transfer process is completed, the processor 10 of the terminal device 1 transmits order data to the management server 5 (step S303). The order data includes, in addition to the identification data of the clothes to be ordered, user data indicating user identification data, address, etc., and data that can specify the transfer process.

Processor 50 of management server 5 that has received the order data executes transfer processing (step S304). The transfer process in step S304 is a process of transferring the price of the clothing paid by the terminal device 1 to the management server 5 to the sales company server 6. It may be a process to set the state. At this time, the amount transferred from the terminal device 1 may be transferred to the sales company server 6 after subtracting a predetermined fee.

When the transfer process is completed, the processor 50 of the management server 5 transmits the order data to the sales company server 6 (step S305). Also in step S305, data similar to that in step S303 is transmitted.

After receiving the order data, the sales company server 6 announces the order to the sales company staff by, for example, displaying the contents of the order data, and the ordered clothes are shipped to the registered user.

When the sales company server 6 receives the operation of inputting the shipping completion (step S306), it transmits a shipping notification to the management server 5 (step S307). In addition to data indicating that the package has been shipped, the shipping notification preferably includes identification data (ID) of the package, data indicating the date and time of shipping, and the like. The shipping notification is transferred from the management server 5 to the terminal device 1 (step S308).

By performing the sales operation described above, the registered user can purchase the desired clothes by instructing the purchase from the try-on screen 60 and making a payment to the management server 5 . Moreover, since sales are mediated by the management server 5, highly reliable transactions can be performed. In addition, since the management server 5 mediates sales, the sales company server 6 can reliably receive payment. That is, in the present system 100, exchange of data relating to product sales between the terminal device 1 and the sales company server 6 is mediated by the management server 5, so highly reliable transactions can be performed.

The selling operation shown in FIG. 12 can be performed in the same manner even if the sales company server 6 is replaced with another terminal device 1 used by another user. This is assumed to be a flea market, an auction, or the like where general users sell clothing and accessories. In this case as well, the management server 5 mediates so that both the user on the seller's side and the user on the buyer's side can conduct transactions with high reliability. FIG. 13 shows the selling operation in this case, in which the management server 5 establishes the connection between the terminal device (buying user terminal device) 1A on the buying user side and the terminal device (selling user terminal device) 1B on the selling user side. mediate the exchange of data in

Specifically, referring to FIG. 13, when a purchase instruction is given by button 64 on try-on screen 60 of FIG. 11 (step S401), processor 10 of purchase user terminal device 1A executes transfer processing (step S402). The transfer processing in step S402 is the same as the transfer processing in step S302. When the transfer process is completed, the processor 10 of the purchaser user terminal device 1A transmits the order data to the management server 5 (step S403).

The processor 50 of the management server 5 that has received the order data skips the transfer processing (step S304) in FIG. 12 in the sales operation between users, and transmits the order data to the selling user terminal device 1B (step S404). In the selling operation of FIG. 13, the transfer process for transferring the price is executed after the product is shipped from the selling user side.

The selling user terminal device 1B, which has received the order data, receives the operation of inputting the shipping completion (step S405), and transmits a shipping notification to the management server 5 (step S406). The shipping notification is transferred from the management server 5 to the purchase user terminal device 1A (step S407).

In addition, the management server 5, which has received the shipping notification from the seller user terminal device 1B, executes transfer processing for payment of the price to the seller user terminal device 1B (step S408). This transfer processing is the same as the transfer processing in step S304 of FIG. The management server 5 that executed the transfer process transmits the transfer data to the seller user terminal device 1B (step S409), and ends the series of operations. As a result, it is possible to effectively prevent the trouble that the product from the selling user is not shipped after the payment is transferred.

By performing the sales operation described above, even if the transaction is between users, the registered user can purchase the desired clothes easily by instructing the purchase from the try-on screen 60 and making the payment to the management server 5. Since the management server 5 mediates the sale, highly reliable transactions can be performed.

[Third Embodiment]
The terminal device 1 may have other operation modes in addition to the mirror mode and try-on mode described above. Other modes of operation include, for example, a viewing mode realized by processor 10 executing program M13 shown in FIG.

The viewing mode is a mode in which an image other than the captured image from the camera 14 is displayed on the display 13 . The viewing mode may be executed when an operation mode switching instruction is given by the input device 15 . As a result, the mode is switched to the viewing mode at the timing specified by the user. Alternatively, viewing mode may switch upon detection of a particular event while in mirror mode. The specific event is, for example, when no person is detected from the captured image for a specific period of time, or when no change is detected in the captured image for a specific period of time. As a result, when the specific event is detected during the mirror mode, the mode is automatically switched to the viewing mode without requiring a special user operation, thereby enhancing convenience.

An image other than the image captured by the camera 14 may be an image captured by another camera installed remotely from the terminal device 1, for example. Other cameras may be designated by user operation on the input device 15 . For example, in viewing mode, cameras installed in foreign countries, cameras installed in places where people cannot enter, such as unexplored areas, cameras installed in show spaces such as theaters, stages, and movie theaters, and languages such as English conversation A camera or the like installed in a lecture space such as a classroom or a university classroom may be selectable. As a result, when the terminal device 1 is installed at home, for example, the user can enjoy foreign landscapes and unexplored landscapes while staying at home. In addition, it is possible to appreciate a show in a theater, a stage, a movie theater, or the like, and enjoy a sense of realism on the spot. In addition, you can enjoy various lectures while staying at home. In particular, since the display 13 is a life-sized display, the display is large. Therefore, the user can feel as if he or she is in a foreign country, an unexplored region, a show space, or a lecture space while staying at home.

Also, if the terminal device 1 has a speaker, the sound may also be output in the viewing mode. The audio may be audio, such as regular music. This allows the user to enjoy images and sounds. As another example, when a microphone is installed near another designated camera, the terminal device 1 may output the sound input to the microphone near the other camera. As a result, the user can experience a sense of realism in a foreign country where other cameras are installed, or in an unexplored region. In particular, when watching a show or taking a lecture, the voice can be obtained at the same time, so that these services can be effectively enjoyed remotely.

Also, if the terminal device 1 further has a microphone, audio may be input/output in the viewing mode. As a result, when a microphone and a speaker are installed near another specified camera, the user of the terminal device 1 can, while staying at the position where the terminal device 1 is installed, such as at home, users can enjoy video and conversation in real time. In particular, in the case of lectures, questions and remarks can be made, and these services can be effectively enjoyed remotely.

Also, images other than images captured by the camera 14 may be, for example, distributed contents such as movies and TV programs. As a result, the user of the terminal device 1 can enjoy the content on the life-size display 13 at the location where the terminal device 1 is installed, such as at home.

[Other example 1]
As another example, the processor 10 of the terminal device 1 may select a specified clothing size. In this case, the processor 10 of the terminal device 1 further includes a size selector 106 indicated by dotted lines in FIG. In this case, the processor 50 of the management server 5 does not perform the process of selecting the size in step S203, and stores the CG data of all sizes prepared for the clothes specified by the request in step S202 in step S204. Send to device 1. This simplifies the processing in the processor 50 of the management server 5 and eliminates the need to transfer the body shape data of the registered user from the terminal device 1 to the management server 5 . Therefore, security can be further enhanced.

[Other example 2]
As another example, when body data CG is transmitted from the terminal device 1 to the management server 5 as body data, the processor 50 of the management server 5 generates the CG of the clothes of the selected size and the body data CG of the registered user. may be performed. In this case, the processor 50 of the management server 5 further includes an image synthesizing unit 505 indicated by dotted lines in FIG.

In this case, the processor 50 of the management server 5 transmits the composite image data to the terminal device 1 in step S204. The synthetic image to be sent does not include the face image of the registered user. Therefore, in step S<b>205 , the processor 10 of the terminal device 1 performs a process of synthesizing the face image of the registered user with the synthetic image received from the management server 5 . As a result, since the management server 5 can select the size using more detailed body shape data of the registered user, it is possible to select a more suitable size. Since it is not transmitted to the management server 5, security can also be ensured.

[Other example 3]
In the above example, CG data D22 to D24 of a plurality of sizes are prepared in advance for each piece of clothing, and for each size, the range of suitable sizes for each part of the user is defined by the size data D21 (see FIG. 5). ). For this reason, the management server 5 selects a size that fits within the range defined by the registered user's figure data as a size suitable for the user.

On the other hand, furnishings such as clothes, shoes, and hats may be created or partially modified (ordered or semi-ordered) according to the user's size. In this case, the size selection unit 501 of the processor 50 of the management server 5 determines the size of each part of the clothing according to the user's body shape data, and determines the size of the stored clothing (regular size) CG. change to be The relationship between the size of each part of the registered user and the part to be changed in the CG of the clothes and its size may be a general relationship in the order of clothes. In step S<b>204 above, the processor 50 transmits the changed CG data to the terminal device 1 . Accordingly, in step S205, the CG data of the clothes whose size has been changed according to the registered user's body type data is used for the synthesis process. Therefore, even when clothing and accessories such as clothes are made to order, the system 100 can be used to see the image of the fitting on the display 13 .

[Other example 4]
As another example of the input device 15 of the terminal device 1, a remote controller (remote controller) 2 may be provided instead of or in addition to the input device 15 as shown in FIGS. The remote controller 2 performs wireless communication with the terminal device 1 and inputs an operation signal indicating a user operation to the terminal device 1 .

The operation unit of the remote controller 2 may be a touch panel 21 as shown in FIGS. In this case, the processor 10 of the terminal device 1 may display the try-on mode on the touch panel 21 of the remote controller 2 instead of or in addition to the display 13 . In this case, the display control unit 104 performs a process of transforming (reducing) the screen displayed on the display 13 so as to fit the size of the touch panel 21 . Thereby, the user can enjoy the image of the state of trying on at a desired position without being restricted by the installation position of the terminal device 1 . In addition, in the mirror mode, the display is performed at least on the display 13 and may be displayed on the touch panel 21 as well. As a result, in the mirror mode, the user can see his or her own appearance on the life-size display 13, and the terminal device 1 can be used in the same way as a full-length mirror.

Note that the remote controller 2 may be a dedicated device, or may be a user's mobile device such as a smart phone or a tablet terminal. When a user's portable device is used as the remote controller 2, a program (application) is installed in the portable device to cause the processor to function as the remote controller 2. FIG. The program may be pre-installed in the mobile device, or may be downloaded via communication such as the Internet.

[Fourth Embodiment]
The above description is an example in which the registered user image is the registered user CG and the clothing image is CG. As another example, the registered user image may be a photographed image (photograph) of the registered user. Also, the image of the clothes may be, as another example, a photographed image (photograph) of the clothes. That is, both the registered user image and the clothing image may be CG, or only one (the registered user image or the clothing image) may be CG and the other may be a photographed image, or both may be photographed. It may be an image. As a result, even in a state in which no CG has been created, an image in which the user wears furnishings such as clothes can be confirmed on the display 13 by using a synthesized image using the photographed image.

The terminal device 1 performs processing for synthesizing the registered user image and the clothing image, and displays the synthesized image on the display 13 . If the registered user image is the registered user's captured image and/or the clothing image is CG, the terminal device 1 replaces the registered user CG with the registered user's captured image and/or Alternatively, similar processing is performed by replacing the CG of the clothes with the photographed image of the clothes. In this case, the synthesized image of the registered user's photographed image and the image of the clothes may be the following four types (1) to (4). Regardless of the combination of images, the combined image allows the user to confirm the image of the person wearing furnishings such as clothes on the display 13 .
(1) Composite image of CG of registered user and CG of clothes (2) Composite image of CG of registered user and photographed image of clothes (3) Composite image of photographed image of registered user and CG of clothes (4) Composite image of registered user's photographed image and clothing photographed image

The present invention is not limited to the above embodiments, and various modifications are possible.

1 : Terminal device 1A : Terminal device 1B : Terminal device 2 : Remote controller 3 : Massage chair 4 : Server 5 : Management server 6 : Sales company server 7 : Wireless communication 7A : Wireless communication 9 : Internet 10 : Processor 11 : Memory 12 : Communication Device 13 : Display 14 : Camera 15 : Input device 21 : Touch panel 40 : Try-on screen 41 : Area 42 : Area 43 : Button 44 : Button 45 : Button 50 : Processor 51 : Memory 52 : Communication device 60 : Try-on screen 61 : Area 62 : Area 63 : Button 64 : Button 65 : Button 66 : Button 67 : Button 100 : System 101 : CG creation unit 102 : Transmission processing unit 103 : Image composition unit 104 : Display control unit 105 : Purchase processing unit 106 : Size selection unit 107: Extraction processing unit 111: Personal data DB
112: Program 501: Size selection unit 502: Transmission processing unit 504: Purchase processing unit 505: Image synthesizing unit 511: Clothing data DB
512: Transmission data DB
513: Program A: User (registered user)
CD: Clothes data CD1: Clothes data CD2: Clothes data CD3: Clothes data CG1: CG data CG2: CG data D11: Photographed image data D12: User identification data D13: Registered user CG data D14: Attribute data D15: Body shape data D21: Size data D23 : CG data D24 : CG data D31 : Clothing data D32 : User identification data D33 : Billing data IM1 : Photographed image IM2 : Photographed image L1 : Part L11 : Part L12 : Part L13 : Part L14 : Part L15 : Part L16 : Site L17 : Site L18 : Site L2 : Site L3 : Site L4 : Site L5 : Site L6 : Site L7 : Site L8 : Site M11 : Program M12 : Program M13 : Program M21 : Program M22 : Program PD : Personal data PD1 : Personal data PD2: Personal data PD3: Personal data R: Shooting range SI2-1: CG data for body data SI1: Body data SI1-1: CG data for body data SI2: Body data TD: Transmission data TD1: Transmission data TD2: Transmission data TD3: transmission data

Claims (20)

a terminal device;
a management server;
a memory area accessible from the terminal device and inaccessible from the management server for storing personal data including a registered user image of a registered user who is a user of the terminal device;
The terminal device transmits the personal data excluding at least the registered user image to the management server;
The management server stores the personal data excluding the registered user image transmitted from the terminal device in association with the identification information of the registered user;
executing a try-on management process using the personal data from the terminal device;
The fitting management process includes transmitting product data for displaying an image of the product to the terminal device,
The product fitting system, wherein the terminal device displays an image obtained by synthesizing an image of the product and the registered user image using the product data. The product try-on system according to claim 1, wherein the image of the product is CG (computer graphics) of the product. The product fitting system according to claim 1 or 2, wherein the registered user image is a registered user CG of the registered user. 4. The product fitting system according to claim 3, wherein the terminal device creates the registered user CG using a plurality of captured images from a plurality of directions including the face of the registered user, and stores the CG as the personal data in the memory area. . the terminal device includes a camera and a display;
5. According to claim 4, when the image of the registered user is detected from the image captured by the camera, the terminal device acquires body type data of the registered user from the captured image, and further uses the data to create the registered user CG. Product try-on system as described. 6. The product fitting system according to claim 5, wherein, when acquiring body shape data of said registered user from said photographed image, said terminal device updates said registered user CG stored in said memory area using said body shape data. 7. The terminal device according to claim 5 or 6, wherein, when a predefined image of the registered user is included in the photographed image, the terminal device acquires body type data of the registered user from the predefined image. Product try-on system. 8. The product fitting system according to any one of claims 5 to 7, wherein the terminal device further acquires information about the appearance of the registered user from the captured image, and stores the information in the memory area as the personal data. The terminal device has a mirror mode and a try-on mode as operation modes,
displaying an image captured by the camera on the display in the mirror mode;
The product fitting system according to any one of claims 5 to 8, wherein in the fitting mode, the registered user CG and the product CG are synthesized and displayed. Of the display used for display in the try-on mode and the display used for display in the mirror mode, at least the display used for display in the mirror mode of the terminal device is the full body of the registered user. 10. The product fitting system according to claim 9, wherein the size is displayed in large. 11. The product according to any one of claims 3 to 10, wherein the terminal device reads user attribute information other than the registered user's physical data from another device, and further uses the attribute information to create the registered user CG. try-on system. The terminal device includes a camera and a display,
As an operation mode, having a mirror mode for displaying an image captured by the camera on the display,
12. The terminal device according to any one of claims 5 to 11, wherein the operation mode can be switched from the mirror mode to another mode for displaying an image other than the image captured by the camera. product try-on system. 13. The product fitting system according to claim 12, wherein the operation in said other mode includes displaying an image captured by a camera other than said camera. further comprising a size selection unit that selects the size of the product based on the personal data of the registered user;
14. The product fitting system according to any one of claims 1 to 13, wherein the terminal device displays an image obtained by synthesizing an image of the product corresponding to the selected size and the registered user image. The personal data includes a plurality of registered user images,
According to any one of claims 1 to 14, wherein the plurality of registered user images are images representing a plurality of different poses of the registered user and/or consecutive images constituting a moving image. Product try-on system as described. 16. The terminal device according to any one of claims 1 to 15, wherein the terminal device transmits to the management server excluding at least the registered user image, the registered user name of the registered user, and the photographed image from the personal data. Product try-on system as described. A terminal device capable of communicating with a management server,
comprising a processor capable of accessing a memory area not permitted to be accessed by the management server;
the memory area stores personal data including a registered user image of a registered user;
The processor
transmitting the personal data excluding at least the registered user image to the management server;
displaying an image obtained by synthesizing the image of the product and the registered user image using the product data for displaying the image of the product received from the management server;
The management server stores the personal data excluding the registered user image transmitted from the terminal device in association with the identification information of the registered user. a communication device capable of communicating with a terminal device;
a processor;
The terminal device comprises a memory area accessible from the terminal device, the memory area stores personal data including a registered user image of a registered user who is a user of the terminal device,
The communication device receives personal data that does not include the image of the registered user of the terminal device from the terminal device,
storing the personal data in association with the identification information of the registered user;
The processor executes try-on management processing using the personal data,
The fitting management process includes transmitting product data for displaying an image of the product to the terminal device,
Access to said memory area is not allowed
Management server. A product sales method in a product fitting system including a user terminal device, a management server, and a sales company server on the product sales side,
The product try-on system is a memory that stores personal data including a registered user image of a registered user who is a user of the terminal device and is accessible from the terminal device but inaccessible from the management server and the sales company server. has
a step of transmitting personal data of the registered user excluding the registered user image from the terminal device to the management server, and storing the personal data excluding the registered user image in association with identification information of the registered user; ,
a step of transmitting product data for displaying an image of a product from the management server to the terminal device;
a step of synthesizing an image of the product and the registered user image stored in the memory in the terminal device using the product data, and displaying the synthesized image;
a step of receiving, in the terminal device, a user operation instructing purchase of the product based on the display of the composite image;
a step of transmitting order data for the product from the terminal device to the management server;
and transmitting the order data and data identifying the registered user from the management server to the sales company server. a step of transmitting a shipping notification indicating completion of shipping of the product to the registered user from the sales company server to the management server;
20. The product sales method according to claim 19, further comprising a step of transferring said shipping notification from said management server to said terminal device.

Images