JP7463455B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

Traditionally, shoes have been made to fit wooden lasts handcrafted by craftsmen, which has led to a lack of standardization. To achieve optimal matching, it is necessary to know the inside dimensions of the shoe, taking into account the shoe's stretch and deformation, but with traditional methods it is not possible to quantitatively determine the degree of shoe stretch and deformation. Also, because there is a large margin of error, it is necessary to use destructive methods, which are costly.

JP 2012-033128 A

Therefore, there is a need for a new method to easily and fairly measure items such as shoes, whose shapes are not standardized.

The present invention was made in light of these circumstances, and aims to provide a new method that allows for equal and easy matching even for items whose shapes are not standardized.

In order to achieve the above object, an information processing device according to one aspect of the present invention comprises:
The device is equipped with a measurement means that identifies a facing position based on measurement data of the item, detects one or more landmark points, calculates a reference symbol based on predetermined rules for eliminating the design of the item, calculates each measurement value of a plurality of items based on the facing position, the landmark points, and the reference symbol, and generates information including at least each of the measurement values of the plurality of items as measurement data of the item.

The present invention makes it possible to perform matching fairly and easily even for items whose shapes are not standardized.

1 is a block diagram showing an example of the configuration of an information processing system including an information processing device to which the present invention is applied. Fig. 2 is a conceptual diagram showing a schematic diagram of processing such as machine learning, which is a premise of matching in the present service to which the information processing system shown in Fig. 1 is applied. Fig. 3 is a configuration diagram of an information processing system including a matching device which is an embodiment of the information processing device of the present invention. FIG. 2 is a conceptual diagram showing a schematic diagram of processing such as matching in the present service to which the information processing system shown in FIG. 1 is applied. FIG. 2 is a block diagram showing an example of a hardware configuration of a matching device 5 in the information processing system of FIG. 1 capable of realizing this service (matching). 2. This is a functional block diagram showing an example of a functional configuration capable of realizing the machine learning processing shown in FIG. 2 among the processing executed by the information processing system 1 of FIG. 1 including the matching device 5 having the above hardware configuration. FIG. 1 is a diagram showing a schematic external configuration of a foot to explain a standardized foot measurement method. FIG. 1 is a diagram showing an outline of the external configuration of a shoe and a shoe anchor to explain the standardized shoe measurement method. 4 is a functional block diagram showing an example of a functional configuration capable of implementing the matching process shown in FIG. 3 among the processes executed by the information processing system 1 in FIG. 1 . FIG. 11 is a diagram illustrating an example of a matching method determined based on the results of machine learning. 4 is a diagram showing an overview of a process that can be performed by the information processing system of FIG. 1 and that is different from the example of FIG. 3. 3 is a diagram showing an overview of a process that can be performed by the information processing system of FIG. 1 and that performs machine learning different from the example of FIG. 2 . 13 shows an example of measurement data of a user U (including a tester T) when standardized measurements are performed with the item being clothing. 1 shows an example of measurement data of an item when standardized measurements are performed, assuming that the item is clothing (jacket). This is an example of a mockup of an EC site screen. This is an example of a mockup of an EC site screen. This is an example of a mockup of an EC site screen.

The following describes an embodiment of the present invention with reference to the drawings.

FIG. 1 is a block diagram showing an example of the configuration of an information processing system including an information processing device to which the present invention is applied.
The information processing system shown in Figure 1 is configured to include a shoe measurement device 1, a tester foot measurement device 2, a learning device 3, a user foot measurement device 4, a matching device 5, a user terminal 6, a shoe measurement DB 11, a foot measurement DB 12, and an evaluation DB 13.
The shoe measurement device 1 to the user terminal 6 and the shoe measurement DB 11 to the evaluation DB 13 are connected to each other via a predetermined network N such as the Internet.

The following provides an overview of a service (hereinafter, referred to as "this service") that is realized using the information processing system of FIG.
This service allows for easy and fair measurement of items (e.g., shoes) that do not have a standardized shape, and allows matching with users who wish to purchase them (recommendation of recommended shoes, etc.). As described below, this service allows for easy and fair matching, since it abstractly captures the tendencies seen in the shape of the user's body and the shape of the item, without necessarily comparing the same positions. Furthermore, since it statistically captures errors during measurement, it allows for practical matching, even with measurements taken using inexpensive sensors that are prone to errors.
The items to which this service applies are not particularly limited as long as they are worn on at least a part of the user's body, but in the following example, they are shoes.

Figure 2 is a conceptual diagram that shows the machine learning and other processes that are the basis for matching in this service to which the information processing system shown in Figure 1 is applied.

Here, when modeling the matching target, measurements may be taken using actual shoes, but in this embodiment, m shoe anchors Sh1 to Shm (m is an integer value of 1 or more) are used. When it is not necessary to distinguish between the shoe anchors Sh1 to Shm individually, they will hereinafter be collectively referred to as "shoe anchor Sh."
In step Sa, the shoe measurement device 1 scans each of the shoe anchors Sh1 to Shm to generate each of the 3D scan data Ds1 to Dsm.
In step Sb, the shoe measurement device 1 performs measurements on each of the 3D scan data Ds1 to Dsm in accordance with a shoe measurement method (details of which will be described later, but hereinafter referred to as the "standardized shoe measurement method") devised by the inventors and adopted as the standard in this embodiment, and stores each of the data obtained as a result of the measurements (hereinafter referred to as "shoe measurement data") Ks1 to Ksm in the shoe measurement DB 11.
In the following description, when there is no need to distinguish between the shoe measurement data Ks1 to ksm, they will be collectively referred to as "shoe measurement data Ks."

Meanwhile, n natural persons T1 to Tn (n is an integer value of 1 or more) are each asked to actually wear m shoes and evaluate them. Hereinafter, each of these natural persons T1 to Tn will be referred to as "tester T1 to Tn." When it is not necessary to distinguish between the testers T1 to Tn individually, they will be collectively referred to as "tester T."
In step Sc, the tester foot measuring device 2 scans both feet of each of the testers T1 to Tn to generate 3D scan data Dt1 to Dtn, respectively.
In step Sd, the tester foot measurement device 2 performs measurements on each of the 3D scan data Dt1 to Dtn in accordance with a foot measurement method (details of which will be described later, but hereinafter referred to as the "standardized foot measurement method") devised by the present inventors and adopted as the standard in this embodiment, and stores each of the data obtained as a result of the measurements (hereinafter referred to as "foot measurement data") Kt1 to Ktm in the foot measurement DB 12.
In the following, when there is no need to distinguish between the foot measurement data Kt1 to ktn, they will be collectively referred to as "foot measurement data Kt".
In addition, for each tester T1 to Tn, evaluation data Ht1 to Htn on the comfort of each of the m shoes when actually worn is obtained (hereinafter simply referred to as “evaluation data”) and stored in the evaluation DB 13.
In the following, when there is no need to distinguish between the evaluation data Ht1 to Htn, they will be collectively referred to as "evaluation data Ht."

The learning device 3 performs machine learning using each combination of shoe measurement data Ks1 to Ksm of m shoes (shoe anchors Sh1 to Shm) and foot measurement data Kt1 to Ktn of n testers T1 to Tn, and evaluation data Ht1 to Htn for each of the m shoes of the n testers T1 to Tn.
As a result of this, for example, a trend in comfort can be obtained for each of various combinations of a specific foot and a specific shoe as a learning result. It should be noted here that the shoe measurement data Ks and the foot measurement data Kt do not necessarily need to include data measured at the same position. In other words, even if measurements are taken at different positions, machine learning can provide a trend in comfort for each of various combinations of a specific foot and a specific shoe.
By using such learning results, it becomes possible to abstractly grasp the tendencies of the respective shapes of the feet and shoes, making it possible to perform matching (details will be described later) without necessarily having to compare the same positions.
In addition, the use of such machine learning techniques makes it possible to handle measurement errors statistically, enabling practical matching even when scanning with inexpensive sensors that have large errors.

Figure 3 is a conceptual diagram that shows a schematic diagram of the matching and other processing in this service to which the information processing system shown in Figure 1 is applied.

In step Sf, the user's foot measurement device 4 scans both feet of the user U to generate 3D scan data Du.
In step Sg, the user's foot measurement device 4 performs measurements on the 3D scan data Du in accordance with the standardized foot measurement method, and stores the foot measurement data Ku obtained as a result of the measurements in the foot measurement DB 12 .

In step Si, the matching device 5 performs matching (to estimate) which shoes will be comfortable for the user U, using the foot measurement data Ku of the user U obtained in step Sg and each of the shoe measurement data Ks1 to Ksm of the m shoes (shoe anchors Sh1 to Shm) described above using Figure 2.
Here, in addition to simply comparing the measurement data, the matching device 5 may acquire the preferences of the user U in step Sh and perform matching taking the preferences into consideration in step Si. This enables matching suited to each user U, such as when the user U prefers tight shoes, even if size A shoes are generally determined to be comfortable, size B smaller than size A may be determined to be suitable for the user U.
In step Sj, the matching device 5 recommends shoes suitable for the user U (shoes that the user U will find comfortable to wear) to the user U (user terminal 6) based on the matching results of step Si.

If necessary, the user U evaluates the comfort of the recommended shoes (shoe anchors Sh: h is an integer value between 1 and m) when actually wearing them. In this case, the evaluation data Hu of the user U is stored in the evaluation DB 13.
Then, when the evaluation data Hu of the user U, the foot measurement data Ku of the user U, and the shoe measurement data Ksk of the recommended shoe (shoe anchor Sh) are provided to the learning device 3, the learning device 3 performs machine learning (relearning) in step Sk. In this way, the actual recommendation results and their evaluations are fed back and relearning is performed, thereby enabling more accurate matching.

Figure 4 is a block diagram showing an example of the hardware configuration of the matching device 5 in the information processing system of Figure 1 that can realize this service (matching).

The matching device 5 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a bus 24, an input/output interface 25, an output unit 26, an input unit 27, a memory unit 28, a communication unit 29, and a drive 30.

The CPU 21 executes various processes according to a program recorded in the ROM 22 or a program loaded from the storage unit 28 to the RAM 23 .
The RAM 23 also stores data and the like necessary for the CPU 21 to execute various processes.

The CPU 21, ROM 22, and RAM 23 are interconnected via a bus 24. An input/output interface 25 is also connected to this bus 24. An output unit 26, an input unit 27, a memory unit 28, a communication unit 29, and a drive 30 are connected to the input/output interface 25.

The output unit 26 is composed of a display, a speaker, etc., and outputs various information as images and sounds.
The input unit 27 is composed of a keyboard, a mouse, etc., and is used to input various information.

The storage unit 28 is composed of a hard disk, a DRAM (Dynamic Random Access Memory), etc., and stores various data.
The communication unit 29 communicates with other devices via a network N including the Internet.

Removable media 31, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is appropriately loaded into the drive 30. A program read from the removable media 31 by the drive 30 is installed in the storage unit 28 as necessary.
Further, the removable medium 31 can store various data stored in the storage unit 28 in the same manner as the storage unit 28 .

The shoe measurement device 1, the tester foot measurement device 2, the learning device 3, the user foot measurement device 4, and the user terminal 6 can each have the same hardware configuration as that shown in FIG. 4. Therefore, a description of these hardware configurations will be omitted.

Figure 5 is a functional block diagram showing an example of a functional configuration capable of implementing the machine learning process shown in Figure 2 and other processes executed by the information processing system 1 of Figure 1, which includes a matching device 5 having such a hardware configuration.

As shown in FIG. 5, in the shoe measurement device 1, a scanning unit 111 and a standardized shoe measurement unit 112 function.
In the tester foot measurement device 2, a scanning unit 121, a standardized foot measurement unit 122, and an evaluation acquisition unit 123 function.

The scanning unit 111 of the shoe measurement device 1 generates 3D scan data Ds by scanning the shoe anchor Sh.
The standardized shoe measurement unit 112 performs measurements on the 3D scan data Ds according to the standardized shoe measurement method, and stores the shoe measurement data Ks obtained as a result of the measurements in the shoe measurement DB 11.
Here, the shoe measurement data Ks of the shoe (or shoe anchor Sh) can be linked to various information related to the shoe and stored in the shoe measurement DB 11. Specifically, for example, the 3D scan data Ds can be linked to the shoe measurement data Ks. Also, for example, information on the type of material for each part of the shoe and information on the properties of the material such as tensile strength (specific strength) can be linked to the shoe measurement data Ks.

Meanwhile, the scanning unit 121 of the tester foot measuring device 2 scans both feet of the tester T to generate 3D scan data Dt.
The standardized foot measurement unit 122 performs measurements on the 3D scan data Dt in accordance with the standardized foot measurement method, and stores the foot measurement data Kt obtained as a result of the measurements in the foot measurement DB 12.
Here, various information related to the tester T can be linked to the foot measurement data Kt of the tester T. Specifically, for example, the 3D scan data Dt can be linked to the foot measurement data Kt. In addition, for example, information such as gender, age, weight, average number of steps per day, shoe purchase history and return history, shoes often worn, shoes not interested, favorite brands, disliked brands, and various other information can be linked to the foot measurement data Kt. Furthermore, for example, information such as the address and occupation of the tester T may be linked to the foot measurement data Kt. Note that when a large amount of foot measurement data Kt and the above-mentioned shoe measurement data Ks are obtained, these data can be used as big data for various purposes such as various analyses.
The evaluation acquisition unit 123 also acquires evaluation data Ht indicating evaluations of the comfort of each shoe when the tester T actually wears each shoe, and stores the evaluation data Ht in the evaluation DB 13 .

The standardized foot measurement technique will now be described with reference to FIG. 6, followed by the standardized shoe measurement technique with reference to FIG.
FIG. 6 is a diagram showing a schematic external configuration of a foot for explaining the standardized foot measurement method.
FIG. 7 is a diagram showing the outline of the external configuration of a shoe and a shoe anchor for explaining the standardized shoe measurement method.

The standardized foot measurement unit 122 of the tester foot measurement device 2 performs measurements according to the standardized foot measurement method, for example, as follows.

The standardized foot measurement unit 122 identifies the “frontal position.”
That is, the standardized foot measurement unit 122 uses PCA (principal component analysis) on the point cloud of the 3D scan data Dt of the tester T's foot to find the direction in which the data distribution is wide and uniquely determines the facing direction. This eliminates the need for the tester T to strictly align the orientation of his or her foot during measurement.
In other words, in order to extract and compare lengths and angles according to a set rule from feet that can have various shapes, it is first necessary to determine a reference axis direction. Therefore, the standardized foot measurement unit 122 specifies "front facing" by defining the axis along which the foot footprint shapes are most widely distributed in the 3D scan data Dt as the forward direction axis, taking the direction of gravity as the up-down axis, and defining the axis perpendicular to the forward direction axis and the up-down axis as the left-right axis.

The standardized foot measurement unit 122 detects the "landmark point L."
That is, the standardized foot measurement unit 122 uses the shape characteristics of the foot to detect landmark points L such as the apex of the toe and heel. Points that are difficult to extract from shape characteristics alone (ball joints (axis of the toe), base of the big toe, and ends of the big toe and little toe) are determined by attaching colored markers to the feet of the tester T during measurement, and the standardized foot measurement unit 122 determines them based on these.

The standardized foot measurement unit 122 determines the "measurement value."
That is, the standardized foot measurement unit 122 performs geometric calculations based on the landmark points L obtained in this manner, and treats the values obtained by this calculation as standardized measurement values.

The following measurements are used:
"Foot circumference A": The circumference of the foot when it is sliced at the ball joint J (L).
"Foot width B": The straight-line distance between ball joints J (L).
"Foot length C": The length on the floor plane between the toe (the most forward point) and the top of the heel (the most rearward point).
"Heel shape D", "Toe height E", "Toe shape F", and "Distortion G".
In addition, the following measurements may be used as necessary:
"Heel length": A constant ratio of the length of the foot C.
"Heel width": The width of the foot when extended forward the length of the heel from the rearmost point.
"Heel depth": The distance (depth) from the point extending vertically upward from the heel vertex a specified length to the foot.
"Toe width": The distance between the base of the thumb and the base of the little finger.
"Height of the base of the thumb, thumb, and little finger": The height from each landmark point L to the floor surface.

In response to this, the standardized shoe measurement unit 112 of the shoe measurement device 1 executes measurement according to the standardized shoe measurement method, for example, as follows.
That is, there are various conventional methods for measuring the inside dimensions of shoes (CT imaging, making a mold using plaster, measuring a mold (wooden last) at the time of manufacture, etc.). However, when using machine learning as in this embodiment, it is sufficient to know only the relative differences that characterize the individual, so it is sufficient to obtain the following "measurement values". In other words, any method (various methods) for obtaining the following "measurement values" is a standardized shoe measurement method.

When determining the "measurement value," the standardized shoe measurement unit 112 identifies the "frontal facing" and detects the "landmark point L" in the same manner as in the standardized foot measurement method described above with reference to Figure 6.

The standardized shoe measurement unit 112 also determines the next reference symbol (line, surface).
The following standard symbols are required according to the following rules to eliminate shoe design.
"Toe position"
(Rule) Due to the design, there is a hollow portion into which the foot does not enter, so in order to eliminate the effect of this portion, the position where the foot circumference A becomes constant (for example, 100 mm) is set as the toe position.
"Ball joint position (the joint between the toes and heels)"
(Rule) The point furthest away laterally corresponds to the ball joint of the foot.

Based on such "facing", "landmark point L", and "reference symbol", the standardized shoe measurement unit 112 determines "foot circumference A", "foot width B", "foot length C", "heel shape D", "toe height E", "toe shape F", "distortion G", etc. as "measurement values".

According to the standardized shoe measurement method, for the same reason as the standardized foot measurement method, it is necessary to determine the reference axis direction (front facing) for shoes as well. Also, unlike the foot, which has bones inside, the position of the ball joint is not fixed, so its position must be stably acquired.
Therefore, as shown in Fig. 7, a shoe anchor Sh is created that fits the toe of the foot perfectly, and the shoe anchor Sh is inserted into the shoe and then 3D measurement (scanning) is performed. This allows a reference axis (facing direction) and a reference symbol based on the shoe anchor Sh to be found, and each measurement value can be appropriately calculated based on these.
In this way, the standardized shoe measurement unit 112 can construct a model (shoe measurement data Ks) that excludes design elements such as the toe by using the shoe anchor Sh as a reference.

Any shoe anchor Sh can be used as long as the reference axis (facing direction) and reference symbol mentioned above can be determined, but a shape like that shown in Figure 7 is preferable. That is, as shown in Figure 7, by creating the shoe anchor Sh only up to the ball joint portion, it becomes unnecessary to prepare different shapes for each heel height. Note that while an average shape is used for the toe shape of the shoe anchor Sh in the example of Figure 7, it is even more preferable to create variations using a statistical model.

As described above, the standardized shoe measurement method and the standardized foot measurement method are methods for obtaining a plurality of "measurement values" newly proposed by the present inventors, etc. Machine learning is performed based on these plurality of "measurement values", and the learning results are used to match the foot (of a new user U) with shoes.
That is, the standardized shoe measurement method and standardized foot measurement method are methods that obtain standardized measurement lengths that affect matching from unstandardized shoe and foot shapes. That is, the standardized shoe measurement method and standardized foot measurement method are measurement methods that enable equal comparison regardless of the type of shoe or foot extracted.
Therefore, machine learning is performed using shoe measurement data Ks obtained by measurement using the standardized shoe measurement method and foot measurement data Kt obtained by measurement using the standardized foot measurement method, and as a result of this learning, relationships (weighting) are automatically obtained that, in the past, were determined by craftsmen based on intuition, such as which of the measurement values affects matching (comfort, etc.).

Here, we explain the differences between the measurement values obtained using the standardized shoe measurement method (abbreviated in this paragraph as "shoe") and the measurement values obtained using the standardized foot measurement method (abbreviated in this paragraph as "foot").
Regarding "foot circumference A,""foot" refers to the contour length and shape of the ball joint part cut perpendicular to the ground, while "shoe" refers to the contour length and shape of the corresponding part of the shoe anchor Sh that fits snugly.
Regarding "foot width B," the "foot" is the length of the line connecting the ball joint part horizontally to the ground, while the "shoe" is the horizontal length of the most protruding part.
Regarding the "foot length C," the "foot" is the length on the axis of travel from the tip of the toe to the rear end of the heel, while the "shoe" is (the length from the reference symbol of the shoe anchor Sh to the toe of the shoe anchor Sh) + (the path length along the insole from the reference symbol of the shoe anchor Sh to the rear end of the heel).
Regarding "heel shape D," the "foot" is the change in contour shape when the shoe is cut horizontally to the ground while shifting in the height direction with the rear end of the heel as the reference, whereas the "shoe" is (the change in contour shape when the shoe is cut horizontally to the ground while shifting in the height direction with the rear end of the heel as the reference) + (the shape of the opening).
With regard to "toe height E,""foot" refers to the height (average or distribution) of the foot at a certain distance from the ball joint line, while "shoe" refers to the average height of the foot at a certain distance from the ball joint line (the bottom surface is the top surface of the insole).
Regarding "Toe Shape F," the "foot" is a trapezoidal shape (angle, length, etc.) connecting the ball joint, big toe, and little toe, while the "shoe" is the outline shape when the toe portion is sliced at a certain length from the top surface of the insole.
Regarding "distortion G," the "foot" is the relative angle from the ball joint to the tip of the heel, whereas the "shoe" is the relative angle from the ball joint to the tip of the heel.

In this manner, shoe measurement data Ks1 to Ksm of m shoes (shoe anchors Sh1 to Shm) are obtained by the standardized shoe measurement method. Also, foot measurement data Kt1 to Ktn of n testers T1 to Tn are obtained by the standardized foot measurement method.
Then, the learning device 3 in Figure 5 performs machine learning using each combination of the shoe measurement data Ks1 to Ksm of m shoes (shoe anchors Sh1 to Shm) and the foot measurement data Kt1 to Ktn of n testers T1 to Tn, and the evaluation data Ht1 to Htn for each of the m shoes of the n testers T1 to Tn.

Next, a functional configuration capable of implementing processes such as matching shown in FIG. 3, which are performed using the learning results of the learning device 3, among the processes executed by the information processing system in FIG. 1, will be described.
FIG. 8 is a functional block diagram showing an example of a functional configuration capable of implementing the matching and other processes shown in FIG. 3 among the processes executed by the information processing system 1 in FIG.

As shown in FIG. 8, the user foot measurement device 4 functions as a scan unit 141, a standardized foot measurement unit 142, and an evaluation acquisition unit 143.

The scanning unit 141 of the user's foot measuring device 4 scans both feet of the user U to generate 3D scan data Du.
The standardized foot measurement unit 122 performs measurements on the 3D scan data Du in accordance with the standardized foot measurement method described above, and stores the foot measurement data Ku obtained as a result of the measurements in the foot measurement DB 12.
Here, the foot measurement data Ku of the user U can be linked to various information related to the user U, similar to that of the tester T. Specifically, for example, the 3D scan data Du can be linked to the foot measurement data Ku. In addition, for example, information such as gender, age, weight, average number of steps per day, shoe purchase history and return history, shoes often worn, shoes not interested, favorite brands, disliked brands, and various other information can be linked to the foot measurement data Ku. Furthermore, for example, information such as the address and occupation of the user U may be linked to the foot measurement data Ku. The foot measurement data Ku of the user U can be utilized for various purposes such as various analyses as part of big data together with the measurement data Kt of the tester T.
In addition, the evaluation acquisition unit 143 acquires evaluation data Hu from the user terminal 6, etc., indicating an evaluation of the comfort of the shoes recommended as a result of matching by the matching device 5 when actually worn by the user U, and stores the evaluation data in the evaluation DB 13.

For such a user foot measurement device 4, the matching device 5 functions as a matching method determination unit 151, a user preference acquisition unit 152, a matching unit 153, and a recommendation unit 154.

The matching method determination unit 151 determines a matching method appropriate for the user U based on the results of machine learning by the learning device 3.
The user preference acquisition unit 152 acquires the shoe-related preferences of the user U from the user terminal 6 or the like.
The matching unit 153 uses the foot measurement data Ku of the user U, the shoe measurement data Ks1 to Ksm of each of the m shoes (shoe anchors Sh1 to Shm), and the preferences of the user U to perform matching to estimate which shoes will be comfortable for the user U.
The recommendation unit 154 recommends shoes suitable for the user U (shoes that the user U will likely feel comfortable wearing) to the user U (user terminal 6) based on the matching result of the matching unit 153.

Here, an example of a matching method determined by the matching method determination unit 151 based on the result of machine learning by the learning device 3 will be described with reference to FIG.
FIG. 9 is a diagram illustrating an example of a matching method determined based on the results of machine learning.
As shown in Figure 9, machine learning is performed by the learning device 3 using a combination of shoe measurement data Ksk of a specified shoe (shoe anchor Sh) and foot measurement data Kt of the tester T for each of the testers T1 to Tm, and a set of evaluation data Ht when the specified shoe is actually worn.
Here, as described above, each of the shoe measurement data Ksk and the foot measurement data Kt of the tester T is a collection of multiple measurement values such as "foot circumference A,""foot width B,""foot length C," etc. Similarly, the shoe evaluation data Ht is a collection of individual evaluations of comfort for each item corresponding to multiple measurement values such as "foot circumference A,""foot width B,""foot length C," etc.
9 is performed, a matching method is adopted to predict the user U's evaluation for a combination of the shoe measurement data Ksk of a specific shoe (shoe anchor Sh) and the foot measurement data Ku of the user U. Here, the prediction of the user's evaluation is a prediction of an individual evaluation of comfort for each item corresponding to a plurality of measurement values such as "foot circumference A", "foot width B", "foot length C", etc.

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment, and modifications and improvements that can achieve the object of the present invention are included in the present invention.

In the above embodiment, machine learning was performed on the relationship between the feet of tester U and m shoes, and based on the results of the machine learning, the feet of user U were matched with shoes, and based on the results of the matching, shoes suitable for user U's feet (shoes that are assumed to be highly rated by user U) were recommended, but the targets of the machine learning (targets of matching and recommendations using the results of the machine learning are not particularly limited to this.

For example, the target of machine learning (the target of matching and recommendations using the results of the machine learning) may be one shown in FIG. 10.
FIG. 10 is a diagram showing an outline of a process that can be performed by the information processing system of FIG. 1 and that is different from the example of FIG.
As shown in Figure 10, for each of m existing shoes S1 to Sm (shoe anchors Sh1 to Shm), shoes are matched based on a combination of shoe measurement data Ks1 to Ksm and evaluation data Hu (including evaluation data Ht of tester T) of a number of users U1 to Uk (k is an arbitrary integer value; in the example of Figure 10, k = 3), and an existing shoe with a similar evaluation to the new shoe is recommended.
That is, on a two-dimensional plane with the shoe measurement data Ks as the A axis and the evaluation data Hu as the B axis, each of the m existing shoes (shoe anchors Sh1 to Shm, which are compared for matching) is plotted (projected) based on the combination of the shoe measurement data Ks1 to Ksm and a large number of users U1 to Uk, and machine learning is performed. In other words, machine learning is performed to generate a two-dimensional plane (space) in which shoes (items) with similar evaluations are arranged close to each other.
As a result of this machine learning, by plotting (projecting) a new shoe (the comparison source for matching) onto the two-dimensional plane (space) based on a combination of shoe measurement data Ks and a large number of users U1 to Uk, it becomes possible to search for (match) shoes that are located close to the two-dimensional plane (space) and recommend shoes that have similar evaluations, i.e. shoes that have a similar fit.

Also, for example, the target of machine learning (the target of matching and recommendation using the results of the machine learning) may be one shown in FIG. 11 .
FIG. 11 is a diagram showing an overview of a process that can be performed by the information processing system of FIG. 1 and that performs machine learning different from the example of FIG. 2 .
In the machine learning shown in FIG. 3, a model (existing model) is obtained based on general evaluations by n (a large number of) testers T for m shoes.
11, additional learning is performed taking into consideration feedback information from user U who has been matched and recommended using these existing models, such as information indicating various histories (purchase history and return history) of user U. As a result of this additional learning, a model specialized for the preferences of user U (compared to existing models), that is, a user-specified model, is obtained.
By performing matching using this user-specific model, it becomes possible to recommend shoes that are more suitable for the user U, compared to when an existing model is used.

For example, the target of machine learning (the target of matching and recommendations using the results of the machine learning) is shoes in the above embodiment, but is not limited to this and may be any item that can be worn by the user U (including the tester T), such as clothing, a hat, a helmet worn for protection or support, a corset, a supporter, etc.

FIG. 12 shows an example of measurement data of a user U (including a tester T) when standardized measurements are performed with the item being clothing.
That is, since the measurement method for the human body (user U's body) is standardized, the measurement using that measurement method is adopted as a standardized measurement, and the measurement result is adopted as a standardized measurement value, i.e., an example of the measurement data of user U is shown in Figure 12.

FIG. 13 shows an example of measurement data of an item when standardized measurement is performed, assuming that the item is clothing (jacket).
That is, since the measurement method for clothing (jackets) is standardized for which parts are measured for each category, measurements made using this measurement method are adopted as standardized measurements, and the measurement results are adopted as standardized measurement values. An example of measurement data for clothing (jackets) is shown in Figure 13.

Also, for example, in the above embodiment, matching is performed based on the shape of the item, but this is not particularly limited, and matching may be performed taking other factors into consideration along with or apart from the shape. That is, factors that determine the wearability of an item (if the item is a shoe, the wearability) are not only differences in shape, but also factors such as weight, age, heel height, and shoe material for shoes. Also, factors such as fabric material, gender, and age for clothes.
In the past, if you tried to take into account the influence of these "other factors," you had to create rules for all of the influences. However, by adopting a machine learning method (based on decision trees) that can automatically learn the influence of multiple separate factors, it is now possible to achieve matching that takes into account the influence of "other factors" without creating rules.
Here, the following can be adopted as the "other elements":
That is, if written in the format "other elements" = (data type, impact), it will look like this:
"Material stretchability" = (tensile strength [N/cm], affects the tolerance of shape differences)
"Material type" = (category (cloth, polyester, etc.), affects pain at the contact area)
"Weight" = (kg, affects the load on the toes when wearing heels)
"Heel height" = (cm, affects the load on the toes when wearing heels)
"Age" = (number, affects differences in preferences such as ease of walking)
Affected by differences in preferences such as average daily steps and how easily you get tired

For example, in the above-described embodiment, indirect machine learning was performed using the evaluation made by tester T when he actually wore m shoes, but this is not limited to this. In addition to or instead of this evaluation, for example, a cloth-type pressure distribution sensor may be used to measure the pressure of each of the m shoes when worn by tester T, and machine learning may be performed using the measurement results, so that when matching with user U, predictions can be made even when a pressure distribution sensor is not present.
In this case as well, the result of the prediction is used to estimate the final preference of the user U (for shoes, comfort), and a recommendation to the user U is made based on the estimation result.

Also, for example, in the above-described embodiment, various information (metadata) is associated with the shoe measurement data Ks of the shoe, the foot measurement data Kt of the tester T, and the foot measurement data Ku of the user U. By taking into account these various information (metadata), machine learning, matching (preference estimation), and recommendation can be performed with higher accuracy.
Such metadata is not limited to the above-described embodiment, and any metadata can be adopted. For example, the following metadata can be adopted. Note that metadata with (Better) in parentheses is metadata for which it is preferable to adopt the metadata. Metadata with (Advanced) in parentheses is metadata for which it is not essential to adopt the metadata, but which, if adopted, can perform more appropriate machine learning, matching (preference estimation), and recommendation.
That is, as metadata of material properties, elongation (better), surface hardness (better), material type (enamel, smooth, cloth, etc.) (advanced) can be adopted.
As metadata for manufacturing information, the brand name, manufacturer name, factory name (advance), and manufacturing method (advance) can be adopted.
As attribute information of user U, proficiency level (Better), frequency of use (Better), and age (Advanced) can be adopted.

In the above-described embodiment, the user U goes to a shoe store or the like and has his or her feet scanned there, and matching is performed using the resulting foot measurement data Ku, and based on the results, suitable shoes (shoes predicted to be comfortable to wear) are recommended to the user U.
However, it is not essential to use the foot measurement data Ku during matching.
For example, on an e-commerce site, in order to recommend shoes or clothes when there is no foot or body shape data for a user U, a matching may be performed using shoes or clothes that the user U has previously purchased and that fit well, and items with the same or similar shapes may be searched for and recommended.
In this case, there are a large number of different measurements for shoes and clothes, and simple similarity between them may not necessarily emphasize important values for fitting. Therefore, it is preferable to employ the information processing system to which the above-mentioned machine learning is applied, to use the similarity of the prior evaluation by tester T (test user) to calculate the contribution of the measurements of shoes and clothes to the similarity, and to perform a search that reflects the contribution.

Specifically, for example, the evaluation by the combination of tester T1 and shoes (hereinafter referred to as the "first combination") is (foot length difference, toe thickness difference, heel width difference, ease of slipping off) = (+0.1, -0.8, +0.1, A), and the evaluation by another combination of tester T2 and shoes (hereinafter referred to as the "second combination") is (foot length difference, toe thickness difference, heel width difference, ease of slipping off) = (+0.1, -0.8, +0.1, A).
In this case, the ratio (contribution rate) of the important items for estimating the ease of slipping off is calculated by machine learning as follows, for example: Contribution rate = (foot length difference, toe width difference, heel width difference) = (0.3, 0.1, 0.6).
Using this contribution, the shoe similarity index (each measurement value of the shoe measurement data Ks) can be weighted to perform matching (searching).
For example, the shoe measurement data Ks1 of the first shoe is (foot length, toe width, heel width) = (23.2, 2.4, 4.55), and the shoe measurement data Ks1 of the second shoe is (foot length, toe width, heel width) = (23.6, 3.6, 4.6). In this case, in the above example, the contribution of the heel width difference is 0.6, which is the highest, so matching (searching) is performed with emphasis on the heel width among the measurement values. That is, since the heel width of the first shoe is 4.55 and that of the second shoe is 4.6, it is determined that the first shoe and the second shoe are very similar in terms of ease of slipping off. That is, if the first shoe is a shoe that the user U previously purchased and that fit well, the second shoe is recommended to the user U who places importance on ease of slipping off.

When such a search (recommendation) is performed, the EC site screen displayed on the user terminal 6 and the screen displayed on the store's device (e.g., user foot measuring device 4) after the user U's feet are scanned in the store may look, for example, as shown in Figures 14 to 16.
FIG. 14 is a diagram showing an example of a mock-up of a recommendation screen in a store.
FIG. 15 is a diagram showing an example of a mock-up of a recommendation screen in a store, and an example of a screen that is displayed after the screen of FIG. 14 is displayed.
FIG. 16 shows an example of a mockup of a screen of an EC site.

For example, the hardware configurations shown in FIG. 4 are merely examples for achieving the objectives of the present invention and are not particularly limited.

Furthermore, the functional block diagrams shown in Figures 5 and 8 are merely examples and are not particularly limited. In other words, it is sufficient that the information processing system is provided with a function that can execute the above-mentioned series of processes as a whole, and the type of functional block used to realize this function is not particularly limited to the examples in Figures 5 and 8.

5 and 8, the locations of the functional blocks may be arbitrary. For example, at least some of the functional blocks of each device may be provided on the user terminal 6 side. In other words, an information processing system may be constructed in which, as viewed from the user terminal 6, a browser is used to cause an external server or the like to execute processing, or an information processing system may be constructed in which an installed application program executes processing, or an information processing system consisting of a combination of these may be constructed.
A single functional block may be configured as a single piece of hardware, or may be configured in combination with a single piece of software.

When the processing of each functional block is executed by software, the program constituting the software is installed into a computer or the like from a network or a recording medium.
The computer may be a computer built into dedicated hardware, or may be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

Recording media containing such programs are not only configured as removable media that are distributed separately from the device body in order to provide the programs, but also configured as recording media etc. that are provided to each user in a state where they are pre-installed in the device body.

In this specification, the steps of describing a program to be recorded on a recording medium include not only processes that are performed in chronological order, but also processes that are not necessarily performed in chronological order but are executed in parallel or individually.

In summary, the information processing apparatus to which the present invention is applied is sufficient if it has the following configuration, and can take various different embodiments.
That is, an information processing device to which the present invention is applied (for example, the matching device 5 in FIG. 1)
a prediction means (e.g., a matching unit 153 in FIG. 8 that performs matching in step Si in FIG. 3 ) that predicts the suitability of an item for a user (e.g., user U in FIG. 3 ) based on the results of machine learning (e.g., step Se in FIG. 2 ) using a combination of first type standard data (e.g., shoe measurement data Ks1 to Ksm in FIG. 2 ) obtained by standardizing measurement data (e.g., 3D scan data Ds1 to Dsm in FIG. 2 ) of a predetermined item (e.g., shoes, which may be shoe anchors Sh1 to Shm) and second type standard data (e.g., foot measurement data Kt1 to Ktn in FIG. 2 ) obtained by standardizing measurement data (e.g., 3D scan data Dt1 to Dtn in FIG. 2 ) of at least a part of the body of a predetermined tester (e.g., tester T1 to Tn in FIG. 2 ) for each of a plurality of items and a plurality of testers;
A recommendation unit (e.g., the recommendation unit 154 in FIG. 8 that makes the recommendation in step Sj in FIG. 3 ) that recommends an item to the user based on the prediction result of the prediction unit;
Equipped with.

In addition, another aspect of the information processing device to which the present invention is applied (for example, the shoe measurement device 1 in FIG. 1 ) is as follows:
A measurement means (e.g., the standardized shoe measurement unit 112 in FIG. 5 ) that identifies a facing position based on measurement data of an item (e.g., the 3D scan data Ds1 to Dsm in FIG. 2 generated by the scan unit 111 in FIG. 5 ), detects one or more landmark points, calculates a reference symbol based on a predetermined rule for eliminating the design of the item, calculates each measurement value of a plurality of items based on the facing position, the landmark points, and the reference symbol (e.g., see FIG. 7 ), and generates information including at least each measurement value of the plurality of items as measurement data of the item (e.g., shoe measurement data Ks1 to Ksm in FIG. 2 ).
Equipped with.

1: Shoe measurement device 2: Tester foot measurement device 3: Learning device 4: User foot measurement device 5: Matching device 6: User terminal 11: Shoe measurement DB
12: Foot measurement database
13: Evaluation DB
21: CPU
22: ROM
23: RAM
24: Bus 25: Input/Output interface 26: Output unit 27: Input unit 28: Memory unit 29: Communication unit 30: Drive 31: Removable media 111: Scanning unit 112: Standardized shoe measurement unit 121: Scanning unit 122: Standardized foot measurement unit 123: Evaluation acquisition unit 141: Scanning unit 142: Standardized foot measurement unit 143: Evaluation acquisition unit 151: Matching method determination unit 152: User preference acquisition unit 153: Matching unit 154: Recommendation unit J: Ball joint A: Foot circumference B: Foot width C: Foot length D: Heel shape E: Toe height F: Toe shape G: Distortion H: Evaluation data L: Landmark point U: User

Claims (3)

a shoe measurement means for generating shoe measurement data relating to a shape of a shoe based on 3D measurement data of the shoe to be worn by a natural person and in accordance with a first predetermined rule for eliminating design imperfections of the shoe ;
a natural person measurement means for generating natural person measurement data relating to a shape of a part of a body of a natural person on which the shoe is to be worn, based on 3D measurement data of the part and in accordance with a second rule corresponding to the first rule;
an evaluation acquisition means for acquiring an evaluation of the natural person when the natural person wears the shoes , for the natural person for whom the natural person measurement data was generated;
A means for acquiring other element information regarding one or more other elements different from the shape, for at least one of the shoe and the natural person for which the natural person measurement data is generated;
A learning means for performing a predetermined machine learning using the shoe measurement data for a plurality of the shoes , the natural person measurement data and the evaluation for a plurality of the natural people, and a plurality of the other element information;
An information processing device comprising: An information processing method executed by an information processing device,
a shoe measurement step of generating shoe measurement data relating to a shape of the shoe based on 3D measurement data of the shoe to be worn by a natural person and in accordance with a first predetermined rule for eliminating design features of the shoe ;
a natural person measurement step of generating natural person measurement data relating to a shape of a part of a body of a natural person on which the shoe is to be worn, based on 3D measurement data of the part and in accordance with a second rule corresponding to the first rule;
an evaluation acquisition step of acquiring an evaluation of the natural person when the natural person wears the shoes , for the natural person for whom the natural person measurement data was generated;
A step of acquiring other element information regarding one or more other elements different from the shape for at least one of the shoe and the natural person for which the natural person measurement data was generated;
A learning step of performing a predetermined machine learning using the shoe measurement data for a plurality of the shoes , the natural person measurement data and the evaluation for a plurality of the natural people, and a plurality of the other element information;
An information processing method comprising: A computer that controls an information processing device includes:
a shoe measurement step of generating shoe measurement data relating to a shape of the shoe based on 3D measurement data of the shoe to be worn by a natural person and in accordance with a first predetermined rule for eliminating design features of the shoe ;
a natural person measurement step of generating natural person measurement data relating to a shape of a part of a body of a natural person on which the shoe is to be worn, based on 3D measurement data of the part and in accordance with a second rule corresponding to the first rule;
an evaluation acquisition step of acquiring an evaluation of the natural person when the natural person wears the shoes , for the natural person for whom the natural person measurement data was generated;
A step of acquiring other element information regarding one or more other elements different from the shape for at least one of the shoe and the natural person for which the natural person measurement data was generated;
A learning step of performing a predetermined machine learning using the shoe measurement data for a plurality of the shoes , the natural person measurement data and the evaluation for a plurality of the natural people, and a plurality of the other element information;
A program for executing control processing including:

Images