WO2020111269A1

WO2020111269A1 - Dimensional data calculation device, product manufacturing device, information processing device, silhouette image generating device, and terminal device

Info

Publication number: WO2020111269A1
Application number: PCT/JP2019/046896
Authority: WO
Inventors: 佐藤　大輔; 浩紀八登; 親史有田; 佳久石橋; 嵩士中野; 諒介佐々木; 亮介田嶋; 大田　佳宏
Original assignee: Ａｒｉｔｈｍｅｒ株式会社
Priority date: 2018-11-30
Filing date: 2019-11-29
Publication date: 2020-06-04

Abstract

The present invention addresses the problem of providing highly accurate dimensional data of each part of a target object. A dimensional data calculation device 1020 is provided with an acquisition unit 1024A, an extraction unit 1024B, a conversion unit 1024C, and a calculation unit 1024D. The acquisition unit 1024A acquires image data obtained by photographing the target object, and total length data of the target object. The extraction unit 1024B extracts, from the image data, shape data indicating the shape of the target object. The conversion unit 1024C converts the shape data on the basis of the total length into a silhouette. The calculation unit 1024D, using the shape data converted by the conversion unit 1024C, calculates dimensional data of each part of the target object.

Description

Dimension data calculation device, product manufacturing device, information processing device, silhouette image generation device, terminal device

The present disclosure relates to a dimension data calculation device, a product manufacturing device, an information processing device, a silhouette image generation device, and a terminal device.

Conventionally, an apparatus for manufacturing a product based on the shape of an object has been studied. For example, in Patent Document 1, nail images are obtained by photographing nails of a finger, and nail information necessary for creating artificial nails such as the shape of the nail, the position of the nail, and the curvature of the nail is obtained based on the acquired nail image. There is disclosed a technique for obtaining a false nail part, storing it, and creating a false nail part based on this nail information.
Patent Document 1 Japanese Patent Laid-Open No. 2017-018158

However, with the conventional technology, it was difficult to calculate a large number of shapes in the object with high accuracy.

In the first aspect of the present invention, an acquisition unit that acquires image data of a captured object and full length data of the object, an extraction unit that extracts shape data indicating the shape of the object from the image data, and a shape A conversion unit that converts the data based on the full length data, the dimension data of the shape data converted by the conversion unit is dimensionally reduced, and the weighting coefficient optimized for each reduced value of each dimension and each part of the object is used. Provided is a dimension data calculation device including a calculation unit that calculates the dimension data of each part of an object.

In a second aspect of the present invention, there is provided a product manufacturing apparatus that manufactures a product related to the shape of an object using the dimension data calculated using the dimension data calculation apparatus of the first aspect.

According to a third aspect of the present invention, there is provided a reception unit that receives a silhouette image of an object, and an object engine that associates the silhouette image of the sample object with a predetermined number of shape parameter values associated with the sample object. An estimation unit that estimates the value of the shape parameter of the object from the received silhouette image by using the dimension data, in which the estimated value of the shape parameter of the object is related to an arbitrary part of the object. An information processing device associated with the.

In a fourth aspect of the present invention, there is provided a receiving unit that receives attribute data of an object, and an object engine that associates the attribute data of the sample object with a value of a predetermined number of shape parameters associated with the sample object. An estimation unit that estimates the value of the shape parameter of the object from the received attribute data, and the estimated value of the shape parameter of the object is dimensional data related to an arbitrary part of the object. An information processing device associated with the.

In a fifth aspect of the present invention, an acquisition unit that acquires image data of a captured object and full length data of the object, an extraction unit that extracts shape data indicating the shape of the object from the image data, and a shape Using a transformation unit that transforms the data into a silhouette image based on the full length data, and an object engine that associates the silhouette image of the sample object with the values of a predetermined number of shape parameters associated with the sample object, the silhouette A dimension data calculating device, comprising: an estimating unit that estimates the value of a predetermined number of shape parameters from an image; and a calculating unit that calculates the dimension data of an object based on the value of the estimated predetermined number of shape parameters. Provided.

According to a sixth aspect of the present invention, there is provided a product manufacturing apparatus for manufacturing a product related to the shape of an object using at least one dimension data calculated by using the dimension data calculating apparatus according to the fifth aspect. Provided.

In a seventh aspect of the present invention, an acquisition unit that acquires attribute data including at least one of full length data and weight data of an object, and attribute data using a coefficient learned by machine learning. A dimension data calculation device is provided that includes a calculation unit that calculates the dimension data of each part of the object by performing polynomial regression.

In an eighth aspect of the present invention, an object region of an object is obtained by using an acquisition unit that acquires image data including a depth map, in which the object is photographed, and three-dimensional point cloud data generated from the depth map. An extraction unit for extracting shape data indicating the shape of the object based on the depth data of the depth map corresponding to the object area, and a conversion for converting the shape data to generate a silhouette image of the object And a silhouette image generating device.

In a ninth aspect of the present invention, an acquisition unit that acquires image data of a captured object and full-length data of the object, an extraction unit that extracts shape data indicating the shape of the object from the image data, and a shape A calculation unit that calculates the dimension data of each part of the object using the data, the image data includes a depth map, and the shape data extracted by the extraction unit is associated with the depth data of the object in the depth map. A dimensional data calculation device is provided.

According to a tenth aspect of the present invention, there is provided a product manufacturing apparatus that manufactures a product related to the shape of an object using the dimension data calculated using the dimension data calculating apparatus of the ninth aspect.

In an eleventh aspect of the present invention, an acquisition unit that acquires image data of a captured object and full-length data of the object, an extraction unit that extracts shape data indicating the shape of the object from the image data, and a sample An estimation unit that estimates the value of a predetermined number of shape parameters from the silhouette image of the object by using an object engine that associates the silhouette image of the object and the value of the shape parameter of a predetermined number associated with the sample object And a calculation unit that calculates the dimension data of the object based on the values of the estimated predetermined number of shape parameters, the image data includes a depth map, and the shape data is the depth data of the object in the depth map. There is provided a dimensional data calculation device associated with.

According to a twelfth aspect of the present invention, there is provided a product manufacturing apparatus for manufacturing a product related to the shape of an object using the dimension data calculated using the dimension data calculating apparatus according to the eleventh aspect.

In a thirteenth aspect of the present invention, the terminal device is connected to an information processing device that processes information about an object from image data of the object, and acquires image data of the object. An acquisition unit, a determination unit that determines whether or not the target object included in the image data is a pre-registered target object, the determination result by the determination unit is displayed on the output unit, and the image data is transmitted to the information processing device. A terminal device is provided, which includes a reception unit that receives an input as to whether or not to perform.

In a fourteenth aspect of the present invention, a shape parameter acquisition unit that acquires the value of the shape parameter of the object and three-dimensional mesh data of the object from the value of the shape parameter of the object are configured, A dimension data calculation device is provided that includes a calculation unit that calculates the dimension data of a predetermined region based on the information on the vertices of the three-dimensional mesh data that forms the associated predetermined region.

It is a schematic diagram which shows the structure of the dimension data calculation apparatus 1020 which concerns on 1st Embodiment. 9 is a flowchart for explaining an operation of the dimension data calculation device 1020 according to the same embodiment. It is a schematic diagram which shows the concept of the shape data which concerns on the same embodiment. It is a schematic diagram which shows the concept of the product manufacturing system 1001 which concerns on the same embodiment. It is a sequence diagram for explaining operation of product manufacturing system 1001 concerning the embodiment. It is a schematic diagram which shows an example of the screen displayed on the terminal device 1010 which concerns on the same embodiment. It is a schematic diagram which shows an example of the screen displayed on the terminal device 1010 which concerns on the same embodiment. It is a schematic diagram which shows the structure of the dimension data calculation apparatus 2120 which concerns on 2nd Embodiment. It is a schematic diagram which shows the concept of the data used by the secondary regression which concerns on the same embodiment. It is a schematic diagram which shows the concept of the product manufacturing system 2001S which concerns on the same embodiment. It is a schematic diagram of the dimension data calculation system 3100 which concerns on 3rd Embodiment. 12 is a flowchart showing the operation of the learning device 3025 of FIG. 12 is a flowchart showing the operation of the dimension data calculation device 3020 of FIG. It is a schematic explanatory drawing which shows the characteristic of a shape parameter. It is a schematic graph which shows the characteristic of a shape parameter. It is a schematic diagram which shows the concept of the product manufacturing system 3001 which concerns on 3rd Embodiment. FIG. 17 is a sequence diagram showing an operation of the product manufacturing system 3001 of FIG. 16. It is a schematic diagram which shows an example of the screen displayed on the terminal device 3010 of FIG. It is a schematic diagram which shows an example of the screen displayed on the terminal device 3010 of FIG. It is a schematic diagram of the dimension data calculation system 4100 which concerns on 4th Embodiment. 21 is a flowchart showing the operation of the learning device 4125 of FIG. 21 is a flowchart showing the operation of the dimension data calculation device 4020 of FIG. 20. It is a schematic diagram which shows the concept of the product manufacturing system 4001S which concerns on 4th Embodiment. It is a schematic diagram which shows the structure of the silhouette image generation apparatus 5020 which concerns on other embodiment. It is a flow chart for explaining operation of silhouette image generating device 5020 concerning the embodiment. It is a schematic diagram which shows the structure of the dimension data calculation apparatus 6020 which concerns on other embodiment. It is a schematic diagram of a human body model when an object is a human body. It is a schematic diagram of a human body model when an object is a human body. 11 is a flowchart for explaining the operation of the dimension data calculation device 6020 according to another embodiment. It is the schematic which showed the example which comprises the body area|region and arm area which are cylindrical regions with respect to a human body model. It is the schematic which showed the example which comprises the body area|region and arm area which are cylindrical regions with respect to a human body model. It is the schematic which showed the example which comprises the body area|region and arm area which are cylindrical regions with respect to a human body model. It is a schematic plan view which showed the example which extracts the calculation point regarding "hip". It is a schematic solid figure showing the example which extracts the calculation point about "hip". It is a schematic plan view which showed the example which extracts the calculation point regarding a "waist." It is a schematic solid figure showing the example which extracts the calculation point about a "waist." It is a schematic plan view which showed the example which extracts the calculation point regarding a "wrist." It is a schematic solid figure showing the example which extracts the calculation point about a "wrist." It is a schematic plan view which showed the example which extracts the calculation point regarding an "armhole." It is a schematic solid figure showing the example which extracts the calculation point about an "armhole." It is a schematic diagram which shows the structure of the terminal device 7020 which concerns on other embodiment. It is a schematic diagram which shows an example of the screen displayed on the terminal device 7020 of FIG. FIG. 37 is a sequence diagram for explaining the operation of the terminal device 7020 of FIG. 36.

<First Embodiment>
(1-1) Configuration of Dimension Data Calculation Device FIG. 1 is a schematic diagram showing the configuration of the dimension data calculation device 1020 according to this embodiment.

The dimension data calculation device 1020 can be realized by any computer, and includes a storage unit 1021, an input/output unit 1022, a communication unit 1023, and a processing unit 1024. The dimension data calculation device 1020 may be realized as hardware using an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

The storage unit 1021 stores various kinds of information, and is realized by an arbitrary storage device such as a memory and a hard disk. For example, the storage unit 1021 stores the weighting factor necessary for executing the information processing described later in association with the length and weight of the target object. The weighting factor is acquired in advance by performing machine learning from teacher data including attribute data, image data, and dimension data described later.

The input/output unit 1022 is realized by a keyboard, a mouse, a touch panel, etc., and inputs various information to the computer and outputs various information from the computer.
The communication unit 1023 is realized by an arbitrary network card or the like, and enables communication with a communication device on the network by wire or wirelessly.

The processing unit 1024 executes various types of information processing, and is realized by a processor such as a CPU or GPU and a memory. Here, the processing unit 1024 functions as the acquisition unit 1024A, the extraction unit 1024B, the conversion unit 1024C, and the calculation unit 1024D by reading the program stored in the storage unit 1021 into the CPU, GPU, or the like of the computer.

The acquisition unit 1024A acquires image data of a captured object, full length data/weight data of the object, and the like. In addition, here, the acquisition unit 1024A acquires a plurality of image data obtained by photographing the object from different directions.

The extraction unit 1024B extracts shape data indicating the shape of the object from the image data. Specifically, the extraction unit 1024B extracts the target area included in the image data by using the semantic segmentation algorithm (Mask R-CNN or the like) prepared for each type of the target object, The shape data of is extracted. Here, the semantic segmentation algorithm is constructed using teacher data in which the shape of the object is not specified.

Note that if the semantic segmentation algorithm is constructed using teacher data of an object whose shape is unspecified, it may not always be possible to accurately extract the shape of the object. In such a case, the extraction unit 1024B extracts the shape data of the target object from the target object area by the GrabCut algorithm. This makes it possible to extract the shape of the object with high accuracy. Furthermore, the extraction unit 1024B may correct the image of the target object specified by the grab cut algorithm based on the color image of the specific portion of the target object. This makes it possible to generate the shape data of the object with higher accuracy.

The conversion unit 1024C converts the shape data into silhouettes based on the full length data. Thereby, the shape data is standardized.

The calculation unit 1024D uses the shape data converted by the conversion unit 1024C to calculate the dimension data of each part of the object. Specifically, the calculation unit 1024D reduces the dimension of the shape data converted by the conversion unit 1024C. The dimension reduction here is realized by a method such as principal component analysis, particularly kernel principal component analysis (Kernel PCA) or linear discriminant analysis.

Then, the calculation unit 1024D calculates the dimension data of each part of the object using the reduced value of each dimension and the weighting coefficient optimized for each part of the object.

More specifically, the calculation unit 1024D linearly combines the value of each dimension reduced for the first time and the weighting coefficient W1pi optimized for each part of the object to obtain the predetermined value Zpi. The symbol p is the number of dimensions obtained by reduction and is a value of 10 or more. Then, the calculating unit 1024D performs the second dimension reduction using the predetermined value Zpi and the attribute data including at least the attributes of the length and weight of the target object, and each dimension obtained by the second dimension reduction. Based on the value of, the dimensional data of each part of the object is calculated. The number of weighting factors W1pi is prepared as many as the reduced dimension for each dimensional location (i) of the object.

In the above description, the calculation unit 1024D calculates the predetermined value Zpi by using the linear combination, but the calculation unit 1024D may calculate these values by a method other than the linear combination. Specifically, the calculation unit 1024D generates a quadratic feature amount from the value of each dimension obtained by the dimension reduction, and calculates the quadratic feature amount and the weighting coefficient optimized for each part of the object. The predetermined value may be obtained by combining

(1-2) Operation of Dimension Data Calculation Device FIG. 2 is a flowchart for explaining the operation of the size data calculation device 1020 according to this embodiment.

First, the dimension data calculation device 1020 acquires a plurality of image data obtained by photographing the entire target object from different directions via an external terminal device and the like together with the total length data indicating the total length of the target object (S1001).

Next, the dimension data calculation device 1020 extracts shape data indicating the shape of each part of the object from each image data (S1002).
Subsequently, the dimension data calculation device 1020 executes a rescale process for converting each shape data into a predetermined size based on the total length data (S1003).

Next, the dimension data calculation device 1020 combines the plurality of converted shape data to generate new shape data (hereinafter, also referred to as shape data for calculation). Specifically, as shown in FIG. 3, the shape data of h rows and w columns are combined to form an m×h×w data string. The symbol m is the number of shape data (S1004).

After that, the dimension data calculation device 1020 uses the newly generated shape data and the weighting coefficient W1pi optimized for each part of the object, and the i-th (i=1 to j) object. The dimensional data of each part is calculated (S1005 to S1008). It should be noted that the symbol j is the total number of dimension portions for which dimension data is to be calculated.

(1-3) Features of dimensional data calculation device (1-3-1)
As described above, the dimension data calculation device 1020 according to this embodiment includes the acquisition unit 1024A, the extraction unit 1024B, the conversion unit 1024C, and the calculation unit 1024D. The acquisition unit 1024A acquires image data of a captured object and full-length data of the object. The extraction unit 1024B extracts shape data indicating the shape of the object from the image data. The conversion unit 1024C converts the shape data based on the full length data to form a silhouette. The calculation unit 1024D uses the shape data converted by the conversion unit 1024C to calculate the dimension data of each part of the object.

Therefore, since the dimension data calculation device 1020 calculates the dimension data of each part of the object using the image data and the full length data, it is possible to provide highly accurate dimension data. Further, since the dimension data calculation device 1020 can process many pieces of image data and full length data at once, it is possible to highly accurately provide many pieces of dimension data.

Then, by using such a dimension data calculation device 1020, for example, the dimension data of each part of a living thing as an object can be calculated with high accuracy. Further, it is possible to calculate with high accuracy the dimensional data of each part of an arbitrary object such as a car or various luggage as the object.
Further, by incorporating the dimension data calculation device into a product manufacturing device that manufactures various products, it becomes possible to manufacture a product that conforms to the shape of the target object.

(1-3-2)
Further, in the dimension data calculation device 1020, the acquisition unit 1024A acquires a plurality of image data obtained by photographing the object from different directions. With such a configuration, the accuracy of the dimensional data can be improved.

(1-3-3)
In the dimension data calculation device 1020, the calculation unit 1024D reduces the dimension of the shape data converted by the conversion unit 1024C. Then, the calculation unit 1024D calculates the dimension data of each part of the object using the reduced value of each dimension and the weighting coefficient W1pi optimized for each part of the object. With such a configuration, it is possible to improve the accuracy of the dimensional data while suppressing the calculation load.

Specifically, the calculation unit 1024D linearly combines the reduced value of each dimension and the weighting coefficient W1pi optimized for the i-th part of the object to obtain the predetermined value Zi. Further, the calculation unit 1024D executes the second dimension reduction using the predetermined value Zi and the attribute data including at least the attributes of the length and the weight of the object, and the i-th dimension data of the object. To calculate. With such a configuration, it is possible to further improve the accuracy of the dimensional data while suppressing the calculation load. In the above description, the calculation unit 1024D generates a secondary feature amount from the value of each dimension obtained by dimension reduction, instead of the linear combination, and for each of the secondary feature amount and the object part. The predetermined value may be obtained by combining with the optimized weight coefficient.

(1-3-4)
Further, in the dimension data calculation device 1020, the extraction unit 1024B extracts the target object region included in the image data by using the semantic segmentation algorithm constructed using the teacher data prepared for each kind of the target object. Thus, the shape data of the object is extracted. With such a configuration, the accuracy of the dimensional data can be improved.

Note that although some semantic segmentation algorithms are open to the public, those that are open to the public are usually constructed using teacher data in which the shape of the object is not specified. Therefore, depending on the purpose, the accuracy of extracting the object region included in the image data may not always be sufficient.

Therefore, in such a case, the extraction unit 1024B extracts the shape data of the object from the object area by the grab cut algorithm. With such a configuration, the accuracy of the dimensional data can be further improved.

Further, the extraction unit 1024B may correct the image of the object extracted by the grab cut algorithm based on the color image of the specific portion in the image data to generate new shape data. With such a configuration, the accuracy of the dimensional data can be further improved. For example, when the object is a person, by setting the hand and the back as specific parts and correcting them based on the color images of these specific parts, it is possible to obtain the shape data of the person who is the object with high accuracy. it can.

(1-4) Modification (1-4-1)
In the above description, the acquisition unit 1024A acquires a plurality of image data obtained by photographing the target object from different directions, but a plurality of image data is not necessarily required. It is possible to calculate the dimension data of each part even if the image data of the object is one.

As a modified example of the present embodiment, a depth data measuring device that can also acquire depth data can be applied, and a depth map having depth data for each pixel may be configured based on the depth data. By applying such a depth data measuring device, the image data that can be acquired by the acquisition unit 1024A can be RGB-D (Red, Green, Blue, Depth) data. Specifically, the image data can include a depth map in addition to the RGB image data that can be acquired by a normal monocular camera.

An example of a depth data measuring device is a stereo camera. In the present specification, the “stereo camera” refers to an imaging device of any form capable of simultaneously capturing an object from a plurality of different directions and reproducing binocular parallax to form a depth map. In addition to the stereo camera, a depth map may be configured by obtaining depth data using a LiDAR (Light Detection and Ranging) device.

(1-4-2)
In the above description, the semantic segmentation algorithm and/or the grab cut algorithm are adopted to obtain the shape data of the object. Additionally or alternatively, for example, when a stereo camera is applied, the depth map obtained from the stereo camera can be used to associate the depth data of the object with the shape data of the object. .. This makes it possible to generate the shape data of the object with higher accuracy.

Specifically, when a stereo camera is used, the extraction unit 1024B determines, based on the depth map acquired by the acquisition unit 1024A, the target object region that is a part in which the target object is captured from the image data in which the target object is captured. To extract. For example, the object region is extracted by removing the region whose depth data is not within the predetermined range from the depth map. In the extracted object region, the shape data is associated with the object depth data on a pixel-by-pixel basis.

The conversion unit 1024C converts the shape data based on the full length data. In addition, the conversion unit 1024C converts the shape data into monochrome image data based on the depth data of the target area in addition to the full length data to generate a “gradation silhouette image” (new shape data). (See below).

The gradation silhouette image to be generated is not simple black and white binarized data, but is a single color represented by data with a brightness value of 0 (“black”) to 1 (“white”) based on depth data. It is a multi-tone monochrome image. That is, the gradation silhouette image data is associated with the depth data and has a larger amount of information regarding the shape of the object. The gradation silhouette image data is standardized by full length data.

According to this modification, the shape data of the object is extracted with higher accuracy by extracting the object region based on the depth map configured by the acquisition unit 1024A using any machine capable of measuring the depth data. It becomes possible to do. Further, since the gradation silhouette image data (shape data to be converted) is associated with the depth data of the target object, it has a larger amount of information regarding the shape of the target object, and the calculation unit 1024D calculates each part of the target object. It is possible to calculate the dimension data with higher accuracy.

Supplementally, the calculation unit 1024D reduces the dimension of the gradation silhouette image data (shape data) converted by the conversion unit 1024C. In this case, the number of dimensions reduced for the first time is about 10 times larger than that of the silhouette image of the binarized data. In addition, the number of weighting factors is prepared for each dimensional location (i) of the object according to the reduced dimension.

Note that the gradation silhouette image is described here as being distinguished from a simple silhouette image, but in other embodiments and other modified examples, it may be simply described as a silhouette image without distinguishing both.

(1-4-3)
In the above description, the calculation unit 1024D executes dimension reduction twice, but such processing is not always necessary. The calculation unit 1024D may calculate the dimension data of each part of the object from the value of each dimension obtained by executing the dimension reduction once. Depending on the purpose, the dimension data calculation device 1020 may calculate the dimension data without reducing the dimension of the shape data.

(1-4-4)
In the above description, the extraction unit 1024B extracts the target object area included in the image data by using the semantic segmentation algorithm constructed using the teacher data in which the shape of the target object is not specified. It is not necessary to use such teacher data. For example, a semantic segmentation algorithm constructed using teacher data in which the shape of the object is specified may be used. By using the teacher data in which the shape of the object is specified, it is possible to improve the calculation accuracy of the dimension data and suppress the calculation load according to the purpose.

(1-5) Application to Product Manufacturing System Hereinafter, an example in which the above-described dimension data calculation device 1020 is applied to the product manufacturing system 1001 will be described.
(1-5-1) Configuration of Product Manufacturing System FIG. 4 is a schematic diagram showing the concept of the product manufacturing system 1001 according to this embodiment.
The product manufacturing system 1001 is a system for manufacturing a desired product 1006, including a dimension data calculation device 1020 capable of communicating with the terminal device 1010 owned by the user 1005, and a product manufacturing device 1030. In FIG. 4, as an example, the concept when the object 1007 is a person and the product 1006 is a chair is shown. However, the object 1007 and the product 1006 of the product manufacturing system according to the present embodiment are not limited to these.

The terminal device 1010 can be realized by a so-called smart device. Here, the terminal device 1010 exerts various functions by installing the user program in the smart device. Specifically, the terminal device 1010 generates image data captured by the user 1005. Here, the terminal device 1010 may have a stereo camera function of simultaneously capturing an object from a plurality of different directions and reproducing binocular parallax. Note that the image data is not limited to that captured by the terminal device 1010, and, for example, data captured using a stereo camera installed in a store may be used.

Further, the terminal device 1010 accepts input of attribute data indicating the attribute of the target object 1007. Examples of the “attribute” include the total length, weight, and elapsed time (including age) from the generation of the object 1007. Further, the terminal device 1010 has a communication function, and executes transmission and reception of various information with the dimension data calculation device 1020 and the product manufacturing device 1030.

The dimension data calculation device 1020 can be realized by an arbitrary computer. Here, the storage unit 1021 of the dimension data calculation device 1020 stores the information transmitted from the terminal device 1010 in association with the identification information that identifies the user 1005 of the terminal device 1010. The storage unit 1021 also stores parameters and the like necessary for executing information processing described below. For example, the storage unit 1021 stores the weighting factor W1pi necessary for executing the information processing described later in association with the item of the attribute of the target object 1007 and the like.

Further, the processing unit 1024 of the dimension data calculation device 1020 functions as the acquisition unit 1024A, the extraction unit 1024B, the conversion unit 1024C, and the calculation unit 1024D, as described above. Here, the acquisition unit 1024A acquires the image data captured by the user 1005 and the attribute data of the target object 1007. The extraction unit 1024B also extracts shape data indicating the shape of the object 1007 from the image data. For example, when “person” is set in advance as the type of object, a semantic segmentation algorithm is constructed using teacher data for identifying a person. Further, the extraction unit 1024B corrects the image of the target object 1007 specified by the grab cut algorithm based on the color image of the specific portion of the target object 1007, and further highly accurately generates the shape data of the target object 1007. Further, the conversion unit 1024C converts the shape data based on the full length data to make a silhouette. The calculation unit 1024D calculates the dimension data of each part of the user 1005 using the shape data converted by the conversion unit 1024C. Here, the calculation unit 1024D linearly combines the reduced value of each dimension and the weighting coefficient W1pi optimized for each part of the object 1007 to obtain the predetermined value Z1i. Then, the calculation unit 1024D reduces the dimension using the predetermined value Z1i and the attribute data of the object 1007, and calculates the dimension data of each part of the object 1007 based on the reduced value of each dimension.

The product manufacturing apparatus 1030 is a manufacturing apparatus that manufactures a desired product related to the shape of the object 1007 using the dimension data calculated by the dimension data calculation apparatus 1020. Note that the product manufacturing apparatus 1030 can employ any device that can automatically manufacture and process a product, and can be realized by, for example, a three-dimensional printer.

(1-5-2) Operation of Product Manufacturing System FIG. 5 is a sequence diagram for explaining the operation of the product manufacturing system 1001 according to this embodiment. 6 and 7 are schematic diagrams showing screen transitions of the terminal device 1010.
First, the entire object 1007 is imaged multiple times via the terminal device 1010 so that the object 1007 is imaged from different directions, and a plurality of image data of the object 1007 is generated (T1001). Here, a plurality of front and side pictures as shown in FIGS. 6 and 7 are taken.

Next, the user 1005 inputs the attribute data indicating the attribute of the object 1007 to the terminal device 1010 (T1002). Here, as the attribute data, full length data, weight data, elapsed time data (including age, etc.) of the object 1007, etc. are input.
Then, the plurality of image data and attribute data are transmitted from the terminal device 1010 to the dimension data calculation device 1020.

When the dimension data calculation device 1020 receives a plurality of image data and attribute data from the terminal device 1010, the dimension data calculation device 1020 calculates the dimension data of each part of the object 1007 using these data (T1003). The terminal device 1010 displays the dimension data on the screen according to the setting.

Then, the product manufacturing apparatus 1030 manufactures the desired product 1006 based on the dimension data calculated by the dimension data calculation apparatus 1020 (T1004).

(1-5-3) Characteristics of Product Manufacturing System As described above, the product manufacturing system 1001 according to the present embodiment includes the dimension data calculation device 1020 capable of communicating with the terminal device 1010 owned by the user 1005, and the product manufacturing system. Apparatus 1030. The terminal device 1010 (imaging device) captures a plurality of images of the object 1007. The dimension data calculation device 1020 includes an acquisition unit 1024A, an extraction unit 1024B, a conversion unit 1024C, and a calculation unit 1024D. The acquisition unit 1024A acquires the image data of the object 1007 from the terminal device 1010 together with the full length data of the object 1007. The extraction unit 1024B extracts shape data indicating the shape of the object 1007 from the image data. The conversion unit 1024C converts the shape data based on the full length data to form a silhouette. The calculation unit 1024D calculates the dimension data of each part of the object 1007 using the shape data converted by the conversion unit 1024C. The product manufacturing apparatus 1030 manufactures the product 1006 using the dimension data calculated by the calculation unit 1024D.

With such a configuration, the dimension data calculation device 1020 calculates each part of the target object 1007 with high accuracy, and thus a desired product related to the shape of the target object 1007 can be provided.

For example, the product manufacturing system 1001 can manufacture a model of an organ by measuring the shapes of various organs such as the heart.
Further, for example, various healthcare products and the like can be manufactured by measuring the waist shape of a person.
Also, for example, a person's figure product can be manufactured from the person's shape.
Further, for example, a chair or the like suitable for a person can be manufactured from the shape of the person.
Also, for example, car toys can be manufactured from car shapes.
Further, for example, a diorama or the like can be manufactured from an arbitrary landscape painting.

Note that, in the above description, the dimensional data calculation device 1020 and the product manufacturing device 1030 are described as separate device devices, but they may be integrally configured.

<Second Embodiment>
Hereinafter, the configurations and functions that have already been described will be denoted by the same reference numerals, and description thereof will be omitted.
(2-1) Configuration of Dimension Data Calculation Device FIG. 8 is a schematic diagram showing the configuration of the dimension data calculation device 2120 according to this embodiment.

The dimension data calculation device 2120 can be realized by an arbitrary computer and includes a storage unit 2121, an input/output unit 2122, a communication unit 2123, and a processing unit 2124. The dimension data calculation device 2120 may be realized as hardware using an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

The storage unit 2121 stores various kinds of information, and is realized by an arbitrary storage device such as a memory and a hard disk. For example, the storage unit 2121 stores the weighting factor Wri necessary for executing the information processing described later in association with the length and weight of the target object. The weighting factor is acquired in advance by performing machine learning from teacher data including attribute data and size data described later.

The input/output unit 2122 has the same configuration and function as the input/output unit 1022 described above.
The communication unit 2123 has the same configuration and function as the communication unit 1023 described above.

The processing unit 2124 executes various types of information processing, and is realized by a processor such as a CPU or GPU and a memory. Here, the processing unit 2124 functions as the acquisition unit 2124A and the calculation unit 2124D by reading the program stored in the storage unit 2121 into the CPU, GPU, or the like of the computer.

The acquisition unit 2124A acquires the attribute data Dzr (r is the number of elements of the attribute data) including at least one of the full length data, the weight data, and the elapsed time data (including age) of the object.

The calculation unit 2124D calculates the dimension data of each part of the object using the attribute data acquired by the acquisition unit 2124A. Specifically, the calculation unit 2124D calculates the dimension data of each part of the target object by performing a quadratic regression on the attribute data using the weight coefficient Wsi that has been machine-learned. The weighting factor is optimized for each part of the object, and the weighting factor of the i-th part of the object is denoted by Wsi. Note that i=1 to j, and j is the total number of dimension portions for which dimension data is to be calculated. The symbol s is the number of elements used for the calculation obtained from the attribute data.

Specifically, as shown in FIG. 9, the calculation unit 2124D calculates a value obtained by squaring each element x1, x2, and x3 of the attribute data Dzr (r=3) (the value in the first row in FIG. Value), a value obtained by multiplying each element (a value on the second line in FIG. 9, also referred to as an interaction term), a value of each element itself (a value on the third line in FIG. 9, and a primary (Also referred to as a term) is used to calculate dimensional data. Note that in the example shown in FIG. 9, there are nine values obtained from the three elements x1, x2, and x3 of the attribute data, and s=9, so the weighting factor Wsi has i×9 elements. Become.
(2-2) Features of dimensional data calculation device

As described above, the dimension data calculation device 2120 according to this embodiment includes the acquisition unit 2124A and the calculation unit 2124D. The acquisition unit 2124A acquires the attribute data including at least one of the full length data, the weight data, and the elapsed time data of the target object. The calculation unit 2124D calculates the dimension data of each part of the object using the attribute data.

Therefore, since the dimension data calculation device 2120 calculates the dimension data of each part of the object using the attribute data, it is possible to provide highly accurate dimension data. Specifically, the calculation unit 2124D performs quadratic regression on the attribute data using the machine-learned coefficient to calculate the dimension data of each part of the object with high accuracy.
Further, since the dimension data calculation device 2020 can process a large number of data at once, it is possible to provide a large number of dimension data at high speed.

When the attribute data includes at least one of the full length data, the weight data, and the elapsed time data, the dimension data of each part of the living thing can be calculated with high accuracy.
Further, by incorporating the dimension data calculation device 2120 into a product manufacturing device that manufactures various products, it is possible to manufacture a product that conforms to the shape of the target object.

In the above description, the calculation unit 2124D calculates the dimension data of each part of the object by performing the quadratic regression of the attribute data, but the calculation of the calculation unit 2124D is not limited to this. .. The calculation unit 2124D may be a unit that linearly combines the attribute data to obtain the dimension data.

(2-3) Application to Product Manufacturing System FIG. 10 is a schematic diagram showing the concept of the product manufacturing system 2001S according to this embodiment.
The dimension data calculation device 2120 according to the present embodiment can also be applied to the product manufacturing system 2001S, similarly to the dimension data calculation device 1020 according to the first embodiment.

The terminal device 2010S according to the present embodiment only needs to accept the input of attribute data indicating the attribute of the target object 2007. Examples of the “attribute” include the total length, weight, and elapsed time (including age) from the generation of the object 1007.

Also, as described above, the processing unit 2124 of the dimension data calculation device 2120 functions as the acquisition unit 2124A and the calculation unit 2124D. The calculation unit 2124D calculates the dimension data of each part of the object 2007 using the attribute data acquired by the acquisition unit 2124A. Specifically, the calculation unit 2124D calculates the dimension data of each part of the object by performing a quadratic regression on the attribute data using the weight coefficient Wsi that has been machine-learned.

In the product manufacturing system 2001S, the dimensional data calculation device 2120 calculates each part of the target object 2007 with high accuracy, so that a desired product related to the shape of the target object 2007 can be provided. In addition, the product manufacturing system 2001S according to the second embodiment can exhibit the same effects as the product manufacturing system 1001 according to the first embodiment.

A dimension data calculation system according to an embodiment of the information processing apparatus, the information processing method, the product manufacturing apparatus, and the dimension data calculation apparatus of the present invention will be described below with reference to the accompanying drawings. In the following description of the embodiments, the information processing device and the dimension data calculation device are implemented as part of the dimension data calculation system.

In the accompanying drawings, the same or similar reference numerals are given to the same or similar elements, and the duplicate description of the same or similar elements may be omitted in the description of each embodiment. Further, the features shown in each embodiment can be applied to other embodiments as long as they do not contradict each other. Furthermore, the drawings are schematic and do not necessarily match actual dimensions and ratios. The drawings may include portions having different dimensional relationships and ratios.

Note that, in the following description, when a plurality of elements are collectively expressed by using a matrix or vector, they may be expressed in uppercase, and individual elements of the matrix may be expressed in lowercase. For example, a matrix Λ may be used to represent a set of shape parameters, and an element λ may be used to represent elements of the matrix Λ.

(3-1) Configuration of Dimension Data Calculation System FIG. 11 is a schematic diagram showing the configuration of the dimension data calculation system 3100 according to this embodiment. The dimension data calculation system 3100 includes a dimension data calculation device 3020 and a learning device 3025.

Each of the dimension data calculation device 3020 and the learning device 3025 can be realized by an arbitrary computer. The dimension data calculation device 3020 includes a storage unit 3021, an input/output unit 3022, a communication unit 3023, and a processing unit 3024. The learning device 3025 also includes a storage unit 3026 and a processing unit 3027. The dimension data calculation device 3020 and the learning device 3025 may be realized as hardware using an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

Each of the

storage units

3021 and 3026 stores various kinds of information, and is realized by an arbitrary storage device such as a memory and a hard disk. For example, the storage unit 3021 stores various data including the target engine 3021A, programs, information, and the like in order for the processing unit 3024 to execute information processing regarding dimension data calculation. The storage unit 3026 also stores training data used in the learning stage to generate the target engine 3021A.

The input/output unit 3022 is realized by a keyboard, a mouse, a touch panel, etc., and inputs various information to the computer and outputs various information from the computer.
The communication unit 3023 is realized by a network interface such as an arbitrary network card and enables communication with a communication device on the network by wire or wirelessly.

Each of the processing units 3024 and 3027 is realized by a processor such as a CPU (Central Processing Unit) and/or a GPU (Graphical Processing Unit) and a memory in order to execute various information processing. The processing unit 3024 functions as an acquisition unit 3024A, an extraction unit 3024B, a conversion unit 3024C, an estimation unit 3024D, and a calculation unit 3024E when the program stored in the storage unit 3021 is read by the CPU, GPU, etc. of the computer. .. Similarly, the processing unit 3027 functions as the preprocessing unit 3027A and the learning unit 3027B by the CPU, GPU, etc. of the computer reading the program stored in the storage unit 3026.

In the processing unit 3024 of the dimension data calculation device 3020, the acquisition unit 3024A acquires the image data of the object, and the attribute data such as the total length data and the weight data of the object. The acquisition unit 3024A acquires, for example, a plurality of pieces of image data obtained by shooting an object from a plurality of different directions with an imaging device.

The extraction unit 3024B extracts shape data indicating the shape of the object from the image data. Specifically, the extraction unit 3024B extracts the target area included in the image data by using the semantic segmentation algorithm (Mask R-CNN or the like) prepared for each type of the target object to extract the target object. The shape data of is extracted. The semantic segmentation algorithm can be constructed using training data in which the shape of the object is not specified.

Note that if the semantic segmentation algorithm is constructed using the training data of an object whose shape is unspecified, it may not always be possible to extract the shape of the object with high accuracy. In such a case, the extraction unit 3024B extracts the shape data of the target object from the target object area by the Grab Cut algorithm. This makes it possible to extract the shape of the object with high accuracy.

Further, the extraction unit 3024B separates the target object and the background image other than the target object by correcting the image of the target object specified by the grab cut algorithm based on the color image of the specific portion of the target object. Good. This makes it possible to generate the shape data of the object with higher accuracy.

The conversion unit 3024C converts the shape data into silhouettes based on the full length data. That is, the shape data of the object is converted to generate a silhouette image of the object. Thereby, the shape data is standardized. The conversion unit 3024C also functions as a reception unit for inputting the generated silhouette image to the estimation unit 3024D.

The estimation unit 3024D estimates the values of a predetermined number of shape parameters from the silhouette image. The object engine 3021A is used for the estimation. The value of the shape parameter of the predetermined number of objects estimated by the estimation unit 3024D is associated with the dimension data related to an arbitrary part of the object.

The calculation unit 3024E calculates, from the values of the predetermined number of shape parameters estimated by the estimation unit 3024D, the dimension data of the object associated with the shape parameter values. Specifically, the calculation unit 3024E configures three-dimensional data of a plurality of vertices in the object from the shape parameter values of the object estimated by the estimation unit 3024D, and further, the target based on the three-dimensional data. Calculate dimensional data between any two vertices of an object.

In the processing unit 3027 of the learning device 3025, the preprocessing unit 3027A carries out various preprocessing for learning. In particular, the pre-processing unit 3027A specifies a predetermined number of shape parameters through feature extraction of the three-dimensional data of the sample object by dimension reduction. Also, a predetermined number (dimension) of shape parameter values is obtained for each sample object. The value of the shape parameter of the sample object is stored in the storage unit 3026 as training data.

Further, the preprocessing unit 3027A virtually constructs a three-dimensional object of the sample object in the three-dimensional space based on the three-dimensional data of the sample object, and then virtually provides it in the three-dimensional space. A silhouette image of a sample target object is generated by projecting a three-dimensional object from a predetermined direction using an imaging device. Data of the generated silhouette image of the sample object is stored in the storage unit 3026 as training data.

The learning unit 3027B learns to associate the relationship between the silhouette image of the sample object and the values of the predetermined number of shape parameters associated with the sample object. As a result of the learning, the target engine 3021A is generated. The generated object engine 3021A can be held in the form of an electronic file. When the dimension data calculation device 3020 calculates the dimension data of the object, the object engine 3021A is stored in the storage unit 3021 and referred to by the estimation unit 3024D.

(3-2) Operation of Dimension Data Calculation System The operation of the size data calculation system 3100 of FIG. 11 will be described with reference to FIGS. 12 and 13. FIG. 12 is a flowchart showing the operation (S3010) of the learning device 3025, which generates the target engine 3021A based on the sample target data. FIG. 13 is a flowchart showing the operation of the dimension data calculation device 3020, and calculates the dimension data of the target object based on the image data of the target object.

(3-2-1) Operation of Learning Device First, data of the sample target is prepared and stored in the storage unit 3026 (S3011). In one example, the data provided is data for 400 sample objects, including 5,000 three-dimensional data for each sample object. The three-dimensional data includes three-dimensional coordinate data of the vertices of the sample object. Further, the three-dimensional data includes apex information of each mesh forming the three-dimensional object, mesh data such as a normal direction of each apex, full length data, weight data, and elapsed time data (including age and the like). Attribute data such as "" may be included.

The vertex number is associated with the three-dimensional data of the sample object. In the above example, three-dimensional data of 5,000 vertices are associated with vertex numbers #1 to #5,000 for each sample object. Further, all or part of the vertex number is associated with the information on the part of the object. For example, when the object is a "person", the vertex number #20 is associated with the "head apex", and similarly, the vertex number #313 is the "acromion of the left shoulder" and the vertex number #521 is the "shoulder of the right shoulder". Associated with a "peak".

Subsequently, the preprocessing unit 3027A performs feature conversion into shape parameters by dimension reduction (S3012). Specifically, for each sample object, feature extraction is performed by dimension reduction of the three-dimensional data of the sample object. As a result, a predetermined number (number of dimensions) of shape parameters are obtained. In one example, the dimensionality of the shape parameter is 30. Dimension reduction is realized by methods such as principal component analysis and Random Projection.

The preprocessing unit 3027A converts the three-dimensional data for each sample object into a predetermined number of shape parameter values using the projection matrix of the principal component analysis. This makes it possible to remove noise from the three-dimensional data of the sample object and compress the three-dimensional data while maintaining related characteristic information.

As in the above example, there are 400 pieces of sample object data, each sample object includes three-dimensional (coordinate) data of 5,000 vertices, and each three-dimensional data is characterized by a 30-dimensional shape parameter. It is supposed to be converted. Here, the vertex coordinate matrix of [400 rows, 15,000 columns (5,000×3)] representing the data of 400 sample objects is defined as a matrix X. A matrix W is a projection matrix of [15,000 rows, 30 columns] generated by the principal component analysis. By multiplying the vertex coordinate matrix X by the projection matrix W from the right, a matrix Λ that is a shape parameter matrix of [400 rows, 30 columns] can be obtained.

That is, the shape parameter matrix Λ can be calculated from the following equation.

As a result of the matrix calculation using the projection matrix W, the 15,000-dimensional data of each of the 400 sample objects is converted into the shape parameter (λ ₁ ,..., λ ₃₀ ) of the 30-dimensional main component. To be done. In the principal component analysis, the average value of 400 values (λ ₁ , ₁ ,..., λ _400,1 ) for λ _i is calculated to be zero.

After obtaining the shape parameter matrix Λ as a result of S3012, the preprocessing unit 3027A expands the data set of the shape parameters included in the shape parameter matrix Λ using random numbers (S3013). In the above example, 400 data sets (λ _i,1 ,..., λ _i,30 (1≦i≦400)) are converted to 10,000 extended data sets (λ _j ) of shape parameters. , 1,..., λ _j,30 (1≦j≦10,000)). Data expansion is performed using random numbers having a normal distribution. The expanded data set has a normal distribution with a variance of 3σ for each shape parameter value.

By performing the inverse transformation based on the projection matrix W on the extension data set, the three-dimensional data of the extension data set can be constructed. In the above example, a [10,000 rows, 30 columns] extended shape parameter matrix representing 10,000 extended data sets is further transformed into a matrix Λ′(λ _j,k ,..., λ _{j, k} (1≦j≦10,000 and 1≦k≦30)). The vertex coordinate matrix X′ representing the three-dimensional data of 10,000 sample objects is [30 rows, 15,000 columns] the transposed matrix W ^T of the projection matrix W with respect to the extended shape parameter matrix Λ′. It is obtained by multiplying from the right.

That is, the vertex coordinate matrix X′ can be calculated from the following formula, and 5,000 (15,000/3) three-dimensional data are obtained for each sample object expanded to 10,000. be able to.

After the vertex coordinate matrix X′ is obtained as a result of S3013, the preprocessing unit 3027A generates each silhouette image based on the expanded three-dimensional data of the sample object (S3014). In the above example, for every 10,000 sample objects, a three-dimensional object of the sample object is virtually constructed from 5,000 three-dimensional data in the three-dimensional space. Then, a three-dimensional object is projected using a projection device that is also virtually provided in the three-dimensional space and is capable of projecting from any direction. In the above example, it is preferable that two silhouette images in the front direction and the side direction are acquired by projection for every 10,000 sample objects. The acquired silhouette image is represented by monochrome binarized data.

Finally, the learning unit 3027B associates the relationship between the shape parameter value associated with the sample object and the silhouette image of the sample object by learning (S3015). Specifically, it is preferable that the pair of the shape parameter data set obtained in S3013 and the silhouette image obtained in S3014 is used as training data, and the relationship between the two is learned by deep learning.

More specifically, for the sample objects expanded to 10,000 in the above example, the binary data of each silhouette image is input to the deep learning network architecture. In feature extraction in deep learning, the weighting factor of the network architecture is set so that the data output from the network architecture approaches the values of the 30 shape parameters. Note that the deep learning here can use a convolutional neural network (CNN: Convolutional Neural Network) in one example.

In this way, in S3014, the relationship between the value of the shape parameter associated with the sample object and the silhouette image of the sample object is learned by deep learning, and a deep learning network architecture is constructed. As a result, the object engine 3021A, which is an estimation model for estimating the value of the shape parameter, is generated in response to the input of the silhouette image of the object.

(3-2-2) Operation of Dimension Data Calculating Device The dimension data calculating device 3020 preliminarily stores the electronic file of the target engine 3021A generated by the learning device 3025 and the projection information of the principal component analysis obtained by the learning device 3025. It is stored in the storage unit 3021 and used for calculating the dimension data of the object.

First, the acquisition unit 3024A acquires, through the input/output unit 3122, a plurality of image data obtained by photographing the entire object from different directions via an external terminal device or the like, together with the full length data indicating the total length of the object ( S3021). Next, the extraction unit 3024B extracts shape data indicating the shape of each part of the object from each image data (S3022). Subsequently, the conversion unit 3024C executes rescaling processing for converting each shape data into a predetermined size based on the full length data (S3023). Through the steps of S3021 to S3023, the silhouette image of the target object is generated, and the dimension data calculation device 3020 receives the silhouette image of the target object.

Next, using the target engine 3021A stored in the storage unit 3021 in advance, the estimation unit 3024D estimates the value of the shape parameter of the target object from the received silhouette image (S3024). Then, the calculation unit 3024E calculates the dimension data related to the part of the target object based on the value of the shape parameter of the target object (S3025).

Specifically, in the calculation of the dimension data in S3025, first, the three-dimensional data of the vertex of the object is constructed from the values of the predetermined number of shape parameters estimated by the object engine 3021A for the object. Here, the inverse transformation of the projection (S3010) related to the dimension reduction performed by the preprocessing unit 3027A in the learning stage may be performed. More specifically, three-dimensional data can be obtained by multiplying the transposed matrix WT of the projection matrix W related to the principal component analysis from the right to the estimated number of shape parameter values (column vectors).

In the above-mentioned example, the three-dimensional data X″ of the object can be calculated from the following formula for the value Λ″ of the shape parameter of the object.

In S3025, the calculation unit 3024E calculates the dimension data between any two apexes of the object using the three-dimensional data. Here, a three-dimensional object is virtually constructed from the three-dimensional data, and dimension data between two vertices is calculated along a curved surface on the three-dimensional object. That is, the distance between the two vertices can be calculated three-dimensionally along the three-dimensional shape of the three-dimensional object. In order to three-dimensionally calculate the distance between two vertices, first, two vertices on a three-dimensional mesh composed of a large number of vertices (5,000 three-dimensional data in the above example) are calculated. The shortest route connecting the routes is searched and the mesh through which the shortest route passes is specified. Next, using the vertex coordinate data of the identified mesh, the distance is calculated for each mesh along the shortest path and summed. The total value is the three-dimensional distance between the two vertices. In addition, for the calculation of the three-dimensional distance, mesh information such as vertex information of each mesh forming the three-dimensional object and a normal direction of each vertex can be used.

For example, assume that the object is a "person" and the "shoulder width" of the person is calculated. As a preliminary preparation, it is specified in advance that “shoulder width=distance between the apex indicating the acromion of the left shoulder and the apex indicating the acromion of the right shoulder”. Further, it is associated in advance that the apex number of the apex indicating the acromion of the left shoulder is #313, and the apex number of the apex indicating the acromion of the right shoulder is #521, for example. These pieces of information are stored in the storage unit 3021 in advance. When calculating the dimension data, the shortest path from the vertex numbers #313 to #521 is specified, and the vertex coordinate data of the mesh specified in relation to the shortest path is used to calculate the distance for each mesh along the shortest path. Can be calculated and summed.

As described above, the dimension data calculation device 3020 according to the present embodiment can highly accurately estimate the value of the predetermined number of shape parameters from the silhouette image by using the target engine 3021A. In addition, since the three-dimensional data of the object can be restored with high accuracy from the value of the shape parameter estimated with high accuracy, not only the specific part but also any two vertices can be used as the measurement target part with high accuracy. Can be calculated. In particular, the calculated dimensional data between the two vertices is highly accurate because it is calculated along a three-dimensional shape based on a three-dimensional object composed of three-dimensional data.

(3-3) Characteristics of Dimension Data Calculation System a) As described above, the dimension data calculation system 3100 according to the present embodiment includes the dimension data calculation device 3020 and the learning device 3025. The information processing device configured as a part of the dimension data calculation device 3020 includes a conversion unit (reception unit) 3024C and an estimation unit 3024D. The conversion unit (reception unit) 3024C receives the silhouette image of the target object. The estimation unit 3024D uses the object engine 3021A that associates the silhouette image of the sample object with the values of the predetermined number of shape parameters associated with the sample object to determine the shape parameter of the object from the received silhouette image. Estimate the value. Then, the value of the estimated shape parameter of the object is associated with the dimensional data relating to an arbitrary part of the object.

Further, the dimension data calculation device 3020 includes an acquisition unit 3024A, an extraction unit 3024B, a conversion unit 3024C, an estimation unit 3024D, and a calculation unit 3024E. The acquisition unit 3024A acquires image data of a captured object and full-length data of the object. The extraction unit 3024B extracts shape data indicating the shape of the object from the image data. The conversion unit 3024C converts the shape data into a silhouette image based on the full length data. The estimation unit 3024D uses the object engine 3021A that associates the silhouette image of the sample object with the value of the predetermined number of shape parameters associated with the sample object to obtain the value of the predetermined number of shape parameters from the silhouette image. presume. The calculation unit 3024E calculates the dimension data of the target object based on the estimated values of the predetermined number of shape parameters.

Therefore, the dimension data calculation device 3020 can highly accurately estimate the values of the predetermined number of shape parameters from the silhouette image by using the target object engine 3021A that has been created in advance. Further, by using the value of the shape parameter estimated with high accuracy, it is possible to efficiently and highly accurately calculate the data related to an arbitrary part of the object. As described above, according to the dimension data calculation device 3020, the dimension data calculated for the object can be efficiently provided with high accuracy.

By using the dimension data calculation device 3020, for example, the dimension data of each part of a living thing as an object can be calculated with high accuracy. Further, it is possible to highly accurately calculate the dimensional data of each part of an arbitrary object such as a car or various kinds of luggage as the object. Furthermore, by incorporating the dimension data calculation device 3020 into a product manufacturing device that manufactures various products, it is possible to manufacture a product that conforms to the shape of the object.

By using the dimension data calculation device 3020, for example, the dimension data relating to each part of the living thing as an object can be calculated with high accuracy. Further, it is possible to highly accurately calculate the dimensional data of each part of an arbitrary object such as a car or various kinds of luggage as the object. Further, by incorporating the dimension data calculation device 3020 into a product manufacturing device that manufactures various products, it is possible to manufacture a product that conforms to the shape of the object.

B) In the dimension data calculation device 3020, a predetermined number of shape parameters associated with the sample object are specified by dimensionally reducing the three-dimensional data of the sample object. In particular, dimension reduction is performed by principal component analysis. Thereby, noise can be effectively removed from the three-dimensional data of the sample object, and the three-dimensional data can be compressed.

c) In the dimension data calculation device 3020, the three-dimensional data of the object is calculated by inverse transformation of the projection related to the above-mentioned principal component analysis with respect to the estimated value of the shape parameter, and the three-dimensional data is the dimension data. Associated with. This makes it possible to accurately configure the three-dimensional data of the target with respect to the input of the silhouette image of the target.

D) In the dimension data calculation device 3020, the silhouette image of the sample target object is a projection image in a predetermined direction on a three-dimensional object composed of the three-dimensional data of the sample target object. That is, a silhouette image is obtained by constructing a three-dimensional object using the three-dimensional data of the sample object and then projecting the three-dimensional object. It is preferable to obtain two silhouette images in the front direction and the side direction by projection. Thereby, the silhouette image of the sample object can be generated with high accuracy.

E) In the dimension data calculation device 3020, the object engine 3021A is generated by learning the relationship between the silhouette image of the sample object and the values of the predetermined number of shape parameters associated with the sample object. The learning can be performed by deep learning. By performing learning by deep learning, the silhouette image of the sample object and the shape parameter value of the sample object can be associated with high accuracy.

F) The calculation unit 3024E of the dimension data calculation device 3020 constructs three-dimensional data of a plurality of vertices of the object from the values of the predetermined number of shape parameters estimated for the object. Then, the dimension data between any two apexes of the object is calculated based on the constructed three-dimensional data. That is, the dimension data is calculated after the three-dimensional object is constructed using the three-dimensional data of the object. Accordingly, the dimension data between the two vertices can be calculated from the shape of the three-dimensional object of the object, so that the measurement target location is not limited to a specific portion.

Particularly, in the calculation unit 3024E of the dimension data calculation device 3020, the dimension data between the two vertices is calculated along the curved surface on the three-dimensional object including the three-dimensional data of the plurality of vertices in the object. .. Thereby, the dimension data can be calculated with higher accuracy.

(3-4) Modification (3-4-1)
In the above description, the acquisition unit 3024A acquires a plurality of image data obtained by photographing the object from different directions. Here, a depth data measuring device capable of acquiring depth data together can be applied as an imaging device capable of simultaneously photographing an object from a plurality of different directions. An example of the depth data measuring device is a stereo camera. In the present specification, the “stereo camera” means an imaging device of any form that simultaneously captures an object from a plurality of different directions and reproduces binocular parallax. On the other hand, plural pieces of image data are not necessarily required, and it is possible to calculate the dimension data of each part even if the image data of the object is one piece.

When the stereo camera described above is applied as the depth data measuring device, the image data that can be acquired by the acquisition unit 3024A can be RGB-D (Red, Green, Blue, Depth) data. Specifically, the image data can include a depth map having depth data for each pixel based on the depth data, in addition to the RGB image data that can be acquired by a normal monocular camera.

(3-4-2)
In the above description, the semantic segmentation algorithm or the grab cut algorithm may be adopted to extract the shape data of the object, and the object and the background image other than the object may be separated. In addition to or instead of this, for example, when a stereo camera is used, the depth map acquired from the stereo camera is used to acquire the shape data of the object in association with the depth data of the object. The background image may be separated. This makes it possible to generate the shape data of the object with higher accuracy.

Specifically, when a stereo camera is used, the extraction unit 3024B, based on the depth map acquired by the acquisition unit 3024A, from the image data in which the target object is captured, the target object region that is the part in which the target object is captured. Is better to extract. For example, the object region is extracted by removing the region whose depth data is not within the predetermined range from the depth map. In the extracted object region, the shape data is associated with the object depth data on a pixel-by-pixel basis.

The conversion unit 3024C converts the shape data into new shape data based on the depth data of the target area in addition to the above-described full length data, and generates a “gradation silhouette image” (described later).

The gradation silhouette image to be generated is not simple black and white binarized data, but is a single color represented by data with a brightness value of 0 (“black”) to 1 (“white”) based on depth data. It is a multi-tone monochrome image. That is, the gradation silhouette image data is associated with the depth data and has a larger amount of information regarding the shape of the object.

When using the gradation silhouette image, the processing in the processing unit 3027 of the learning device 3025 is preferably performed as follows. The gradation silhouette image data is standardized by full length data.

In the pre-processing unit 3027A, when the three-dimensional object of the sample target is projected from the predetermined direction by using the image capturing device virtually provided in the three-dimensional space, the depth data from the image capturing device to the sample target is also collected. Good to get. That is, the silhouette image data of the sample object is associated with the depth data. The generated gradation silhouette image of the sample object is, for example, a monochromatic multi-tone monochrome image with a brightness value of 0 (“black”) to 1 (“white”) based on the depth data. Has much more information about the shape of.

When the object engine 3021A is generated by the learning unit 3027B, learning is performed so as to associate a predetermined number of shape parameter values regarding the sample object with the gradation silhouette image of the sample object associated with the depth data. Is good. That is, since the learning process based on the depth data of the sample target in the learning device 3025 is based on a larger amount of information, the target engine 3021A with higher accuracy can be generated.

According to this modification, the shape data of the object is generated with higher accuracy by extracting the object region based on the depth map configured by the acquisition unit 3024A using any machine capable of measuring the depth data. It becomes possible to do. Further, the gradation silhouette image data (the shape data to be converted) is associated with the depth data of the object. The target engine 3021A is also generated as a result of the learning process based on the depth data. Therefore, the calculation unit 3024E can calculate the dimensional data of each part of the target object with higher accuracy, because the size information of the target object is larger.

Furthermore, in addition to stereo cameras, it is possible to obtain depth data with a LiDAR (Light Detection and Ranging) device and separate the target and background images other than the target. That is, by using an arbitrary machine (depth data measuring device) capable of measuring depth data in the acquisition unit 3024A, it becomes possible to generate the shape data of the object with high accuracy.

(3-4-3)
In the above description, in step S3013, the preprocessing unit 3027A performs data expansion processing for expanding the shape parameter data set using random numbers. In the data expansion processing, it is only necessary to determine the number of samples to be expanded according to the number of sample objects. If the number of samples is sufficiently prepared in advance, the expansion process of S3013 may not be performed.

(3-4-4)
In the above description, the shape parameters (λ ₁ ,..., λ ₃₀ ) of the sample object are acquired by the principal component analysis in the preprocessing unit 3027A of the learning device 3025. Here, the shape parameter when the sample object is a “person” will be further considered. As a result of performing the principal component analysis using the three-dimensional data of 400 sample objects and 5,000 vertices as in the above example, the shape parameter when the object is “person” is at least It was considered to have properties.

[Characteristic 1]
The first-ranked principal component λ ₁ was associated to have a linear relationship with human height. Specifically, as shown in FIG. 14, the larger the first-order principal component λ ₁ , the smaller the person's height.

Considering the characteristic 1, when estimating the value of the human shape parameter by the estimation unit 3024D, the height data acquired by the acquisition unit 3024A without using the target engine 3021A is used for the first-order principal component λ _1. You can use it. Specifically, the value of the first-order principal component λ ₁ may be configured to be calculated separately by using a linear regression model in which the height of a person is an explanatory variable.

In this case, even when the learning unit 3027B generates the target engine 3021A in the learning stage, the main component λ1 may be excluded from the learning target. As described above, in the learning stage in the learning device 3025, the network architecture is weighted. At this time, the error between the second and subsequent main components excluding the first-priority main components obtained by performing the principal component analysis of the input silhouette image and the values after the shape parameter λ2, which is the training data, is minimum. The weighting factors of the network architecture may be set to be customized. As a result, when the object is a “person”, the estimation accuracy of the value of the shape parameter in the estimation unit 3024D can be improved in addition to the use of the linear regression model.

Further, when the object engine 3021A is generated in the learning unit 3027B in the learning stage, the weighting coefficient of the network architecture is minimized so as to minimize the error with the values after the shape parameter λ1 including the main component of the first rank. May be set. Then, the value of the first-order principal component λ1 may be replaced with a value calculated separately by using a linear regression model in which the height of a person is used as an explanatory variable. As a result, when the object is a “person”, the estimation accuracy of the value of the shape parameter in the estimation unit 3024D can be improved in addition to the use of the linear regression model.

[Characteristic 2]
FIG. 15 is a schematic graph showing the recall of the shape parameter which is the main component. In the graph, the horizontal axis represents the principal component ranked by the contribution rate, and the vertical axis represents the variance explanation rate of the eigenvalue. Then, the bar graph shows individual variance explanation rates for each rank. Further, the solid line graph shows the accumulation of the variance explanation rates from the first rank. In FIG. 15, for simplification, a graph regarding 10 principal components up to the 10th rank is schematically shown. The eigenvalue of the covariance matrix obtained in the principal component analysis represents the size of the eigenvector (principal component), and the variance explanation ratio of the eigenvalue may be considered as the recall ratio for the principal component.

Referring to the graph in FIG. 15, the cumulative variance ratios of the ten principal components from the first rank to the tenth rank show about 0.95 (broken line arrow). That is, it is understood by those skilled in the art that the recall rate of the ten principal components from the first rank to the tenth rank is about 95%. That is, in the above-mentioned example, the shape parameter subjected to the feature conversion by dimension reduction is 30 dimensions, but the shape parameter is not limited to this, and even if the shape parameter is 10 dimensions, about 95% can be covered. That is, although the number of shape parameters (the number of dimensions) is 30 in the above description, it may be about 10 in consideration of the characteristic 2.

(3-4-5)
In the above description, in the preprocessing unit 3027A of the learning device 3025, two silhouette images in the front direction and the side direction are obtained by projection for every 10,000 sample objects. On the other hand, it is not always necessary to have two silhouette images of the target object, but only one.

(3-5) Application to Product Manufacturing System An example in which the above-described dimension data calculation device 3020 is applied to the product manufacturing system 3001 will be described below.

(3-5-1) Configuration of Product Manufacturing System FIG. 16 is a schematic diagram showing the concept of the product manufacturing system 3001 according to this embodiment. The product manufacturing system 3001 is a system for manufacturing a desired product 3006, including a dimension data calculation device 3020 capable of communicating with the terminal device 3010 owned by the user 3005 and a product manufacturing device 3030. In FIG. 16, as an example, the concept when the object 3007 is a person and the product 3006 is a chair is shown. However, in the product manufacturing system 3001 according to the present embodiment, the object 3007 and the product 3006 are not limited to these.

The terminal device 3010 can be realized by a so-called smart device. Here, the terminal device 3010 exerts various functions by installing the user program in the smart device. Specifically, the terminal device 3010 generates image data captured by the user 3005. Here, the terminal device 3010 may have a stereo camera function of simultaneously capturing images of a target object from a plurality of different directions and reproducing binocular parallax. Note that the image data is not limited to that captured by the terminal device 3010, and, for example, data captured using a stereo camera installed in a store may be used.

Also, the terminal device 3010 accepts input of attribute data indicating the attribute of the target object 3007. The “attribute” includes the total length, weight, and elapsed time (including age) from the generation of the object 3007. In addition, the terminal device 3010 has a communication function, and executes transmission/reception of various information between the terminal device 3010 and the dimension data calculation device 3020 and the product manufacturing device 3030.

The dimension data calculation device 3020 can be realized by any computer. Here, the storage unit 3021 of the dimension data calculation device 3020 stores the information transmitted from the terminal device 3010 in association with the identification information that identifies the user 3005 of the terminal device 3010. The storage unit 3021 also stores parameters and the like necessary for executing information processing for calculating dimension data.

Also, the processing unit 3024 of the dimension data calculation device 3020 functions as the acquisition unit 3024A, the extraction unit 3024B, the conversion unit 3024C, the estimation unit 3024D, and the calculation unit 3024E, as described above. Here, the acquisition unit 3024A acquires the image data captured by the stereo camera by the user 3005 and the attribute data of the target object 3007. The extraction unit 3024B also extracts shape data indicating the shape of the object 3007 from the image data. For example, when “person” is set in advance as the type of object, a semantic segmentation algorithm is constructed using training data for identifying a person. Further, the extraction unit 3024B may separate the target object and the background image other than the target object by using the depth map based on the depth data acquired from the stereo camera. In this case, the conversion unit 3024C converts the shape data associated with the depth data of the object in the depth map into a gradation silhouette image based on the full length data. The generated gradation silhouette image is preferably a monochrome image with a single color and multiple gradations based on the depth data. The conversion unit 3024C also functions as a reception unit for inputting the generated silhouette image to the estimation unit 3024D.

The estimation unit 3024D uses the object engine 3021A that associates the silhouette image of the sample object with the value of the predetermined number of shape parameters associated with the sample object to obtain the value of the predetermined number of shape parameters from the silhouette image. presume. The calculation unit 3024E calculates the dimension data of the target object based on the estimated values of the predetermined number of shape parameters. Specifically, the shape parameter values of the target object estimated by the estimation unit 3024D form three-dimensional data of a plurality of vertices of the target object, and further, based on the three-dimensional data, an arbitrary 2 of the target object is calculated. Calculate dimensional data between two vertices.

The product manufacturing apparatus 3030 is a manufacturing apparatus that manufactures a desired product related to the shape of the object 3007 by using at least one size data calculated using the size data calculation device 3020. Note that the product manufacturing apparatus 3030 can employ any device that can automatically manufacture and process a product, and can be realized by, for example, a three-dimensional printer.

(3-5-2) Operation of Product Manufacturing System FIG. 17 is a sequence diagram for explaining the operation of the product manufacturing system 3001 according to this embodiment. 18 and 19 are schematic diagrams showing screen transitions of the terminal device 3010.

First, a plurality of images of the target 3007 are captured via the terminal device 3010 so that the entire target 3007 is captured from different directions, and a plurality of image data of the captured target 3007 is generated (T3001). Here, a plurality of front and side photographs as shown in FIGS. 18 and 19 are taken. Such front and side photographs are preferably taken with the stereo camera function of the terminal device 3010 turned on.

Next, the user 3005 inputs the attribute data indicating the attribute of the target object 3007 to the terminal device 3010 (T3002). Here, as the attribute data, full length data, weight data, elapsed time data (including age, etc.) of the object 3007, etc. are input. Then, the plurality of image data and the attribute data are transmitted from the terminal device 3010 to the dimension data calculation device 3020.

When the dimension data calculation device 3020 receives a plurality of image data and attribute data from the terminal device 3010, the dimension data calculation device 3020 calculates the dimension data of each part of the object 3007 using these data (T3003). Note that the terminal device 3010 displays size data on the screen according to the settings. Then, the product manufacturing apparatus 3030 manufactures the desired product 3006 based on the dimension data calculated by the dimension data calculation apparatus 3020 (T3004).

(3-5-3) Characteristics of Product Manufacturing System As described above, the product manufacturing system 3001 according to the present embodiment includes the dimension data calculation device 3020 capable of communicating with the terminal device 3010 owned by the user 3005, and the product manufacturing system. A device 3030.

The terminal device 3010 (imaging device) captures a plurality of images of the object 3007. The dimension data calculation device 3020 includes an acquisition unit 3024A, an extraction unit 3024B, a conversion unit 3024C, an estimation unit 3024D, and a calculation unit 3024E. The acquisition unit 3024A acquires image data of a captured object and full-length data of the object. The extraction unit 3024B extracts shape data indicating the shape of the object from the image data. The conversion unit 3024C converts the shape data into a silhouette image based on the full length data. The estimation unit 3024D uses the object engine 3021A that associates the silhouette image of the sample object with the value of the predetermined number of shape parameters associated with the sample object to obtain the value of the predetermined number of shape parameters from the silhouette image. presume. The calculation unit 3024E calculates the dimension data of the target object based on the estimated values of the predetermined number of shape parameters. The product manufacturing apparatus 3030 manufactures the product 3006 using the dimension data calculated by the calculation unit 3024E. With such a configuration, the dimension data calculation device 3020 calculates each part of the target object 3007 with high accuracy, and thus a desired product related to the shape of the target object 3007 can be provided.

For example, the product manufacturing system 3001 can manufacture a model of an organ by measuring the shapes of various organs such as the heart. Further, for example, various healthcare products and the like can be manufactured by measuring the waist shape of a person. Also, for example, a person's figure product can be manufactured from the person's shape. Further, for example, a chair or the like suitable for a person can be manufactured from the shape of the person. Also, for example, car toys can be manufactured from car shapes. Also, for example, a diorama or the like can be manufactured from an arbitrary landscape picture.

Note that, in the above description, the dimension data calculation device 3020 and the product manufacturing device 3030 are described as separate device devices, but they may be integrally configured.

<Fourth Embodiment>
Hereinafter, the configurations and functions that have already been described will be denoted by the same reference numerals, and description thereof will be omitted.
(4-1) Configuration of Dimension Data Calculation Device FIG. 20 is a schematic diagram showing the configuration of the dimension data calculation system 4200 according to this embodiment. The dimension data calculation system 4200 includes a dimension data calculation device 4120 and a learning device 4125.

The dimension data calculation device 4120 includes a storage unit 4121, an input/output unit 4122, a communication unit 4123, and a processing unit 4124. The learning device 4125 also includes a storage unit 4126 and a processing unit 4127. The dimension data calculation device 4120 and the learning device 4125 may be realized as hardware using an LSI, ASIC, FPGA, or the like.

Each of the

storage units

4121 and 4126 stores various information, and is realized by an arbitrary storage device such as a memory and a hard disk. For example, the storage unit 4121 stores various data including the target engine 4121A, programs, information, and the like in order for the processing unit 4124 to execute information processing regarding dimension data calculation. The storage unit 4126 also stores training data used in the learning stage to generate the target engine 4121A.

The input/output unit 4122 has the same configuration and function as the input/output unit 3022 described above. The communication unit 4123 has the same configuration and function as the communication unit 3023 described above.

The processing unit 4124 functions as an acquisition unit 4124A, an estimation unit 4124D, and a calculation unit 4124E when the program stored in the storage unit 4121 is read by the CPU, GPU, etc. of the computer. Similarly, the processing unit 4127 functions as the preprocessing unit 4127A and the learning unit 4127B by the CPU, GPU, etc. of the computer reading the program stored in the storage unit 4126.

In the processing unit 4124 of the dimension data calculation device 4120, the acquisition unit 4124A acquires the attribute data including at least one of the total length data, weight data, and elapsed time data (including age) of the object. In the present embodiment, the acquisition unit 4124A also functions as a reception unit for inputting the attribute data to the estimation unit 4124D.

The estimating unit 4124D estimates the values of a predetermined number of shape parameters from the attribute data. The object engine 4121A is used for the estimation. The value of the shape parameter of the object estimated by the estimation unit 4124D can be associated with the dimension data related to an arbitrary part of the object, as described later.

The calculation unit 4124E calculates the dimension data of the object from the value of the shape parameter of the object estimated by the estimation unit 4124D. Specifically, the calculation unit 4124E configures three-dimensional data of a plurality of vertices in the object from the shape parameter values of the object estimated by the estimation unit 4124D, and further, based on the three-dimensional data, the target Calculate dimensional data between any two vertices of an object.

In the processing unit 4127 of the learning device 4125, the preprocessing unit 4127A carries out various preprocessing for learning. In particular, the preprocessing unit 4127A specifies a predetermined number of shape parameters by extracting the features of the three-dimensional data of the sample object by dimension reduction. The value of the shape parameter of the sample object and the corresponding attribute data are stored in the storage unit 4126 as training data in advance.

Note that it is assumed that the corresponding attribute data (full length data, weight data, elapsed time data (including age, etc.)) is prepared as three-dimensional data of the sample object. The corresponding attribute data is stored in the storage unit 4126 as training data.

The learning unit 4127B learns to associate the value of the shape parameter of the sample object with the corresponding attribute data. As a result of the learning, the target engine 4121A is generated. The generated object engine 4121A can be held in the form of an electronic file. When the dimension data calculation device 4120 calculates the dimension data of the object, the object engine 4121A is stored in the storage unit 4121 and referred to by the estimation unit 4124D.

(4-2) Operation of Dimension Data Calculation System With reference to FIGS. 21 and 22, the operation of the size data calculation system 4200 of FIG. 20 will be described. FIG. 21 is a flowchart showing the operation (S4110) of the learning device 4125, in which the target object engine 4121A is generated based on the sample target object data. FIG. 22 is a flowchart showing the operation of the dimension data calculation device 4120. Here, the dimension data of the target object is calculated based on the image data of the target object.

(4-2-1) Operation of Learning Device First, data of the sample target is prepared and stored in the storage unit 4126 (S4111). In one example, the prepared data is data of 400 sample objects, and includes 5,000 three-dimensional data prepared for each sample object and attribute data prepared for each sample object. Including. The three-dimensional data includes three-dimensional coordinate data of the vertices of the sample object. Further, the three-dimensional data may include apex information of each mesh forming the three-dimensional object and mesh data such as a normal direction of each apex.

Also, as in the first embodiment, the three-dimensional data of the sample object is associated with the part number information together with the vertex number.

Subsequently, the preprocessing unit 4127A performs feature conversion into a predetermined number (dimension) of shape parameters by dimension reduction (S4112). This feature conversion process is also the same as in the first embodiment. In the above example, as a result of the matrix calculation using the projection matrix according to the principal component analysis, the 15,000-dimensional (5,000×3) data that each of the 400 sample objects has is, for example, the 30-dimensional principal component. Is transformed into a shape parameter Λ of.

Then, the learning unit 4127B uses the combination of the attribute data of the plurality of sample objects prepared in S4111 and the data set of the plurality of shape parameters obtained in S4112 as the training data to machine-learn the relationship between them. Yes (S4115).

Specifically, the learning unit 4027B obtains the conversion attribute data Y from the attribute data of the object. An element of the conversion matrix Z that associates the element y _r of the conversion attribute data Y with the element λ _m of the shape parameter Λ is expressed as zrm. The conversion matrix Z is a matrix composed of [s rows, n columns]. The symbol m is 1≦m≦n, and n is 30 which is the number of dimensions of the shape parameter Λ in the above example. The symbol r is 1≦r≦s, and s is the number of elements used for the operation obtained from the conversion attribute data Y.

For example, it is assumed that the attribute data of the object consists of full length data h, weight data w, and elapsed time data a. That is, the attribute data is a set of (h,w,a) elements. In this case, the learning unit 4027B multiplies a value obtained by squaring each element (h, w, a) of the attribute data of the object (also called a quadratic term) and a value obtained by multiplying each element (also called an interaction term). .) and the value of each element itself (also called the primary term).

As a result, conversion attribute data Y having the following nine elements is obtained.

Next, the learning unit 4027B performs regression analysis on a set of the conversion attribute data Y obtained from the attribute data associated with 400 sample objects and the shape parameter Λ obtained from the three-dimensional data of the sample objects. As a result, a conversion matrix Z composed of [9 rows, 30 columns] as shown below is obtained.

The data of the conversion matrix Z thus obtained is stored in the storage unit 4026 as the object engine 4121A.

(4-2-2) Operation of Dimension Data Calculation Device The dimension data calculation device 4120 stores the electronic file of the object engine 4121A generated by the learning device 4125 and the projection information of the principal component analysis obtained by the learning device 4125. It is stored in the unit 4121 and used for calculating the dimension data of the object.

First, the acquisition unit 4124A acquires the attribute data of the target object via the input/output unit 4122 (S4121). Thereby, the attribute data of the target object is received. Next, using the target engine 4121A stored in advance in the storage unit 4121, the estimation unit 4124D estimates the value of the shape parameter of the target from the received attribute data (S4124).

For example, it is assumed that the attribute data of the object consists of full length data h, weight data w, and elapsed time data a. That is, the attribute data is a set of (h,w,a) elements. In S4124, as described above, the estimation unit 4124D calculates the value obtained by squaring each element (h, w, a) of the attribute data of the object, the value obtained by multiplying each element, and the value of each element itself. To obtain the conversion attribute data Y.

Finally, the calculation unit 4124E calculates the dimensional data related to the part of the object based on the value of the shape parameter of the object (S4125). Specifically, the conversion attribute data Y is calculated from the attribute data of the target object acquired by the acquisition unit 4124A. Then, the shape parameter Λ is calculated by multiplying the conversion attribute data Y by the above-mentioned conversion matrix Z. After this, similarly to the one in the third embodiment (S3025), a three-dimensional object is virtually constructed from the three-dimensional data, and the dimension data between the two vertices is calculated along the curved surface on the three-dimensional object. To do. In addition, for the calculation of the three-dimensional distance, mesh information such as vertex information of each mesh forming the three-dimensional object and a normal direction of each vertex can be used.

As described above, the dimension data calculation device 4120 of the present embodiment can highly accurately estimate the value of the predetermined number of shape parameters from the attribute data of the object by using the object engine 4121A. Unlike the third embodiment, there is no need to input an image of an object, and neither S3022 (shape data extraction processing) nor S3023 (rescale processing) of FIG. 13 is required, which is efficient.

In addition, since the three-dimensional data of the object can be restored with high accuracy from the value of the shape parameter estimated with high accuracy, not only the specific part but also any two vertices can be used as the measurement target part with high accuracy. Can be calculated. In particular, the calculated dimensional data between the two vertices is highly accurate because it is calculated along a three-dimensional shape based on a three-dimensional object composed of three-dimensional data.

(4-3) Characteristics of Dimension Data Calculation System As described above, the dimension data calculation system 4200 according to the present embodiment includes the dimension data calculation device 4120 and the learning device 4125. The information processing device configured as a part of the dimension data calculation device 4120 includes an acquisition unit (reception unit) 4124A, an estimation unit 4124D, and a calculation unit 4124E. The acquisition unit (reception unit) 4124A receives the attribute data of the target object. The estimation unit 4124D uses the target object engine 4121A that associates the attribute data of the sample target object with the values of the predetermined number of shape parameters associated with the sample target object, and then uses the target object shape parameter from the received attribute data. Estimate the value of. Then, the value of the estimated shape parameter of the object is associated with the dimensional data relating to an arbitrary part of the object.

Therefore, the dimension data calculation device 4120 can efficiently estimate the values of the predetermined number of shape parameters from the attribute data by using the object engine 4121A that has been created in advance. Moreover, the value of the estimated shape parameter is highly accurate. Further, by using the value of the shape parameter estimated with high accuracy, it is possible to efficiently and highly accurately calculate the data related to an arbitrary part of the object. As described above, according to the dimension data calculation device 4120, the dimension data calculated for the object can be efficiently provided with high accuracy.

(4-4) Application to Product Manufacturing System FIG. 23 is a schematic diagram showing the concept of the product manufacturing system 4001S according to this embodiment. The dimension data calculation device 4120 according to the present embodiment can also be applied to the product manufacturing system 4001S, similarly to the dimension data calculation device 3020 according to the third embodiment.

The terminal device 4010S according to the present embodiment only needs to accept the input of attribute data indicating the attribute of the target object 4007. Examples of the “attribute” include the total length, weight, and elapsed time (including age) from the generation of the object 1007.

Further, as described above, the processing unit 4124 of the dimension data calculation device 4120 functions as the acquisition unit 4124A, the estimation unit 4124D, and the calculation unit 4124E. The calculation unit 4124E calculates the dimension data related to the part of the target object based on the value of the shape parameter of the target object obtained by the estimation unit 4124D.

In the product manufacturing system 4001S, the dimension data calculation device 4120 efficiently and accurately calculates the dimension data of the target object 1007, so that a desired product related to the shape of the target object 1007 can be provided. In addition, the product manufacturing system 4001S according to the fourth embodiment can exhibit the same effects as the product manufacturing system 3001 according to the third embodiment.

<Other Embodiment 1: Silhouette Image Generation Device>
(5-1) Configuration of Silhouette Image Generating Device FIG. 24 is a schematic diagram showing the configuration of a silhouette image generating device 5020 according to another embodiment. The silhouette image (including the gradation silhouette image) generated in the first embodiment and the third embodiment may be generated according to this silhouette image generation device 5020. That is, the silhouette image generation device 5020 may be configured as a part of the dimension data calculation device 1020 according to the first embodiment or as a part of the dimension data calculation device 3020 according to the third embodiment.

The silhouette image generation device 5020 can be realized by any computer, and includes an acquisition unit 5024A, an extraction unit 5024B, and a conversion unit 5024C.

The acquisition unit 5024A may correspond to all or a part of the acquisition unit 1024A of the dimension data calculation device 1020 according to the first embodiment and/or the acquisition unit 3024A of the dimension data calculation device 3020 according to the third embodiment. The extraction unit 5024B corresponds to all or a part of the extraction unit 1024B of the dimension data calculation device 1020 according to the first embodiment and/or the extraction unit 3024B of the dimension data calculation device 3020 according to the third embodiment. Good. Similarly, the conversion unit 5024C corresponds to all or a part of the conversion unit 1024C of the dimension data calculation device 1020 according to the first embodiment and/or the conversion unit 3024C of the dimension data calculation device 3020 according to the third embodiment. You may.

The acquisition unit 5024A acquires image data in which the object is photographed. The acquisition unit 5024A acquires, for example, a plurality of pieces of image data obtained by shooting an object from a plurality of different directions with an imaging device. Here, a depth data measuring device capable of acquiring depth data is applicable, and a depth map having depth data for each pixel is configured based on the depth data. By applying the depth data measuring device, the image data that can be acquired by the acquisition unit 5024A can include RGB-D (Red, Green, Blue, Depth) data. Specifically, the image data can acquire such a depth map in addition to the RGB image data that can be acquired by a normal monocular camera.

An example of the depth data measuring device is a stereo camera, and the stereo camera is also applied in the following description. When photographing an object (particularly a person) with a stereo camera, it is preferable to guide the user so that the entire object is accommodated within a predetermined range of the display so that the object can be accurately specified. In one example, a guide area may be displayed on the display, or a guide message may be displayed to prompt the user. As a result, the target object can be positioned in a desired direction and distance from the stereo camera, and noise during silhouette image generation can be reduced.

The extraction unit 5024B extracts shape data indicating the shape of the object from the image data. More specifically, the extraction unit 5024B includes a three-dimensional point cloud generation unit 5124, a background point cloud removal unit 5224, a plane point cloud removal unit 5324, an object region extraction unit 5424, a shape data extraction unit 5524, and an object detection unit 5624. Including.

The 3D point cloud generation unit 5124 generates 3D point cloud data from the acquired depth map, and deploys a 3D point cloud consisting of a set of points in a virtual 3D coordinate space. Each point has three-dimensional coordinates in a virtual three-dimensional space. In the virtual three-dimensional coordinate space, the stereo camera is virtually arranged at the origin, and the three-dimensional coordinate (xyz) system is defined according to the orientation of the stereo camera. In particular, the optical axis direction of the stereo camera is defined as the depth direction (z-axis direction).

The background point cloud removing unit 5224 removes, from the generated three-dimensional point cloud data, the data of the three-dimensional point cloud existing apart from the predetermined distance along the depth direction of the virtual three-dimensional coordinate space. The point to be removed can be regarded as constituting a background image because it exists far from the stereo camera, and it is preferable to remove the background portion from the image data in which the object is photographed. As a result, the three-dimensional point group that becomes noise can be effectively removed, so that the accuracy of specifying the object area extracted by the object area extraction unit 5424 can be improved.

The plane point cloud removing unit 5324 removes the three-dimensional point cloud data existing corresponding to the plane portion from the generated three-dimensional point cloud data. In order to accurately specify the target object area, it is preferable to specify and remove the flat surface portion existing around the target object from the image data obtained by capturing the target object. For that purpose, it is necessary to estimate the plane part using the data of the generated three-dimensional point cloud.

Specifically, the plane point cloud removing unit 5324 may operate as follows.
First, the plane portion in the image data is estimated from the three-dimensional point cloud data generated from the depth map. The plane here is, for example, the floor. That is, when the object is a person, the plane portion is the floor portion which is in contact with the upright person.

Usually, the xyz coordinate plane in the three-dimensional space is expressed by the following equation f(x, y, z)=0.
f(x, y, z)=ax+by+cz+d=0

In one example, in the virtual three-dimensional coordinate space where the three-dimensional point cloud is arranged, one plane part is selected from a plurality of sample planes sampled by a known random sampling method. Typically, for random sampling, a robust estimation algorithm by RANSAC (Random Sample Consensus) can be applied.

More specifically, first, the sample plane is sampled by randomly determining the normal vectors (a, b, c) and d. Then, the points of the three-dimensional point group that satisfy the following inequalities are identified with respect to how many three-dimensional point groups are associated with the sample plane. DST is a predetermined threshold distance.
|f(x, y, z)|=|ax+by+cz+d|≦DST

Points that satisfy the above inequality (each has x, y, and z values) are considered to be on the sample plane of the three-dimensional point cloud. Theoretically, the threshold distance DST=0, but the threshold distance DST is 0 in order to include the three-dimensional point cloud data within a predetermined minute distance from the sample plane in consideration of the shooting environment and the performance of the stereo camera. It is better to set it to a value close to. Of a plurality of randomly determined sample planes, a sample plane having a large number of three-dimensional point groups satisfying the above inequality, that is, a sample plane having the largest content rate of the three-dimensional point group is a desired plane portion in the image data. Is estimated to be

In other words, the plane point cloud removing unit 5324 repeats the extraction of the sample planes a plurality of times and then determines the sample plane having the largest content rate of the three-dimensional point cloud, so that the robustness to the estimation of the desired plane portion is increased. be able to.

Subsequently, the plane point cloud removing unit 5324 removes the 3D point cloud data of the points existing in the estimated plane part from the generated 3D point cloud data. The plane portion having the points to be removed is, for example, the floor portion in the image data. That is, the plane point cloud removing unit 5324 can remove the floor portion from the image data in which the object is photographed. As a result, the three-dimensional point group that becomes noise can be effectively removed, so that it is possible to improve the accuracy of specifying the target object area of the target object extracted by the target object area extracting unit 5424.

Further, by repeating the processing by the plane point cloud removing unit 5324, another plane part may be further estimated, and the three-dimensional point cloud data of points existing in the other plane part may be further removed. For example, the floor portion is estimated, the three-dimensional point cloud data is once removed from the entire three-dimensional point cloud data, and then a plane is estimated again from the sample planes sampled by the random sampling method described above. With this, it is possible to estimate the wall portion. That is, not only the floor portion but also the three-dimensional point cloud data of the wall portion can be removed from the image data, and the accuracy of specifying the target object area of the target object can be further improved.

Note that the accuracy of estimation of the plane part also depends on the shooting environment when shooting the target object. For example, in order to accurately estimate the plane portion, it is necessary to make the number of points forming the plane portion larger than the number of points forming the object. Therefore, for example, it is preferable to allow the user to select a shooting environment in which many walls are not reflected, or fixedly install the stereo camera in the store in a place where many walls are not reflected.

The target area extraction unit 5424 extracts the target area of the target using the three-dimensional point cloud data. The three-dimensional point cloud removed from the three-dimensional point cloud generated from the depth map by the three-dimensional point cloud generator 5124 by the background point cloud remover 5224 and/or the plane point cloud remover 5324, that is, noise-removed Using the data, a three-dimensional point cloud corresponding to the object is further specified. For example, a three-dimensional point group in a predetermined space range in the virtual three-dimensional space may be specified. Then, the object area of the object in the image data can be extracted based on the specified three-dimensional point cloud data. The object region extracted in this way is effectively noise-removed and has high accuracy. Thereby, the accuracy of the silhouette image converted by the conversion unit 5024C can be further improved.

The shape data extraction unit 5524 extracts shape data indicating the shape of the target object based on the depth data of the region in the depth map corresponding to the target region extracted by the target region extraction unit 5424.

The object detection unit 5624 uses the RGB image data acquired by the acquisition unit 5024A to extract the image area of the target object in the image data by object detection. The image area of the object is defined by a two-dimensional (xy) coordinate area that is perpendicular to the depth (z) direction. A known method may be used for object detection, and for example, region identification using an object detection algorithm by deep learning is applicable. An example of an object detection algorithm by deep learning is R-CNN (Regions with Convolutional Neural Networks).

The above-described three-dimensional point cloud generation unit 5124 may generate the three-dimensional point cloud data based on the depth map of the portion corresponding to the image area extracted by the object detection unit 5624. Thereby, three-dimensional point cloud data with less noise can be generated, and as a result, the accuracy of the shape data extracted by the shape data extraction unit 5524 can be improved.

The conversion unit 5024C converts the shape data extracted by the shape data extraction unit 5524 to generate a silhouette image of the object. The converted silhouette image is not simply represented by black and white binarized data, but based on the depth data, the image area of the object has, for example, a luminance value of 0 (“black”) to 1 (“white”). It is possible to obtain a monochromatic multi-tone monochrome image (gradation silhouette image) represented by the data up to ). That is, the silhouette image data can have a larger amount of information by associating the image area of the object with the depth data.

(5-2) Operation of Silhouette Image Generating Device FIG. 25 is a flowchart for explaining the operation of the silhouette image generating device 5020 according to another embodiment described in FIG. Through this flow chart, a silhouette image of the object can be generated from the image data of the image of the object (S5000).

First, the acquisition unit 5024A acquires image data including a depth map in which an object is photographed (S5010). Next, the object detection unit 5624 extracts the image area of the object from the RGB image data included in the image data (S5020). In addition, the said step may be arbitrary. Subsequently, the 3D point cloud generation unit 5124 generates 3D point cloud data corresponding to the depth map included in the image data to configure a virtual 3D coordinate space (S5030). When S5020 is executed, it is preferable to generate the three-dimensional point cloud data based on the depth map of the portion corresponding to the image area (xy coordinate area) of the object.

Next, the background point cloud removing unit 5224 removes the 3D point cloud data that is away from the predetermined threshold distance along the depth (z) direction of the virtual 3D coordinate space (S5040). Further, the plane point cloud removing unit 5324 estimates the plane part in the image data (S5050), and further removes the three-dimensional point cloud data corresponding to the plane part from the three-dimensional point cloud data (S5060). Note that S5050 and S5060 may be repeated to estimate a plurality of plane portions in the image data and remove the three-dimensional point cloud data.

Next, the target area extraction unit 5424 extracts the target area of the target based on the removed three-dimensional point cloud data (S5070). Then, the shape data extraction unit 5524 extracts the shape data indicating the shape of the target object based on the depth data of the target object region in the depth map (S5080). Finally, the conversion unit 5024C converts the shape data to generate a silhouette image of the object.

(5-3) Features of silhouette image generation device (5-3-1)
As described above, the silhouette image generation device 5020 according to this embodiment includes the acquisition unit 5024A, the extraction unit 5024B, and the conversion unit 5024C. The acquisition unit 5024A acquires image data including a depth map in which an object is photographed. The extraction unit 5024B extracts the object region of the object using the three-dimensional point cloud data generated from the depth map, and indicates the shape of the object based on the depth data of the depth map corresponding to the object region. Extract shape data. The conversion unit 5024C converts the shape data to generate a silhouette image of the object.

Therefore, the silhouette image generation device 5020 generates three-dimensional point cloud data using the depth map and then generates a silhouette image of the object. Since it is possible to effectively identify and remove the three-dimensional point cloud that becomes noise in the three-dimensional point cloud data, it is possible to improve the identification accuracy of the object region of the object. Thereby, a highly accurate silhouette image can be obtained. Further, by using the depth map, a gradation silhouette image, which is a monochrome image associated with the data, can be generated as the silhouette image, and a large amount of information can be provided regarding the shape of the object.

(5-3-2)
In addition, in the silhouette image generation device 5020, the extraction unit 5024B removes the 3D point cloud data existing apart from the predetermined threshold distance along the depth direction from the 3D point cloud data based on the target. The object area of the object is extracted. As a result, it is possible to effectively remove the three-dimensional point group that constitutes the background and becomes noise in the screen data, and thus improve the accuracy of specifying the object area of the object extracted by the object area extraction unit 5424. be able to.

(5-3-3)
Further, in the silhouette image generation device 5020, the extraction unit 5024B further estimates a plane portion in the image data from the three-dimensional point cloud data generated from the depth map, and estimates the estimated plane of the three-dimensional point cloud data. The object area of the object is extracted based on the data obtained by removing the three-dimensional point cloud data existing in the part.

(5-3-4)
Here, the extraction unit 5024B estimates the plane portion based on calculating the content rate of the three-dimensional point cloud data associated with the sample plane sampled according to the random sampling. Then, the plane portion is estimated by repeating the estimation. As a result, the sampling plane is extracted a plurality of times and then the sampling plane having the highest content rate of the three-dimensional point cloud data is determined, so that the robustness with respect to the estimation of the desired plane portion can be improved.

Also, the extraction unit 5024B estimates a plurality of plane portions by repeating the process of estimating the plane portions. As a result, it is possible to effectively remove the three-dimensional point group that constitutes a plane and becomes noise in the screen data, and thus improves the accuracy of specifying the target object area of the target object extracted by the target object area extracting unit 5424. be able to.

(5-3-5)
Further, in the silhouette image generation device 5020, the acquisition unit 5024A further acquires RGB image data, and the extraction unit 5024B further extracts the image area of the target object using the RGB image data, and a part corresponding to the image area is extracted. Three-dimensional point cloud data is generated from the depth map. By extracting the image area of the object in advance before the generation of the three-dimensional point cloud data, it is possible to further improve the identification accuracy of the object area of the object extracted by the object area extraction unit 5424. it can.

(5-3-6)
Furthermore, in the present embodiment, it is preferable that the object is a person and the plane portion includes the floor. This effectively creates a silhouette of a person standing upright on the floor.

(6-1) Configuration of Dimension Data Calculation Device FIG. 26 is a schematic diagram showing the configuration of the dimension data calculation device 6020 according to another embodiment. The dimension data calculation device 6020 includes a shape parameter acquisition unit 6024D and a calculation unit 6024E.

The dimension data calculation device 6020 according to the present embodiment can be applied to the third and fourth embodiments and their modifications. For example, the shape parameter acquisition unit 6024D included in the dimension data calculation device 6020 according to the present embodiment is configured as all or part of the acquisition unit 3024A, the extraction unit 3024B, the conversion unit 3024C, and the estimation unit 3024D of the third embodiment. Good. Alternatively, it may be configured as all or part of the acquisition unit 4124A and the estimation unit 4124D of the fourth embodiment. Further, the calculation unit 6024E included in the dimension data calculation device 6020 according to the present embodiment is configured as all or part of the calculation unit 3024E in the third embodiment, or as all or part of the calculation unit 4124E in the fourth embodiment. Good.

The shape parameter acquisition unit 6024D acquires the value of the shape parameter of the object.
The calculation unit 6024E configures the three-dimensional data of the target object from the value of the shape parameter of the target object, and based on the information of the vertices of the three-dimensional data that configures the predetermined part region associated with the predetermined part Calculate the dimensional data of the part. The dimensional data can be related to any part. In order to calculate the dimensional data, a calculation algorithm can be set according to each part.

The calculation unit 6024E includes a three-dimensional data configuration unit 6124, a part region configuration unit 6224, a calculation point extraction unit 6324, and a dimension data calculation unit 6424.

The three-dimensional data configuration unit 6124 of the calculation unit 6024E configures the three-dimensional data of the target object from the acquired shape parameter values. As described in the third embodiment and the fourth embodiment, by performing the inverse projection transformation process regarding the dimension reduction performed in the learning stage on the value of the shape parameter acquired by the shape parameter acquisition unit 6024D, 3 Dimensional data is constructed. The three-dimensional data to be constructed is preferably three-dimensional mesh data, and is information on a set of vertices of meshes forming a three-dimensional object (for example, three-dimensional coordinates of vertices).

In the following examples, it is assumed that, for example, the dimension measurement target is a human body and the configured three-dimensional object is a human body model, although not limited thereto. 27a and 27b are schematic views of a human body model in a three-dimensional space when the object is a human body. The human body model is composed of three-dimensional mesh data (mesh is not shown). The human body model is preferably a model that stands upright on a horizontal plane, for example.

27a is a plan view of the human body model, and FIG. 27b is a front view thereof. It is assumed that the three-dimensional coordinate system is adjusted with respect to the human body model such that the side direction is the x-axis, the front direction is the y-axis, and the height direction is the z-axis in the three-dimensional space. More specifically, the positive direction of the x-axis is the direction of the left half of the body from the center of gravity of the body when the human body model is viewed from the front, and the negative direction is the direction of the right half of the body. Regarding the y-axis, the positive direction is from the body center of gravity to the back direction, and the negative direction is from the body center of gravity to the front direction. The positive direction of the z axis is the upper body direction (or the vertical upward direction) from the body center of gravity in the height direction, and the negative direction is the lower body direction (or the vertical downward direction).

Referring back to FIG. 26, the part region forming unit 6224 of the calculating unit 6024E forms a predetermined part region associated with a predetermined part from the information on the vertices of the three-dimensional mesh data formed by the three-dimensional data forming unit 6124. .. For example, the part region may be a tubular region having a center of gravity axis. The tubular region is composed of a set of three-dimensional mesh data vertices that partially form a three-dimensional object within a predetermined range according to a predetermined region. Further, the configuration of the part region may include classifying (clustering) the distribution of the set of vertices under a predetermined condition.

More specifically, the body regions that accommodate these parts are associated with the parts “hip” and “waist”, and similarly, the arm region is associated with the part “wrist” and the shoulder region is associated with the part “armhole”. Be done. The part region is not limited to the cylindrical region, and may be a region in any three-dimensional space. Further, the part area may be a flat area as well as a three-dimensional area.

The calculation point extraction unit 6324 extracts a predetermined number of calculation points from the set of vertices of the three-dimensional mesh data that constitutes the part area. More specifically, the calculation point extraction unit 6324 selectively extracts calculation points partially associated with the part region from the set of vertices of the three-dimensional mesh data according to a predetermined part. For example, the tubular region is divided with respect to each quadrant of a (two-dimensional) coordinate system that is defined orthogonally to the centroid axis of the tubular region and has the centroid axis as the origin, and the calculation points are selectively extracted individually from each quadrant. To do.

The number of calculation points extracted by the calculation point extraction unit 6324 is preferably 3 to 5 from the viewpoint of calculation amount and accuracy when the region area is a cylindrical area. Based on the inventor's deep knowledge, when 6 or more calculation points are extracted, the calculation amount may increase and the calculation efficiency may decrease, while if at least 3 calculation points can be extracted, the dimension data can be calculated with high accuracy. It turns out that you can.

Also, it is recommended that the mode of dividing the region and the number of divisions be set individually according to the region. Furthermore, in addition to extracting the calculation points associated with the divided part regions, the vertices of the three-dimensional mesh data that satisfy a predetermined condition may be additionally extracted as the calculation points.

The dimension data calculation unit 6424 concretely calculates the dimension data based on the calculation points extracted by the calculation point extraction unit 6324. For example, when the region area is a tubular area, the dimension data is calculated by calculating the length of the circumference along the circumference of the tubular area based on the information on the extracted calculation points. More specifically, the circumference length is calculated by calculating the total sum of the distances so that the calculation points adjacent to each other along the circumference of the tubular region are connected by lines. As described above, the dimension data can be calculated efficiently and highly accurately by calculating the length of the circumference of the cylindrical region using the information on the extracted calculation points.

(6-2) Operation of Dimension Data Calculation Device FIG. 28 is a flowchart for explaining the operation of the size data calculation device 6020 described with reference to FIG. Through the process (S6000) according to this flowchart, it is possible to calculate the dimension data of a predetermined part of the object from the shape parameter values.

First, the shape parameter acquisition unit 6024D acquires the value of the shape parameter of the object (S6010). In addition, the said step is good to be implemented through the estimation process in the estimation part 3024D in 3rd Embodiment and the estimation part 4124D in 4th Embodiment.

Next, the three-dimensional data forming unit 6124 of the calculating unit 6024E forms three-dimensional mesh data from the shape parameter values of the target object (S6020). The configured three-dimensional mesh data is information (for example, three-dimensional coordinates of the vertices) of a set of vertices of meshes that form a three-dimensional object.

Subsequently, the part region construction unit 6224 forms a predetermined part region that is associated in advance with a predetermined part from the information on the set of vertices of the formed three-dimensional mesh data (S6030).

Then, the calculation point extraction unit 6324 selectively extracts a plurality of characteristic calculation points associated with the part area (S6040). More specifically, the part region is divided according to a predetermined part, and about 3 to 5 calculation points are individually selected from the divided part regions (a calculation point extraction example regarding a specific part will be described later).

Finally, the dimension data calculation unit 6424 calculates the dimension data based on the extracted calculation points (S6050). For example, when the region area is a tubular area, the dimension data is calculated by calculating the circumference length of the tubular area based on the extracted calculation points. More specifically, the three-dimensional circumference length is calculated by connecting the calculation points adjacent to each other along the circumference of the tubular region with a line and calculating the sum of the distances.

As described above, the characteristic calculation points are extracted from the model of the object, and by using the calculated calculation points, the circumference length of the cylindrical region is calculated, thereby efficiently and highly accurately calculating the dimension data. be able to.

(6-3) Example of Extraction of Calculation Points for Each Part With reference to FIGS. 29 to 35b, regarding the calculation point extraction unit 6324 included in the calculation unit 6024E of the dimension data calculation device 6020, a plurality of calculation points for a specific part An example of selectively extracting will be described. Here, examples of the parts “hip”, “waist”, “wrist circumference”, and “armhole” will be described, but the extraction of calculation points according to the present embodiment is not limited to these parts.

29 to 31 are schematic diagrams showing an example in which the range of the body region and the arm region, which are cylindrical regions, are specified with respect to the human body model and each region is configured. 32a and 32b are schematic diagrams showing an example of extracting calculation points regarding the hips. Similarly, FIGS. 33a and 33b are schematic diagrams showing an example of extracting calculation points regarding a waist, FIGS. 34a and 34b around a wrist, and FIGS. 35a and 35b.

(6-3-1) Identification of Body Region and Arm Region As shown in the front view of FIG. 29, the body region BR and arm region AR of the human body model are extracted over a predetermined range in the height direction (z-axis direction). It is specified by cutting out the three-dimensional area. For example, although not limited to this, in a human body model, a three-dimensional mesh is obtained by cutting out a region of a predetermined distance (±Dcm) vertically from a position of a predetermined ratio of the height (R% from the top) with respect to the z axis.・It is better to extract a set of data vertices. The region to be cut out includes the right arm region AR _r , the left arm region AR ₁ , and the body region BR. It should be noted that the values of the ratio R and the distance D are preferably selected individually depending on the part to be calculated.

FIG. 30 shows an xy plane in which the right arm region AR _r , the left arm region AR ₁ , and the body region BR are viewed from the z-axis positive direction. C(c _x , c _y ) is the center point of the human body model and is calculated from each coordinate value of the set of vertices of the three-dimensional mesh data. For example, the center point C(c _x , c _y ) should be the center of gravity of the body.

FIG. 31 is a diagram showing the distribution of the vertices of the three-dimensional mesh data regarding the right arm region AR _r , the left arm region AR ₁ , and the body region BR shown in FIGS. 29 and 30. The horizontal axis of the coordinate system in FIG. 31 indicates the distance in the x-axis direction from the center point C(c _x , c _y ) to each vertex. The vertical axis indicates the distance from the center point C(c _x , c _y ) to each vertex.

That is, the values on the horizontal axis and the vertical axis of the graph of FIG. 31 are calculated by the following mathematical expressions.
・Value on the horizontal axis = |x−c _x |
Value of vertical axis=(|x−c _x | ² +|y−c _y | ² ) ^1/2

In the distribution shown in FIG. 31, the area ar is an area corresponding to the right arm area AR _r and the left arm area AR ₁ . The region br is a region corresponding to the body region BR. That is, a set of vertices of a three-dimensional area extracted over a predetermined range in the height direction (z-axis direction) can be classified (clustered) into areas ar and br. Thereby, it is possible to determine whether each vertex of the three-dimensional region belongs to any of the body region BR and the arm region AR (here, the right arm region AR _r and the left arm region AR _l ).

As will be described below, the calculation points in the parts “hip”, “waist”, “wrist”, and “arm hole” are information on the vertices of the three-dimensional mesh data of these body regions BR or arm regions AR. Can be extracted based on.

(6-3-2) Hip FIG. 32a shows five calculation points for calculating the hip size data from the set of vertices of the three-dimensional mesh data forming the tubular body region BR shown in FIG. It is a schematic plan view showing an example of extracting the. FIG. 32b is a schematic three-dimensional view thereof.

32a and 32b, x'y has a plane x'y' which has a center of gravity axis AX1 of the body region BR1 along the height direction as the z'axis, and is orthogonal to the z'axis and has the center of gravity axis AX1 as the origin. A'z' coordinate system is defined. Particularly, regarding the x′y′ plane coordinate system viewed from the direction of the center of gravity axis AX1, the x′ direction is the long axis direction of the cross section of the body region BR1 having a substantially elliptical shape (that is, the side direction of the human body model), and y The'direction is the minor axis direction of the same section (that is, the front (back) direction of the human body model).

As shown in FIGS. 32a and 32b, five calculation points (a1, b1, c1, d1, e1) should be extracted in order to calculate hip size data. Of these, four (a1, b1, c1, d1) are individually extracted from the set of vertices of the three-dimensional mesh data existing in the quadrant of the x'y' plane when viewed from the direction of the center of gravity axis AX1. A total of four calculation points may be extracted, one from each quadrant, or a total of three calculation points may be individually extracted from any three quadrants. In the following, it is assumed that a total of four calculation points are extracted.

Specifically, a restriction region LR1 having a long axis x'value of -k ₁ ≤x' ≤k ₁ is provided, and four vertices of three-dimensional mesh data located in the restriction region LR1 are provided in each quadrant. It is preferable to individually extract the calculation points a1 to d1. The value of k ₁ is preferably about ¼ of the length of the cross-section of the body region BR1 in the long axis direction.

More specifically, in each quadrant, the four vertices that are farthest from the origin (the center of gravity axis AX1) in the x′-axis direction, that is, the four vertices that exist close to the boundary within the restricted region LR1 are individually extracted as calculation points. Is good. That is, in the first quadrant and the fourth quadrant, it is preferable to extract the vertices a1 and d1 having the maximum x′ value in the restricted region LR1 as calculation points, and in the second quadrant and the third quadrant, It is preferable to extract the vertices having the smallest x′ value in the restricted region LR1 as the calculation points b1 and c1.

In addition, especially the hip is usually a shape that projects in the front and back directions in the human body model. Therefore, in addition to the above-mentioned four calculation points (a1 to d1), as the remaining one calculation point e1, it is preferable to additionally extract a vertex located at a portion corresponding to the protrusion in the front direction. Specifically, the calculation point e1 is preferably extracted as the vertex that is farthest from the origin in the y'axis (short axis) direction (here, the y'value is the minimum).

In this way, regarding the hip, by extracting the calculation points according to the hip shape in the human body model, it is possible to further improve the accuracy of dimension measurement. It should be noted that such extraction of the additional calculation point e1 is optional and does not necessarily have to be extracted.

(6-3-3) Waist FIG. 33a shows five calculation points for calculating waist size data from the set of vertices of the three-dimensional mesh data forming the tubular body region BR shown in FIG. It is a schematic plan view showing an example of extracting the. Further, FIG. 33b is a schematic stereoscopic view thereof.

The x'y'z' coordinate system defined in FIGS. 33a and 33b is defined similarly to FIGS. 32a and 32b. Here, the center of gravity axis AX2 of the body region BR2 is the origin of the x'y' plane.

As shown in FIGS. 33a and 33b, five calculation points (a2, b2, c2, d2, e2) should be extracted in order to calculate waist size data. Four of them (a2, b2, c2, d2) are individually extracted from a set of vertices of the three-dimensional mesh data existing in the quadrant of the x'y' plane when viewed from the direction of the center of gravity axis AX2. A total of four calculation points may be extracted from each quadrant, or a total of three calculation points may be extracted from arbitrary three quadrants. Below, the example which extracts a total of four calculation points is assumed.

Particularly, regarding the waist, it is preferable to first select the reference vertex std as the reference point and individually extract the vertices associated with the reference point std as the four calculation points a2, b2, c2, d2. In general, it is known that the waist has a shape in which the front direction (here, the negative direction of the y′ axis) projects in the human body model, and the set of vertices of the three-dimensional mesh data is biased toward the projecting portion. In this regard, also in FIGS. 33a and 33b, the center of the substantially sectional shape of the body region BR2 is shifted from the origin, which is the center of gravity axis AX2, in the positive direction of the y′ axis.

Therefore, regarding the calculation of dimensional data, the calculation points should be extracted in consideration of such a protruding portion of the waist. The above-mentioned reference point std specifies the apex located at the location corresponding to such a protrusion. The reference point std is extracted as a vertex that is farthest from the origin (the center of gravity axis AX2) in the frontal direction (here, the negative direction of the y'axis) in the human body model (the value of the y'axis is the minimum). Good. However, in the waist example, the reference point std itself does not necessarily have to be adopted as the calculation point.

After the reference point std is extracted, it is preferable to individually extract the vertices close to the position of the reference point std in the z′-axis (centroid axis AX2) direction as calculation points a2, b2, c2, d2 in each quadrant. .. That is, the calculation points a2, b2, c2, and d2 are vertices whose z′-axis values are close to the z′ value of the reference point std and have approximately the same height. In a region such as a waist whose dimension is to be measured along a horizontal plane, it is preferable to extract a vertex having a height close to the height direction, which can further improve the dimension accuracy.

In the example of West further preferably provided a limited area LR2 which the value _{_{-k 2 ≦ x '≦ k 2}} ' x is a major axis in the coordinate system 'x'y. Then, as the remaining one calculation point e2, an arbitrary vertex on the opposite side (here, the positive direction of the y'axis) from the reference point std with respect to the origin along the y'axis which is the short axis is additionally added. It is good to extract. The value of k ₂ may be, for example, about ¼ of the length of the long axis of the cross section of the body region BR2.

In this way, in addition to the four calculation points a2, b2, c2, d2, the fifth calculation point e2 is additionally extracted, as described above, especially in the case of the waist, the center of gravity is in the front direction. This is because the bias is taken into consideration. That is, the set of vertices of the three-dimensional mesh data forming the body region BR2 is distributed from the origin (center of gravity axis AX2) to the −y′ region. In consideration of such a situation, by additionally extracting the vertex on the opposite side of the reference point std as the fifth calculation point e2, not only a dimensional error does not occur but the dimensional accuracy is further improved. I am trying to improve it.

(6-3-4) Wrist circumference FIG. 34a calculates wrist size data from a set of vertices of the three-dimensional mesh data forming the cylindrical arm region AR _r or AR _l shown in FIG. FIG. 6 is a schematic plan view showing an example of extracting four calculation points for this purpose. Further, FIG. 34b is a schematic stereoscopic view thereof. Note that, in the arm region, either AR _r or AR _l may be a target for dimension measurement (hereinafter, generically referred to as arm region AR).

In FIGS. 34a and 34b, x'y' has an x'y' plane which has the center of gravity axis AX3 of the arm region AR along the height direction as the z'axis, and is orthogonal to the z'axis and has the center of gravity axis AX3 as the origin. A'z' coordinate system is defined. Particularly, regarding the x′y′ plane coordinate system, the x′ direction is the major axis direction of the cross section of the arm region AR having a substantially elliptical shape (that is, the side direction of the human body model), and the y′ direction is the same cross section. The short axis direction (that is, the front direction of the human body model).

As shown in FIGS. 34a and 34b, four calculation points (a3, b3, c3, d3) should be extracted in order to calculate the dimension data around the wrist. The region around the wrist can be assumed to be the most recessed part of the arm region.

That is, in the case of the region around the wrist, first, the arm region AR is scanned in the direction of the center of gravity to extract the cross-sectional region cs having the minimum length in the major axis direction and/or the minimum length in the minor axis direction. Therefore, it is better to identify the recessed part. Here, it may be based on only the length in the major axis direction, based on the length only in the minor axis direction, or both. The cross-sectional area cs may be defined as a plane area or may be a three-dimensional area having a thickness in the z′-axis direction. The cross-sectional area cs may be defined as the back of the hand.

Next, the arm region AR is scanned in the direction of the center of gravity axis, and in the set of vertices of the three-dimensional mesh data, the two vertices most distant in the long axis direction and the two vertices most distant in the short axis direction. Four points are extracted as calculation points (a3, b3, c3, d3). More specifically, in the cross-sectional area cs, at the position farthest in the positive direction of the x'axis (here, the x'value is the maximum), the vertex closest to the z'axis is the calculation point a3, and The vertex closest to the x'-axis in the negative direction in the negative direction (here, the x'value is the minimum) and closest to the z'-axis is extracted as the calculation point c3. Similarly, in the cross-sectional area cs, the closest vertex in the z'-axis direction at the position most distant in the positive direction of the y'-axis (here, the y'-value becomes maximum) is the calculation point b3, and the y'-axis. The closest vertex in the z'direction at the position farthest away in the negative direction (here, the y'value is the minimum) is extracted as the calculation point d3.

In this way, the arm region is scanned in the direction of the center of gravity, and it is estimated that the region portion in which the length in the major axis direction and/or the length in the minor axis direction is the minimum is the relevant part, and four calculation points are calculated. Extract as (a3, b3, c3, d3). As a result, it is possible to efficiently calculate the dimension data around the wrist. In particular, since the length around the wrist is shorter than that at the hip or waist, it is sufficient to extract four calculation points by such a method, and the fifth point is not necessarily required.

(6-3-5) Armhole FIG. 35a shows an armhole from a set of vertices of the three-dimensional mesh data forming the tubular shoulder region SR defined based on the tubular body region BR shown in FIG. 5 is a schematic plan view showing an example of extracting four calculation points in order to calculate the dimension data of FIG. Further, FIG. 35b is a schematic stereoscopic view thereof.

In the example of the armhole, of the set of vertices of the three-dimensional mesh data that configures the body region, in the x'y'z' coordinate system (not shown) defined in FIG. Extract the x'values of distant vertices. Further, of the entire set of vertices of the three-dimensional mesh data, the set of vertices having the x'value is extracted.

In the set of vertices having this x'value, the furthest (highest) vertex in the positive direction of the z'axis (center of gravity axis) is defined as the shoulder vertex, and x'y'z' is based on the shoulder vertex. Cut out in a predetermined range in the direction. Then, the cylindrical shoulder region SR is configured by using a set of vertices of the three-dimensional mesh data existing within the cut out range.

In FIGS. 35a and 35b, an x'' having a y''z'' plane which has the center of gravity axis AX4 of the shoulder region SR as the x'' axis and which is orthogonal to the x'' axis and has the center of gravity axis AX4 as the origin. A y″z″ coordinate system is defined. Particularly, regarding the y″z″ plane coordinate system viewed from the direction of the center of gravity axis AX4, the y″ direction is the minor axis direction of the cross section of the shoulder region SR having a substantially elliptical shape (that is, the front surface (wavefront) of the human body model). Direction), and the z″ direction is the major axis direction of the same cross section (that is, the height direction of the human body model).

As shown in FIGS. 35a and 35b, four calculation points (a4, b4, c4, e4) should be extracted in order to calculate the dimension data of the armhole. Three of them (a4, b4, c4) are the vertices most distant from the center of gravity axis AX4 in the positive and negative directions of the short axis and the positive direction of the long axis (z' axis) in the y''z'' plane. That is, it is preferable to set the vertex that is most distant in the vertical direction).

More specifically, the vertex farthest from the center of gravity axis AX4 in the positive direction of the short axis (y″ axis) (here, the maximum y″ value) is extracted as the calculation point a4, and similarly, the center of gravity axis is extracted. It is preferable to extract the vertex farthest away from AX4 in the negative direction of the short axis (here, the y″ value is the minimum) as the calculation point c4. Further, it is preferable to extract the vertex (here, the z'value is the maximum) farthest from the center of gravity axis AX4 in the positive direction of the long axis (z' axis) (that is, the vertically upward direction) as the calculation point b4.

For the remaining one calculation point e4, a restricted region LR4 is set in the negative direction of the z″ axis (that is, vertically downward direction) so that the z″ value of the long axis falls within a predetermined range (z″≦−j). It is preferable to extract vertices close to the boundary in the restricted region LR4 (here, the z″ value becomes maximum in the restricted region LR4).

Here, the restricted area LR4 is defined according to the side area of the human body model. For example, although not limited to this, it is preferable that the side region is divided into a vertically lower quarter and the limited region LR4 is defined. The side region (not shown) is cut out in a predetermined range in the x"y"z" direction with the side apex as a reference, and the apex of the three-dimensional mesh data existing in the cut out range is extracted. It is better to use a set. The side vertices are preferably defined as vertices having a height (z value) such that they cannot be distinguished from each other in the classification of the body region BR and the arm region AR as shown in FIG. Supplementally, in the shoulder region SR, the armpit region has a convex shape upward, so when scanning in the x″-axis direction, the apex of the armpit is classified into the body region BR and the arm region AR when both are classified. It corresponds to the height where it cannot be distinguished.

In this way, in the armhole, four calculation points are specified using the shoulder area SR specified based on the body area. Particularly, by setting the restricted area LR4 based on the side area, the dimension data can be calculated efficiently and highly accurately.

(6-3-6) Others Regarding the human body model, we have explained the example of extracting calculation points for each part of the hip, waist, wrist, and armholes. By applying these, calculation points for other parts can be calculated. Can also be extracted. For example, it is possible to extract three to five calculation points applicable to the tubular regions such as “chest measurement” and “two arms”. As a result, the dimension data of the predetermined region can be calculated efficiently and highly accurately on the condition that the method of extracting the calculation points is previously defined for the cylindrical region associated with the predetermined region. it can.

Also, the extraction of the calculation points for calculating the length of the circumference in the cylindrical region has been described, but the present invention is not limited to this, and can be applied to the calculation of dimension data other than the cylindrical region. For example, based on the information of the apex of the back of the hand specified during the extraction of the calculation points around the wrist, and the apex of the shoulder specified during the extraction of the calculation points of the armhole, the length of the part "sleeve length" Can be calculated. Further, the length of the part “shoulder width” can be calculated based on the information on the vertices of the left and right shoulders.

(6-4) Features of the calculation unit of the dimension data calculation device (6-4-1)
As described above, the dimension data calculation device 6020 according to this embodiment includes the shape parameter acquisition unit 6024D and the calculation unit 6024E. The shape parameter acquisition unit 6024D acquires the value of the shape parameter of the object. The calculation unit 6024E configures the three-dimensional mesh data of the target from the value of the shape parameter of the target, and based on the vertex information of the three-dimensional mesh data that configures the predetermined body region associated with the predetermined body. Then, the dimensional data of the predetermined part is calculated.

According to the dimension data calculation device 6020, the dimension data of the predetermined portion is calculated based on the information of the vertices of the three-dimensional mesh data forming the portion region, thereby calculating the dimension data regarding the measurement target portion. The accuracy can be improved.

(6-4-2)
Further, in the dimension data calculation device 6020, the calculation unit 6024E selectively extracts the calculation points partially associated with the region of the region from the set of vertices of the three-dimensional mesh data according to a predetermined region. Dimension data is calculated based on the calculation points. As a result, the calculation of the dimension data can be made efficient by effectively performing the selective extraction of the calculation points.

(6-4-3)
Further, in the dimension data calculation device 6020, the region area is a cylindrical area, and the dimension data is calculated by calculating the circumference length of the cylindrical area based on the calculation points. Thereby, the calculation of the cylindrical dimension data can be performed with high accuracy and efficiency.

(6-4-4)
In addition, the length of the circumference is calculated so as to be the sum of the distances calculated by connecting the adjacent calculation points with lines along the circumference of the tubular region. As a result, the calculation of the cylindrical dimension data can be performed with higher accuracy and efficiency.

(6-4-5)
In addition, the dimension data calculation method according to the embodiment according to the present embodiment obtains the three-dimensional mesh data of the object from the step of acquiring the value of the shape parameter of the object (S6010) and the value of the shape parameter of the object. Configuring step (S6020), configuring a predetermined part region previously associated with a predetermined part from the information on the set of vertices of the formed three-dimensional mesh data (S6030), associating part part region The method further includes a step of selectively extracting the calculated calculation points (S6040) and a step of calculating dimensional data based on the extracted calculation points (S6050).

According to such a dimension data calculation method, by calculating the dimension data of a predetermined region based on the information of the vertices of the three-dimensional mesh data forming the region of the region, the accuracy of the calculation of the dimension data regarding the region to be measured is calculated. Can be improved.

<Other Embodiment 3: Terminal Device>
FIG. 36 is a schematic diagram showing the configuration of a terminal device 7020 according to another embodiment.
The terminal device 7020 has the functions of the terminal device 1010 according to the first embodiment, the terminal device 2010S according to the second embodiment, the terminal device 3010 according to the third embodiment, or the terminal device 4010S according to the fourth embodiment. It is a thing. Further, it can be connected to each of the above-mentioned dimension

data calculation devices

1020, 2120, 3020, 4120, 6020. Furthermore, the terminal device 7020 is not limited to the dimension data calculation device, and can be connected to any information processing device that processes information about the object 7007 from image data of the object 7007 captured.

The terminal device 7020 includes an acquisition unit 7011, a communication unit 7012, a processing unit 7013, and an input/output unit 7014.

The acquisition unit 7011 acquires image data of the object 7007 captured. For example, the acquisition unit 7011 is composed of an arbitrary monocular camera. The data acquired by the acquisition unit 7011 is processed by the processing unit 7013.

The communication unit 7012 is realized by a network interface such as an arbitrary network card and enables communication with a communication device on the network by wire or wirelessly.

The processing unit 7013 is realized by a processor such as a CPU (Central Processing Unit) and/or a GPU (Graphical Processing Unit) and a memory in order to execute various types of information processing, and executes processing by reading a program. Here, the processing unit (determination unit) 7013 determines whether the target object included in the image data (that is, reflected in the image data) is a target object registered in advance. Specifically, the processing unit 7013 uses the “target identification model” for identifying whether each pixel is a predetermined target or not, and determines whether the target included in the image data is a pre-registered target. Determine whether or not.

The input/output unit 7014 receives input of various information to the terminal device 7020, and outputs various information from the terminal device 7020. For example, the input/output unit 7014 is realized by an arbitrary touch panel. The input/output unit (reception unit) 7014 shows the determination result by the processing unit 7013 described later on the screen (output unit) of the input/output unit 7014. Further, the input/output unit 7014 superimposes the determination image data obtained from the identification result for each pixel identified using the object identification model on the image data acquired by the acquisition unit 7011, and the input/output unit 7014 of the input/output unit 7014. Shown on the screen.

For example, as shown in FIG. 37, the object 7007 (here, a person) is displayed by being superimposed on the image data (hatched portion in FIG. 37) of the area BG other than the object. For the superimposition, a method of superimposing and drawing a transparent image such as alpha blending can be used. Further, the input/output unit (reception unit) 7014 receives an input as to whether or not to transmit the image data to the information processing device (for example, the dimension

data calculation device

1020, 2120, 3020, 4120, 6020). As a result, the user can visually check the superimposed image and confirm that the object that induces the misrecognition is not present, and then transmit the image data to the dimension data calculation device.

FIG. 38 is a sequence diagram between the terminal device 7020 and the information processing device (for example, the dimension

data calculation devices

1020, 2120, 3020, 4120, 6020) for explaining the operation of the terminal device 7020.

First, the image data of the object 7007 is acquired through the terminal device 7020 by the operation of the user (V1). Next, the terminal device 7020 determines whether or not the target object 7007 included in the image data is a target object registered in advance by using the target object identification model, and displays it on the screen configuring the input/output unit 7014. The judgment result is output (V2). For example, a screen as shown in FIG. 37 is displayed as the determination result.

Next, whether or not to transmit the acquired image data to the dimension data calculation device is input via the terminal device 7020 by a user operation (V3). When the terminal device 7020 receives the input of the transmission permission from the input/output unit 7014, the terminal device 7020 transmits the acquired image data to the dimension data calculation device through the communication unit 7012 (V3-Yes, V4).

Then, the dimension data calculation device that has received the image data calculates the dimension data of the object 7007 using the image data transmitted from the terminal device 7020 (V5, V6).

As described above, the terminal device 7020 outputs the determination result as to whether or not the object included in the image data is a previously registered object, and transmits the image data to the dimension data calculation device. Since the input of whether or not is accepted, it is possible to provide a terminal device that can reduce the operation time by the user.

Supplementally, the dimension

data calculation devices

1020, 2120, 3020, 4120, 6020 receive image data and segment the background by segmentation to generate a silhouette image. Then, the dimension data calculation device calculates, for example, the dimension data of each part of the human body. In such a dimension data calculation device, the image data is transmitted from the terminal device 7020 to the dimension data calculation device, and the reliability of the calculation result is confirmed only after the image data information processing by the dimension data calculation device is completed. I can't. If the reliability of the calculation result is low, the user needs to acquire the image data again using the terminal device 7020. On the other hand, when the terminal device 7020 is used, it is possible to prompt the user to confirm the validity of the image data before transmitting the image data in which the object is photographed. In some cases, the time required to obtain a highly accurate calculation result can be shortened.

For example, the user of the terminal device 7020 predicts that the generation of a silhouette image by the dimension data calculation device will be successful even in an environment where various colors are arranged, such as in a store, or an environment in which a mannequin or the like is mistakenly recognized as a human body. Since it can be executed while checking, the operation time may be shortened in some cases.

Note that the object identification model installed in the terminal device 7020 is required to execute segmentation at high speed. Therefore, a model that can infer at high speed is preferable, even if the accuracy of segmentation is sacrificed to some extent. In short, by preparing both the segmentation on the side of the dimension data calculation device and the segmentation on the side of the terminal device 7020, it is possible to achieve both generation of a precise silhouette image and removal of unnecessary objects.

The present disclosure is not limited to the above embodiments as they are. The present disclosure can be embodied by modifying the constituent elements within a range not departing from the gist of the present invention at the implementation stage. Further, the present disclosure can form various disclosures by appropriately combining a plurality of constituent elements disclosed in each of the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiment. Further, the constituent elements may be appropriately combined with different embodiments.

According to the present specification, the configurations of the following aspects are also disclosed.
The dimension data calculation device according to the first aspect includes an acquisition unit, an extraction unit, a conversion unit, and a calculation unit. The acquisition unit acquires image data of a captured object and full-length data of the object. The extraction unit extracts shape data indicating the shape of the object from the image data. The conversion unit converts the shape data based on the full length data. The calculation unit reduces the dimension of the shape data converted by the conversion unit, and uses the reduced value of each dimension and the weighting coefficient optimized for each part of the target to calculate the dimension data of each part of the target. calculate.
The dimension data calculation device according to the second aspect is the dimension data calculation device according to the first aspect, in which the acquisition unit acquires a plurality of image data obtained by photographing the object from different directions.
The dimension data calculation device according to the third aspect is the dimension data calculation device according to the first or second aspect, and the calculation unit performs the first dimension reduction on the shape data converted by the conversion unit. The calculation unit linearly combines the value of each dimension obtained in the first dimension reduction and the weighting coefficient optimized for each part of the object to obtain a predetermined value, or obtains the predetermined value in the first dimension reduction. A quadratic feature amount is generated from the obtained values of each dimension, and the quadratic feature amount and the weighting coefficient optimized for each part of the object are combined to obtain a predetermined value. The calculation unit performs the second dimension reduction using the predetermined value and the attribute data including at least the attributes of the length and the weight of the object. The calculation unit calculates the dimension data of each part of the object based on the value of each dimension obtained in the second dimension reduction.
A dimension data calculation device according to a fourth aspect is the dimension data calculation device according to the first to third aspects, in which the extraction unit performs the semantic segmentation constructed using the teacher data prepared for each type of object. The shape data of the object is extracted by extracting the object area included in the image data using an algorithm.
The dimension data calculation device according to the fifth aspect is the dimension data calculation device according to the fourth aspect, in which the extraction unit extracts the shape data of the target object from the target object region by a grab cut algorithm.
A dimension data calculation device according to a sixth aspect is the dimension data calculation device according to the fifth aspect, in which the extraction unit causes the image of the object extracted by the grab cut algorithm to be based on the color image of the specific portion in the image data. Correction is performed to generate new shape data.
The dimension data calculation device according to the seventh aspect is the dimension data calculation device according to any of the first to sixth aspects, and the object is a person.
The product manufacturing apparatus according to the eighth aspect manufactures a product related to the shape of the target object by using the dimension data calculated by using the dimension data calculation apparatus according to any one of the first to seventh aspects.
The dimension data calculation program of the ninth aspect causes a computer to be realized as an acquisition unit, an extraction unit, a conversion unit, and a calculation unit. The acquisition unit acquires image data of a captured object and full-length data of the object. The extraction unit extracts shape data indicating the shape of the object from the image data. The conversion unit converts the shape data based on the full length data. The calculation unit calculates the dimension data of each part of the object using the shape data converted by the conversion unit.
The dimension data calculation method according to the tenth aspect obtains image data of a photographed object and full length data of the object. Next, shape data indicating the shape of the object is extracted from the image data. Next, the shape data is converted based on the full length data. Then, using the converted shape data, the dimension data of each part of the object is calculated.
A product manufacturing system according to an eleventh aspect includes an acquisition unit, an extraction unit, a conversion unit, a calculation unit, and a product manufacturing device. The acquisition unit acquires the image data of the target together with the full length data of the target from a photographing device that captures a plurality of images of the target. The extraction unit extracts shape data indicating the shape of the object from the image data. The conversion unit converts the shape data based on the full length data. The calculation unit calculates the dimension data of each part of the object using the shape data converted by the conversion unit. The product manufacturing apparatus manufactures a product related to the shape of the object by using the dimension data calculated by the calculation unit.
The information processing device according to the twelfth aspect includes a reception unit and an estimation unit. The reception unit receives the silhouette image of the object. The estimating unit uses the object engine that associates the silhouette image of the sample object and the values of the shape parameters associated with the sample object to obtain the shape parameter value of the object from the received silhouette image. presume. The estimated value of the shape parameter of the object is associated with the dimensional data related to an arbitrary part of the object.
An information processing apparatus according to a thirteenth aspect is the information processing apparatus according to the twelfth aspect, wherein a predetermined number of shape parameters associated with the sample target object are obtained by dimensionally reducing three-dimensional data of the sample target object. Be done.
The information processing apparatus according to the fourteenth aspect is the information processing apparatus according to the thirteenth aspect, in which dimension reduction is performed by principal component analysis. In addition, three-dimensional data of the object is calculated by inversely transforming the projections related to the principal component analysis for the estimated values of the shape parameters. The three-dimensional data is associated with the dimension data.
An information processing apparatus according to a fifteenth aspect is the information processing apparatus according to the fourteenth aspect, in which a predetermined number of main components after the second rank excluding the first rank main components are selected as shape parameters.
An information processing apparatus according to a sixteenth aspect is the information processing apparatus according to the fifteenth aspect, in which the object is a person and the main component of the first rank is associated with the height of the person.
An information processing apparatus according to a seventeenth aspect is the information processing apparatus according to any one of the twelfth aspect to the sixteenth aspect, wherein the silhouette image of the sample object is a three-dimensional object including three-dimensional data of the sample object. It is a projection image of a predetermined direction in.
An information processing apparatus according to an eighteenth aspect is the information processing apparatus according to any one of the twelfth aspect to the seventeenth aspect, wherein the object engine has a predetermined number of objects associated with the silhouette image of the sample object and the sample object. It is generated by learning the relationship with the shape parameter value.
An information processing apparatus according to a nineteenth aspect is the information processing apparatus according to any one of the twelfth aspect to the eighteenth aspect, further including a calculation unit. The calculation unit configures three-dimensional data of a plurality of vertices in the object from the shape parameter values estimated for the object. Then, based on the constructed three-dimensional data, the dimension data between any two of the apexes of the object is calculated.
An information processing apparatus according to a twentieth aspect is the information processing apparatus according to the nineteenth aspect, wherein in the calculation unit, the dimensional data between the two vertices is composed of three-dimensional data of a plurality of vertices in the object. It is calculated along the curved surface on the three-dimensional object.
An information processing apparatus according to a twenty-first aspect is the information processing apparatus according to any one of the twelfth aspect to the twentieth aspect, in which the silhouette image of the object is based on depth data obtained using the depth data measuring device, It is generated by separating the image of the object and the image other than the object.
An information processing apparatus according to a twenty-second aspect is the information processing apparatus according to the twenty-first aspect, wherein the depth data measuring device is a stereo camera.
An information processing method according to a twenty-third aspect is to learn a relationship between a silhouette image of a sample target object and values of a predetermined number of shape parameters associated with the sample target object to generate a target object engine. Next, the silhouette image of the target object is received. Next, the object engine is used to estimate the value of the shape parameter of the object from the received silhouette image. Then, based on the value of the shape parameter of the object, the dimension data related to the part of the object is calculated.
The information processing device according to the twenty-fourth aspect includes a reception unit and an estimation unit. The reception unit receives the attribute data of the target object. The estimation unit uses the object engine that associates the attribute data of the sample object with the values of the predetermined number of shape parameters associated with the sample object to obtain the value of the shape parameter of the object from the received attribute data. presume. The estimated value of the shape parameter of the object is associated with the dimensional data related to an arbitrary part of the object.
An information processing apparatus according to a twenty-fifth aspect is the information processing apparatus according to the twenty-fourth aspect, wherein a predetermined number of shape parameters associated with the sample object are obtained by dimensionally reducing three-dimensional data of the sample object. Be done.
An information processing apparatus according to a twenty-sixth aspect is the information processing apparatus according to the twenty-fifth aspect, in which dimension reduction is performed by principal component analysis. Then, a predetermined number of main components after the second rank, excluding the first rank main components, are selected as the shape parameters.
The information processing apparatus according to the twenty-seventh aspect is the information processing apparatus according to the twenty-sixth aspect, wherein the object is a person. Also, the first-ranked main component is associated with the height of the person. The attribute data includes height data of the object.
An information processing apparatus according to a twenty-eighth aspect is the information processing apparatus according to any one of the twenty-fourth aspect to the twenty-seventh aspect, wherein the object engine is a predetermined number of objects associated with the attribute data of the sample object and the sample object. It is generated by learning the relationship with the shape parameter value.
An information processing apparatus according to a twenty-ninth aspect is the information processing apparatus according to any one of the twenty-fourth aspect to the twenty-eighth aspect, further including a calculation unit. The calculation unit configures three-dimensional data of a plurality of vertices in the object from the shape parameter values estimated for the object. In addition, the calculation unit calculates the dimension data between any two of the apexes of the object based on the configured three-dimensional data.
An information processing apparatus according to a thirtieth aspect is the information processing apparatus according to the twenty-ninth aspect, in which the dimension data between the two vertices is composed of three-dimensional data of a plurality of vertices in the object in the calculation unit. It is calculated along the curved surface on the three-dimensional object.
An information processing method according to a thirty-first aspect is to learn a relationship between attribute data of a sample target object and values of a predetermined number of shape parameters associated with the sample target object to generate a target object engine. Next, the attribute data of the target object is received. Next, the value of the shape parameter of the object is estimated from the received attribute data. Then, based on the value of the shape parameter of the object, the dimension data related to the part of the object is calculated.
The dimension data calculation device according to the thirty-second aspect includes an acquisition unit, an extraction unit, a conversion unit, an estimation unit, and a calculation unit. The acquisition unit acquires image data of a captured object and full-length data of the object. The extraction unit extracts shape data indicating the shape of the object from the image data. The conversion unit converts the shape data into a silhouette image based on the full length data. The estimating unit estimates the value of the predetermined number of shape parameters from the silhouette image by using the object engine that associates the silhouette image of the sample object with the value of the predetermined number of shape parameters associated with the sample object. .. The calculation unit calculates the dimension data of the object based on the estimated values of the predetermined number of shape parameters.
The dimension data calculation device according to the thirty-third aspect is the dimension data calculation device according to the thirty-second aspect, wherein a predetermined number of shape parameters are obtained by dimensionally reducing the three-dimensional data of the sample object.
The dimension data calculation apparatus according to the thirty-fourth aspect is the dimension data calculation apparatus according to the thirty-third aspect, in which dimension reduction is performed by principal component analysis. In addition, three-dimensional data is calculated in the calculation unit by performing inverse transformation on a predetermined number of shape parameter values based on the projection matrix related to principal component analysis. Then, the dimension data is calculated from the three-dimensional data.
A product manufacturing apparatus according to a thirty-fifth aspect manufactures a product related to the shape of an object by using at least one dimension data calculated using the dimension data calculation apparatus according to any one of the thirty-second to thirty-fourth aspects. To do.
The dimension data calculation device according to the thirty sixth aspect includes an acquisition unit and a calculation unit. The acquisition unit acquires attribute data including at least one of full length data and weight data of the object. The calculation unit calculates the dimension data of each part of the object by performing a polynomial regression on the attribute data using the coefficient learned by machine learning.
A dimension data calculation device according to a thirty-seventh aspect is the dimension data calculation device according to the thirty-sixth aspect, in which the calculation unit secondarily regresses the attribute data using a coefficient learned by machine learning, Dimension data of each part of is calculated.
The dimension data calculation device according to the thirty-eighth aspect is the dimension data calculation device according to the thirty-sixth aspect or the thirty-seventh aspect, and the object is a person.
The dimension data calculation program according to the thirty-ninth aspect causes a computer to be realized as an acquisition unit and a calculation unit. The acquisition unit acquires attribute data including at least one of full length data and weight data of the object. The calculation unit calculates the dimension data of each part of the object by performing a polynomial regression on the attribute data using the coefficient learned by machine learning.
A dimension data calculating method according to a fortieth aspect obtains attribute data including at least one of full length data and weight data of an object. Then, the attribute data is subjected to polynomial regression using the coefficient learned by machine learning to calculate the dimension data of each part of the object.
The silhouette image generation device according to the forty-first aspect includes an acquisition unit, an extraction unit, and a conversion unit. The acquisition unit acquires image data including a depth map, in which an object is photographed. The extraction unit extracts a target object area of the target object using the three-dimensional point cloud data generated from the depth map, and a shape indicating the shape of the target object based on the depth data of the depth map corresponding to the target object area. Extract the data. The conversion unit converts the shape data to generate a silhouette image of the object.
A silhouette image generating apparatus according to a forty-second aspect is the silhouette image generating apparatus according to the forty-first aspect, in which the conversion unit is a monochrome image associated with the depth data corresponding to the target region as the silhouette image. Generate an image.
A silhouette image generating apparatus according to a 43rd viewpoint is the silhouette image generating apparatus according to the 41st viewpoint or the 42nd viewpoint, wherein the extraction unit is separated from a predetermined threshold distance along the depth direction in the three-dimensional point cloud data. The object region of the object is extracted based on the data obtained by removing the existing three-dimensional point cloud data.
The silhouette image generating apparatus according to the 44th viewpoint is the silhouette image generating apparatus according to any one of the 41st viewpoint to the 43rd viewpoint, wherein the extraction unit further generates an image from the three-dimensional point cloud data generated from the depth map. Estimate the plane portion of the data. Then, the extraction unit extracts the target object region of the target object based on the three-dimensional point cloud data from which the three-dimensional point cloud data existing in the estimated plane portion is removed.
A silhouette image generating apparatus according to a 45th aspect is the silhouette image generating apparatus according to the 44th aspect, wherein the extraction unit calculates the content rate of the three-dimensional point cloud data associated with the sample plane sampled according to the random sampling. Based on that, the plane part is estimated.
The silhouette image generating apparatus according to the 46th viewpoint is the silhouette image generating apparatus according to any of the 41st viewpoint to the 45th viewpoint, wherein the object is a person and the plane portion includes the floor.
A dimension data calculation device according to a 47th aspect includes an acquisition unit, an extraction unit, and a calculation unit. The acquisition unit acquires image data of a captured object and full-length data of the object. The extraction unit extracts shape data indicating the shape of the object from the image data. The calculation unit calculates the dimension data of each part of the object using the shape data. The image data includes a depth map, and the shape data extracted by the extraction unit is associated with the depth data of the object in the depth map.
A dimension data calculation device according to a forty-eighth aspect is the dimension data calculation device according to the forty-seventh aspect, in which the extraction unit extracts the object region of the object from the image data based on the depth map. Shape data is associated with depth data of the object in the object region.
A dimension data calculation device according to a forty-ninth aspect is the dimension data calculation device according to the forty-seventh aspect or the forty-eighth aspect, further including a conversion unit that converts the shape data based on the full length data.
A dimension data calculation device according to a fiftieth aspect is the dimension data calculation device according to the forty-ninth aspect, in which the conversion unit converts the shape data into new shape data based on the total length data and the depth data of the object region. ..
The dimension data calculation device according to the fifty-first aspect is the dimension data calculation device according to the fifty-ninth aspect or the fiftieth aspect, wherein the calculation unit reduces the dimension of the shape data converted by the conversion unit and reduces the value of each dimension. Using the weighting coefficient optimized for each part of the object, the dimension data of each part of the object is calculated.
The product manufacturing apparatus according to the fifty-second aspect manufactures a product related to the shape of the target object by using the dimension data calculated by using the dimension data calculation apparatus according to any of the 47th to 51st aspects.
The dimension data calculation device according to the fifty-third aspect includes an acquisition unit, an extraction unit, an estimation unit, and a calculation unit. The acquisition unit acquires image data of a captured object and full-length data of the object. The extraction unit extracts shape data indicating the shape of the object from the image data. The estimation unit uses the object engine that associates the silhouette image of the sample object with the value of the predetermined number of shape parameters associated with the sample object, and uses the object engine to extract the value of the predetermined number of shape parameter values from the silhouette image of the object. To estimate. The calculation unit calculates the dimension data of the object based on the estimated values of the predetermined number of shape parameters. The image data includes a depth map, and the shape data is associated with the depth data of the object in the depth map.
The dimension data calculation device according to the fifty-fourth aspect is the dimension data calculation device according to the fifty-third aspect, in which the extraction unit extracts the object region of the object from the image data based on the depth map. Shape data is associated with depth data of the object in the object region.
A dimension data calculation device according to a fifty-fifth aspect is the dimension data calculation device according to the fifty-third aspect or the fifty-fourth aspect, including a conversion unit that converts the shape data into new shape data based on the depth data of the object region. Further prepare.
The dimension data calculation device according to the fifty-sixth aspect is the dimension data calculation device according to any of the fifty-third aspect to the fifty-fifth aspect, wherein the predetermined number of shape parameters dimensionally reduces the three-dimensional data of the sample object. can get.
The product manufacturing apparatus according to the fifty-seventh aspect manufactures a product related to the shape of the target object using the dimension data calculated using the dimension data calculation apparatus according to any one of the fifty-third to the fifty-sixth aspects.
The terminal device according to the fifty-eighth aspect is connected to an information processing device that processes information about an object from image data obtained by capturing the object. The terminal device includes an acquisition unit, a determination unit, and a reception unit. The acquisition unit acquires image data of a target object. The determination unit determines whether or not the target object included in the image data is a target object registered in advance. The acceptance unit indicates the determination result by the determination unit to the output unit, and accepts an input as to whether or not to transmit the image data to the information processing device.
A terminal device according to a fifty-ninth aspect is the terminal device according to the fifty-eighth aspect, wherein the determination unit includes the target object identification model for identifying whether or not each pixel is a predetermined target object in the image data. It is determined whether or not the object is a previously registered object.
A terminal device according to a 60th aspect is the terminal device according to the 59th aspect, in which the accepting unit causes the obtaining unit to obtain the determination image data obtained from the identification result for each pixel identified using the object identification model. It is shown on the output unit by being superimposed on the acquired image data.
A dimension data calculation device according to a sixty-first aspect includes a shape parameter acquisition unit and a calculation unit. The shape parameter acquisition unit acquires the value of the shape parameter of the object. The calculation unit configures the three-dimensional mesh data of the target from the value of the shape parameter of the target, and based on the information of the vertices of the three-dimensional mesh data that configures the predetermined region of the body associated with the predetermined region. , Dimension data of a predetermined part is calculated.
A dimension data calculation device according to a sixty-second aspect is the dimension data calculation device according to the sixty-first aspect, in which the calculation unit partially forms a part region from a set of vertices of the three-dimensional mesh data according to a predetermined part. Selectively extract the calculation points associated with. Then, the calculation unit calculates the dimension data based on each calculation point.
The dimension data calculation device according to the 63rd aspect is the dimension data calculation device according to the 62nd aspect, wherein the part region is a tubular region. Then, the dimension data is calculated by calculating the circumference length of the cylindrical region based on the calculation points.
A dimension data calculation device according to a sixty-fourth aspect is the dimension data calculation device according to the sixty-third aspect, in which the calculation section is orthogonal to the centroid axis of the tubular region and has the origin at the centroid axis. The calculation points are individually extracted from a set of vertices existing in three or more quadrants of the coordinate system defined with respect to the major axis direction and the minor axis direction.
A dimension data calculation apparatus according to a sixty-fifth aspect is the dimension data calculation apparatus according to the sixty-third aspect, in which the calculation unit scans the tubular region in the direction of the center of gravity axis to determine the length and/or the length in the long axis direction. A cross-sectional area of the tubular area having the smallest axial length is extracted. Then, the calculation unit extracts, as the calculation points, two vertices having a minimum length in the long axis direction and two vertices having a minimum length in the short axis direction in the cross-sectional area.
In the above description, a method based on principal component analysis is mentioned as an example of dimension reduction, but the dimension reduction method is not limited to the above. For example, a dimension reduction method such as an automatic encoder, latent semantic analysis (LSA), and independent component analysis (ICA) may be adopted. Further, when using the teacher data of the dimension data, it is also possible to adopt a dimension reduction method such as partial least squares (PLS), canonical correlation analysis (CCA), and linear discriminant analysis (LDA).
Further, in the above description, when the object is a "person", as an example of each part thereof, in particular, neck width, shoulder width, chest measurement, waist circumference, waist, sleeve length, hips, thigh width, knee width, inseam, length・It is possible to calculate dimensional data such as armholes, thickness of upper arms, and wrist circumference with high accuracy.
Further, in the above description, the product manufacturing apparatus may be one that manufactures "clothes" as a product.
Further, in the above description, when the object is a “person”, the size data may be calculated by dividing into a certain age group and using different parameters for each age group. Similarly, the dimensional data may be calculated using different parameters for each gender.

1001 product manufacturing system 1010 terminal device 1020 size data calculation device 1021 storage unit 1022 input/output unit 1023 communication unit 1024 processing unit 1024A acquisition unit 1024B extraction unit 1024C conversion unit 1024D calculation unit 1030 product manufacturing device 2001S product manufacturing system 2120 size data calculation device 2121 storage unit 2122 input/output unit 2123 communication unit 2124 processing unit 2124A acquisition unit 2124D calculation unit 3001 product manufacturing system 3020 dimension data calculation device 3021 storage unit 3022 input/output unit 3023 communication unit 3024 processing unit 3024A acquisition unit 3024B extraction unit 3024C conversion unit 3024D Estimation unit 3024E Calculation unit 3025 Learning device 3026 Storage unit 3027 Processing unit 3027A Pre-processing unit 3027B Learning unit 3100 Dimension data calculation system 4001S Product manufacturing system 4120 Dimension data calculation device 4121 Storage unit 4122 Input/output unit 4123 Communication unit 4124 Processing unit 4124A Acquisition unit 4124D Estimation unit 4124E Calculation unit 4125 Learning device 4126 Storage unit 4127 Processing unit 4127A Pre-processing unit 4127B Learning unit 4200 Dimension data calculation system 5020 Silhouette image generation device 5024A Acquisition unit 5024B Extraction unit 5024C Conversion unit 5124 Three-dimensional point cloud generation unit 5224 background point cloud removal unit 5324 plane point cloud removal unit 5424 object region extraction unit 5524 shape data extraction unit 5624 object detection unit 6020 dimension data calculation device 6024D shape parameter acquisition unit 6024E calculation unit 6124 three-dimensional data configuration unit 6224 site region configuration 6324 Calculation point extraction section 6424 Dimension data calculation section 7007 Object 7011 Acquisition section 7012 Communication section 7013 Processing section 7014 Input/output section 7020 Terminal device

Claims

An acquisition unit that acquires image data of a captured object and full-length data of the object,
An extraction unit that extracts shape data indicating the shape of the object from the image data,
A conversion unit for converting the shape data based on the full length data,
The dimension data of the shape data converted by the conversion unit is reduced, and the dimension data of each portion of the target object is calculated using the reduced value of each dimension and the weighting coefficient optimized for each portion of the target object. A calculation unit that
A dimension data calculation device comprising:
The calculation unit
The first dimension reduction is performed on the shape data converted by the conversion unit,
Each dimension obtained by the first dimension reduction is linearly combined with a value of each dimension obtained by the first dimension reduction and a weighting coefficient optimized for each part of the object to obtain a predetermined value. A quadratic feature amount is generated from the dimension value, the quadratic feature amount and the weighting coefficient optimized for each part of the object are combined to obtain a predetermined value,
A second dimension reduction is performed using the predetermined value and attribute data including at least attributes of the length and weight of the object,
Dimension data of each part of the object is calculated based on the value of each dimension obtained in the second dimension reduction,
The dimensional data calculation device according to claim 1.
A product manufacturing apparatus that manufactures a product related to the shape of the target object using the dimension data calculated using the dimension data calculation apparatus according to claim 1 or 2.
A reception unit that receives the silhouette image of the object,
Estimating the shape parameter value of the object from the accepted silhouette image using an object engine that associates a silhouette image of the sample object with a value of a predetermined number of shape parameters associated with the sample object An estimation unit that
An information processing apparatus, comprising: the value of the estimated shape parameter of the target object is associated with dimension data related to an arbitrary part of the target object.
The information processing according to claim 4, wherein the object engine is generated by learning a relationship between a silhouette image of the sample object and a value of a predetermined number of shape parameters associated with the sample object. apparatus.
Three-dimensional data of a plurality of vertices in the object is configured from the value of the estimated shape parameter for the object, and based on the configured three-dimensional data, any two vertices of the object are calculated. A calculation unit that calculates the dimension data between
The information processing apparatus according to claim 4, further comprising:
The silhouette image of the object is generated by separating an image of the object and an image other than the object based on depth data obtained using a depth data measuring device. The information processing device according to any one of claims.
A reception unit that receives the attribute data of the object,
Estimating the value of the shape parameter of the object from the received attribute data using an object engine that associates the attribute data of the sample object with the value of a predetermined number of shape parameters associated with the sample object An estimation unit that
An information processing apparatus, comprising: the value of the estimated shape parameter of the target object is associated with dimension data related to an arbitrary part of the target object.
The information processing according to claim 8, wherein the object engine is generated by learning a relationship between attribute data of the sample object and values of a predetermined number of shape parameters associated with the sample object. apparatus.
Three-dimensional data of a plurality of vertices in the object is configured from the value of the estimated shape parameter for the object, and based on the configured three-dimensional data, any two vertices of the object are calculated. A calculation unit that calculates the dimension data between
The information processing apparatus according to claim 8, further comprising:
An acquisition unit that acquires image data of a captured object and full-length data of the object,
An extraction unit that extracts shape data indicating the shape of the object from the image data,
A conversion unit that converts the shape data into a silhouette image based on the full length data,
An estimation unit that estimates a value of a predetermined number of shape parameters from the silhouette image using an object engine that associates a silhouette image of a sample object with a value of a predetermined number of shape parameters associated with the sample object. When,
Based on the value of the estimated predetermined number of shape parameters, a calculation unit for calculating the dimension data of the object,
A dimension data calculation device comprising:
A product manufacturing apparatus that manufactures a product related to the shape of the target object using at least one size data calculated using the size data calculation apparatus according to claim 11.
An acquisition unit that acquires attribute data including at least one of full length data and weight data of the object,
The attribute data, by a polynomial regression using a coefficient learned by machine learning, a calculation unit for calculating the dimensional data of each part of the object,
A dimension data calculation device comprising:
The calculation unit calculates the dimension data of each part of the object by performing a second-order regression on the attribute data using a coefficient learned by machine learning.
The dimension data calculation device according to claim 13.
An acquisition unit that acquires image data including a depth map, in which an object is photographed,
An object area of the object is extracted using three-dimensional point cloud data generated from the depth map, and the shape of the object is shown based on the depth data of the depth map corresponding to the object area. An extraction unit for extracting shape data,
A conversion unit that converts the shape data to generate a silhouette image of the object,
A silhouette image generation device comprising:
The extraction unit is further
Estimating the plane portion in the image data from the three-dimensional point cloud data generated from the depth map,
The object region of the object is extracted based on the three-dimensional point cloud data excluding the three-dimensional point cloud data existing in the estimated plane portion,
The silhouette image generation device according to claim 15.
An acquisition unit that acquires image data of a captured object and full-length data of the object,
An extraction unit that extracts shape data indicating the shape of the object from the image data,
A calculator that calculates the dimension data of each part of the object using the shape data;
The dimension data calculation device, wherein the image data includes a depth map, and the shape data extracted by the extraction unit is associated with the depth data of the object in the depth map.
A product manufacturing apparatus that manufactures a product related to the shape of the target object using the dimension data calculated using the dimension data calculation apparatus according to claim 17.
An acquisition unit that acquires image data of a captured object and full-length data of the object,
An extraction unit that extracts shape data indicating the shape of the object from the image data,
Estimating the value of the predetermined number of shape parameters from the silhouette image of the object using an object engine that associates the silhouette image of the sample object with the value of the predetermined number of shape parameters associated with the sample object An estimation unit that
A calculation unit that calculates the dimension data of the object based on the values of the estimated predetermined number of shape parameters;
A dimension data calculation device, wherein the image data includes a depth map, and the shape data is associated with depth data of the object in the depth map.
A product manufacturing apparatus that manufactures a product related to the shape of the object by using the dimension data calculated by the dimension data calculating apparatus according to claim 19.
A terminal device connected to an information processing device that processes information about the object from image data of the object captured,
An acquisition unit that acquires image data of the object captured,
A determination unit that determines whether the target object included in the image data is a pre-registered target object,
While showing the determination result by the determination unit to the output unit, a reception unit that receives an input as to whether or not to transmit the image data to the information processing device,
A terminal device comprising:
The determination unit determines whether or not the target included in the image data is a pre-registered target by using a target identification model that identifies whether or not each pixel is a predetermined target. ,
The terminal device according to claim 21.
A shape parameter acquisition unit that acquires the value of the shape parameter of the object,
Based on the vertex information of the three-dimensional mesh data that forms the three-dimensional mesh data of the object from the value of the shape parameter of the object and that forms a predetermined part region associated with a predetermined part, A calculation unit that calculates the dimension data of the predetermined part,
A dimension data calculation device comprising:
The calculation unit, according to the predetermined site,
From the set of vertices of the three-dimensional mesh data, calculation points that are partially associated with the part region are selectively extracted,
Calculating the dimensional data based on each of the calculation points,
The dimension data calculation device according to claim 23.
The part region is a tubular region,
The dimension data is calculated by calculating the length of the circumference of the tubular region based on the calculation points.
The dimension data calculation device according to claim 24.