JP2018055506A - Information processing apparatus, information processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain three-dimensional data with high similarity with respect to image information specified by a client without causing a trouble on the client.SOLUTION: An information processing apparatus according to the present disclosure includes an input unit to receive an input of three-dimensional data relating to a three-dimensional model and a two-dimensional input image, an image generation unit that generates a two-dimensionally prepared image showing an appearance of the three-dimensional model disposed in virtual space, as viewed from a given viewpoint, based on the three-dimensional data, and an output unit that outputs a similarity between the two-dimensionally prepared image and the two-dimensional input image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  Patent Document 1 below discloses a technique for generating a three-dimensional shape model using a plurality of pieces of image information designated by a client.

Japanese Patent Laying-Open No. 2014-4466

  However, in the conventional information processing apparatus, it is necessary to make the requester troublesome. That is, according to the above-described conventional configuration, in order to ensure that the quality of the completed three-dimensional shape model, specifically, the image information specified by the client and the three-dimensional shape model are similar to each other in advance. It is necessary to communicate with the requester such as sending 3D data to the requester and receiving a correction request if they are not similar, and this makes the requester troublesome.

  The present disclosure has been made in view of the above-described problems, and the purpose thereof is to obtain three-dimensional data having high similarity to image information designated by the client without requiring the client. That is.

  (1) An information processing apparatus according to the present disclosure includes an input unit that receives input of three-dimensional data related to a three-dimensional model and a two-dimensional input image, and the three-dimensional model arranged in a virtual space based on the three-dimensional data An image generation unit that generates a two-dimensional created image showing an appearance viewed from a given viewpoint, and an output unit that outputs a similarity between the two-dimensional created image and the two-dimensional input image.

  (2) In the information processing apparatus according to (1), the image generation unit generates a plurality of two-dimensional created images showing an appearance of the three-dimensional model arranged in the virtual space as viewed from a plurality of viewpoints. The output unit may output the highest similarity among the similarities between each of the plurality of two-dimensional created images and the two-dimensional input image.

  (3) In the information processing apparatus according to (1) or (2), the similarity between each of the plurality of two-dimensional created images and the two-dimensional input image is calculated, and the two-dimensional created image having the highest similarity is calculated. An extraction unit for extracting from the plurality of two-dimensional created images may be further provided.

  (4) In the information processing apparatus according to (3), the input unit receives input of a plurality of two-dimensional input images, and the extraction unit receives the plurality of two-dimensional input images for each of the plurality of two-dimensional input images. A two-dimensional created image having the highest similarity is extracted from the created images according to the number of the plurality of two-dimensional input images, and the plurality of two-dimensional created images extracted by the extraction unit and the plurality of two-dimensional images are extracted. An arithmetic unit that calculates an average value of each similarity with the input image may be further included, and the output unit may output the average value calculated by the arithmetic unit.

  (5) In the information processing apparatus according to (4), the average value may include a weighted average value weighted with respect to a two-dimensional input image including a feature portion among the plurality of two-dimensional input images.

  (6) In the information processing apparatus according to (5), weighting information regarding whether or not a feature part is included may be added to the two-dimensional input image from which the input part receives input.

  (7) In the information processing apparatus according to (3) to (6), the information processing apparatus further includes a dividing unit that divides the two-dimensionally created image extracted by the extracting unit into a plurality of regions, The similarity between the region and the two-dimensional input image is calculated, and the output unit is configured to display a region having a similarity lower than a predetermined value among the plurality of regions differently from a display of another region. Also good.

  (8) In the information processing apparatus according to (7), the output unit may color and display an area where the similarity is lower than a predetermined value.

  (9) In the information processing apparatus according to (1) to (8), the two-dimensional input image may be a photograph.

  (10) An information processing method according to the present disclosure includes an input step of receiving input of three-dimensional data related to a three-dimensional model and a two-dimensional input image, and the three-dimensional model arranged in a virtual space based on the three-dimensional data An image generation step of generating a two-dimensional created image showing an appearance viewed from a given viewpoint, and an output step of outputting the similarity between the two-dimensional created image and the two-dimensional input image.

  (11) A program according to the present disclosure includes an input step of accepting input of 3D data and a 2D input image relating to a 3D model to a computer, and the 3D arranged in a virtual space based on the 3D data An image generation step for generating a two-dimensional created image showing an appearance of the model viewed from a given viewpoint, and an output step for outputting the similarity between the two-dimensional created image and the two-dimensional input image are executed.

FIG. 1 is a schematic diagram of an information processing apparatus according to the present embodiment. FIG. 2 is a flowchart of the information processing method according to the present embodiment. FIG. 3 is a diagram illustrating a concept of calculation steps in the information processing method according to the present embodiment. FIG. 4 is a diagram illustrating a concept of calculation steps in the information processing method according to the present embodiment.

[First Embodiment]
Embodiments of the present disclosure will be described below with reference to the drawings.

  FIG. 1 is a schematic diagram of an information processing apparatus according to the present embodiment.

  As illustrated in FIG. 1, the information processing apparatus 100 is a server apparatus, and controls the communication unit 70 that transmits and receives data to and from the client computer and the modeler computer that creates three-dimensional data, and controls the entire information processing apparatus 100. And a storage unit 50 that stores a program and the like used for the control. In the present disclosure, the three-dimensional data includes, for example, three-dimensional CAD (computer-aided design) data, three-dimensional CG (computer graphics) data, and the like.

(Communication unit 70)
The communication unit 70 transmits / receives data to / from the requester and the modeler computer by wire or wireless. The communication unit 70 communicates with an external device directly or indirectly using, for example, a LAN (Local Area Network), Wi-Fi (Wireless Fidelity, registered trademark), a telephone line, a broadband network, or the like.

(Control unit 40)
The control unit 40 controls the overall operation of the information processing apparatus 100 according to various programs stored in the storage unit 50. The control unit 40 is realized by an electronic circuit such as a CPU (Central Processing Unit) or a microprocessor, for example. Note that the control unit 40 may include a ROM (Read Only Memory) that stores programs to be used, calculation parameters, and the like, and a RAM (Random Access Memory) that temporarily stores parameters that change as appropriate.

  The control unit 40 includes an input unit 10 that receives input of three-dimensional data and a two-dimensional input image, an image generation unit 20 that generates a two-dimensional created image based on the three-dimensional data, and an output unit 30 that outputs similarity. Including. Further, in the present embodiment, the control unit 40 extracts a two-dimensional created image having a highest similarity to the two-dimensional input image from a plurality of two-dimensional created images, and a plurality of two-dimensional created images. And a calculation unit 24 that calculates an average value of similarities between the plurality of two-dimensional created images and the plurality of two-dimensional input images.

(Input unit 10)
The input unit 10 receives an input of a two-dimensional input image from a requester who requests the manufacture of a three-dimensional shape model. Here, the two-dimensional input image is, for example, a photograph of a client pet. In the present embodiment, an example will be described in which a requester has requested the manufacture of a pet three-dimensional shape model. The input of the two-dimensional input image is performed when the client accesses the information processing apparatus 100 that is a server apparatus and inputs the two-dimensional input image to the input unit 10 via the communication unit 70 described above. Note that it is also possible for the user of the information processing apparatus 100 to directly input a two-dimensional input image obtained from the requester via e-mail, recording medium, or the like, to the input unit 10 without going through the communication unit 70. . The two-dimensional input image input to the input unit 10 is stored in the storage unit 50.

  The input unit 10 receives input of three-dimensional data indicating a three-dimensional model created by the modeler based on a two-dimensional input image from the client. The input of the three-dimensional data is performed when the modeler accesses the information processing apparatus 100 that is a server apparatus and inputs the three-dimensional data to the input unit 10 via the communication unit 70 described above. Note that it is also possible for the user of the information processing apparatus 100 to directly input the three-dimensional data obtained from the modeler via mail, a recording medium, or the like to the input unit 10 without going through the communication unit 70. The three-dimensional data input to the input unit 10 is stored in the storage unit 50.

(Image generation unit 20)
The image generation unit 20 generates a two-dimensional created image showing an appearance of a three-dimensional model arranged in the virtual space viewed from a given viewpoint based on the three-dimensional data created by the modeler.

  As a first specific example, a case will be described in which the two-dimensional input image input by the client is a single photograph and a pet face photograph viewed from the front. The image generation unit 20 estimates the camera position from the input face photograph. As a method for estimating the camera position, for example, the image generation unit 20 extracts edge information in the projection image of the three-dimensional model projected on the projection plane arranged between the projection center of the virtual space and the three-dimensional model. As a result, the position where the pet contour shape in the two-dimensional input image input by the client and the contour shape of the three-dimensional model in the projection image most closely match is estimated as the camera position. Next, the position of the three-dimensional model with respect to the projection center in the virtual space is adjusted based on the estimated camera position information. Then, the image generation unit 20 acquires a projection image of the three-dimensional model projected on the projection plane arranged between the projection center of the virtual space and the three-dimensional model as a two-dimensional created image. Further, the image generation unit 20 acquires a plurality of two-dimensional created images projected on the projection plane while rotating the three-dimensional model arranged in the virtual space, for example, by one degree. In this specific example, a plurality of two-dimensional created images showing the appearance of the pet's face viewed from a plurality of angles near the front are generated as the appearance close to the pet's face photograph input by the client.

  As a second specific example, the two-dimensional input image input by the client is a plurality of photographs. For example, the first sheet is a “face photo of a pet viewed from the front direction”, and the second sheet is “from the front direction. In the case of the “total photo of the pet seen”, the third photo “the overall photo of the pet seen from the side direction”, and the fourth photo “the whole photo of the pet seen from the back direction”, the image generating unit 20 In this way, a plurality of two-dimensional created image groups are generated.

  The image generation unit 20 adjusts the positional relationship between the projection center and the three-dimensional model in the virtual space, based on the camera position information estimated from the first “face photograph of the pet seen from the front direction”. For example, in the virtual space, the face area of the three-dimensional model is arranged at a position close to the viewpoint of the camera that is the projection center. Then, the image generation unit 20 rotates the three-dimensional model by 1 degree from the front direction, and generates a first two-dimensional created image group showing the appearance of the pet's face viewed from a plurality of angles near the front.

  The image generation unit 20 adjusts the positional relationship between the projection center and the three-dimensional model in the virtual space based on the camera position information estimated from the second “entire photo of the pet viewed from the front direction”. For example, in the virtual space, the viewpoint of the camera that is the projection center is adjusted to a position that captures the entire front surface of the three-dimensional model. Then, the image generation unit 20 rotates the 3D model from the front direction one by one to generate a second two-dimensional created image group that shows the overall appearance of the pet viewed from a plurality of angles near the front.

  The image generation unit 20 adjusts the positional relationship between the projection center in the virtual space and the three-dimensional model based on the camera position information estimated from the third “entire photo of the pet viewed from the side direction”. For example, in the virtual space, the viewpoint of the camera that is the projection center is adjusted to a position that captures the entire side surface of the three-dimensional model. Then, the image generation unit 20 rotates by one degree from the side surface direction of the three-dimensional model, and generates a third two-dimensional created image group that shows the overall appearance of the pet viewed from a plurality of angles near the side surface.

  The image generation unit 20 adjusts the positional relationship between the projection center and the three-dimensional model in the virtual space based on the camera position information estimated from the fourth “entire photo of the pet viewed from the back direction”. For example, in the virtual space, the viewpoint of the camera that is the projection center is adjusted to a position that captures the entire back surface of the three-dimensional model. Then, the image generation unit 20 rotates the 3D model from the back direction one by one to generate a fourth two-dimensional created image group that shows the overall appearance of the pet viewed from a plurality of angles near the back surface.

  In the present embodiment, the example in which the camera position is estimated and then the three-dimensional model is rotated to acquire a plurality of two-dimensional created images has been described. However, the step of estimating the camera position is omitted, and the three-dimensional model is obtained. The model may be rotated 360 degrees one by one to generate 360 two-dimensionally created image groups, and in the processing of the extraction unit 22 described later, a configuration having a high degree of similarity is extracted from 360 two-dimensionally created image groups. I do not care.

  Further, in the present embodiment, an example in which the camera position is estimated and then the three-dimensional model is rotated to acquire a plurality of two-dimensional created images has been described. However, one two-dimensional created image is obtained from the estimated camera position. May be obtained, and in the processing of the output unit 30 to be described later, the similarity of the one two-dimensionally created image may be output.

(Extractor 22)
When a plurality of two-dimensional created image groups are acquired in the image generation unit 20, the extraction unit 22 calculates the similarity between the two-dimensional created image group and the two-dimensional input image, and the two-dimensional created image having the highest similarity. Are extracted from the two-dimensionally created image group. When one 2D creation image group is acquired in the image generation unit 20, the extraction unit 22 only calculates the similarity between the 2D creation image group and the 2D input image.

  A specific example when a plurality of two-dimensional created image groups are acquired in the image generation unit 20 will be described. For all the two-dimensionally created images of the first two-dimensionally created image group generated by the image generating unit 20, the similarity to the first photo “pet face photograph viewed from the front direction” is calculated. Here, the similarity is, for example, the size of the eyes of the pet, the length of the ear, the length of the nose, the distance between the eyes, the angle of each vertex in the triangle connecting the position of both eyes and the position of the nose, etc. The degree of coincidence is used as a judgment item and calculated. As a method of calculating the similarity, for example, it is possible to calculate using a class classification algorithm such as machine learning including reinforcement learning and deep learning, and the similarity is calculated using a simpler algorithm. May be. As a method of calculating the similarity with a simple algorithm, for example, if the similarity of the pet eye size is calculated, the area of the pet eye in the two-dimensional created image and the area of the pet eye in the photograph are calculated. Of these, a value obtained by dividing the larger one by the smaller one may be calculated as the similarity. If the similarity of the length of the pet's ear or nose is calculated, the larger one of the length of the pet's ear or nose in the two-dimensional created image and the length of the pet's ear or nose in the photo A value obtained by dividing a small value may be calculated as the similarity. If the similarity between the two eyes is to be calculated, for example, the similarity is calculated by dividing the pet's two eyes interval in the two-dimensional created image and the pet's two eyes interval in the photo by dividing the larger one into the smaller one. May be. If the similarity of a triangle connecting the position of both eyes and the position of the nose is calculated, for example, the triangle in the two-dimensional created image is overlapped with the triangle in the photograph, and the area of the triangle in the two-dimensional created image is set to 100. The percentage of the area of the overlapped portion in the case of% may be calculated as the similarity. Then, the first two-dimensional created image having the highest similarity is extracted from the first two-dimensional created image group by the extraction unit 22.

  For all the two-dimensional created images of the second two-dimensional created image group generated by the image generating unit 20, the degree of similarity is calculated with the “total photo of the pet viewed from the front direction” that is the second photograph. Then, the second two-dimensional created image having the highest similarity is extracted from the second two-dimensional created image group by the extraction unit 22.

  For all the two-dimensionally created images of the third two-dimensionally created image group generated by the image generating unit 20, the similarity with the “total photo of the pet viewed from the side direction” that is the third photograph is calculated. Then, the third two-dimensional created image having the highest similarity is extracted from the third two-dimensional created image group by the extraction unit 22.

  For all the two-dimensional created images of the fourth two-dimensional created image group generated by the image generating unit 20, the similarity with the “entire photo of the pet as viewed from the back direction” that is the fourth photograph is calculated. Then, the fourth two-dimensional created image having the highest similarity is extracted from the fourth two-dimensional created image group by the extraction unit 22.

(Calculation unit 24)
The calculation unit 24 calculates an average value of the similarities between the plurality of two-dimensional created images extracted by the extraction unit 22 and the plurality of two-dimensional input images.

  In the present embodiment, as shown in FIG. 3, the similarity between the first photograph and the first two-dimensional created image is 75 points, and the similarity between the second photograph and the second two-dimensional created image. If the degree is 70 points, the similarity between the third photograph and the third two-dimensional created image is 90 points, and the similarity between the fourth photograph and the fourth two-dimensional created image is 80 points, The average value 78.75 is calculated.

  The average value may be a weighted average value weighted with respect to a two-dimensional input image including a feature portion among a plurality of two-dimensional input images. That is, when the arithmetic unit 24 calculates the average value of the respective similarities between the plurality of two-dimensional created images and the plurality of two-dimensional input images, the calculation unit 24 includes a characteristic portion among the plurality of two-dimensional input images. An average value may be calculated by weighting. For example, whether or not the finally created 3D shape model is similar to the pet itself, such as the pet's face viewed from the front and the shape of the skeleton when the pet takes a unique pose. It may be important in determining whether or not. In such a case, weighting information is added to the photo in the front direction of the pet's face and the photo in which the skeleton shape of the entire pet that is posing can be recognized. By doing this, the calculated value of the similarity of the photo including the feature part has an influence on the calculated value of the overall similarity, and the calculated value of the similarity of the other photo is changed to the calculated value of the overall similarity. The degree of influence can be made larger.

  Note that the weighting information as to whether or not this feature is included may be added to the two-dimensional input image when the input unit 10 receives an input of the two-dimensional input image.

(Output unit 30)
When a plurality of two-dimensional created image groups are acquired in the image generation unit 20, the output unit 30 outputs the similarity of the two-dimensional created image determined by the extraction unit 22 as having the highest similarity. When the two-dimensional input image is a plurality of two-dimensional input images, the average value calculated by the calculation unit 24 is output. The modeler can check the output result of the output unit 30 via the communication unit 70. When one 2D creation image group is acquired in the image generation unit 20, the output unit 30 outputs the similarity of the 2D creation image calculated by the extraction unit 22.

  By adopting such a configuration, if the output result does not exceed the predetermined value, the modeler is encouraged to appropriately correct the three-dimensional data until the output result exceeds the predetermined value. It becomes possible. As a result, the quality of the three-dimensional data to be sent to the requester can be ensured, and the number of times the requester makes a correction request can be reduced, so that the effort for the requester can be reduced.

(Division part 26)
Further, the control unit 40 may further include a dividing unit 26 as shown in FIG. The dividing unit 26 divides the two-dimensional created image extracted by the extracting unit 22 into a plurality of regions. For example, the dividing unit 26 divides the two-dimensionally created image into a triangular area connecting both eyes and nose of the pet, a face area other than that, a forefoot area, a hind leg area, a side area of the trunk, a back area of the trunk, and the like. With such a configuration, in the output unit 30 to be described later, it is possible to make the display of an area having a similarity lower than a predetermined value different from the display of another area.

  In addition, the control unit 40 includes the dividing unit 26, the calculation unit 24 calculates the similarity of each region divided by the dividing unit 26, and the output unit 30 displays the region where the similarity is lower than a predetermined value. Can be output differently from the display of other areas, it is easy for the modeler to recognize the points to be corrected, and the work efficiency of the modeler can be improved. Note that the algorithm used when the calculation unit 24 calculates the similarity may be the same as the algorithm used by the extraction unit 22 to calculate the similarity.

  As a specific example in which the output unit 30 makes the display of the region whose similarity is lower than the predetermined value different from the display of the other region, for example, coloring the region where the similarity is lower than the predetermined value, such as red, Alternatively, the boundary line of the region is displayed with a red line, a thick line, and the like. Or, according to the score of the similarity, for example, the output unit 30 is blue for a high-scoring region, yellow for a region that has reached a predetermined value but has not reached a high score, and has a predetermined value. A region that has not reached may be colored in a plurality of colors such as red.

(Storage unit 50)
The storage unit 50 records and reproduces data with respect to a predetermined recording medium. The storage unit 50 is realized as an HDD (Hard Disc Drive), for example. Various recording media, such as a solid-state memory such as a flash memory, a memory card incorporating a solid-state memory, an optical disk, a magneto-optical disk, and a hologram memory, are conceivable. The storage unit 50 can perform recording and reproduction according to the recording medium employed. What is necessary is just to be set as a structure. The storage unit 50 stores a program used by the control unit 40.

  The storage unit 50 can store the two-dimensional input image and the three-dimensional data input to the input unit 10 according to an instruction from the control unit 40, and can also store two-dimensional data according to an instruction from the control unit 40. The input image and the three-dimensional data can be obtained by the control unit 40.

(Information processing method)
Hereinafter, the information processing method will be described with reference to FIG.

  FIG. 2 is a flowchart illustrating an example of the price estimation method according to the present embodiment.

(Input step S310)
As shown in FIG. 2, in the input step S310, the input unit 10 includes a two-dimensional input image input step S311 for receiving an input of a two-dimensional input image, and a three-dimensional data input step S312 for receiving an input of three-dimensional data. Including.

(Two-dimensional input image input step S311)
In the two-dimensional input image input step, the input unit 10 receives a two-dimensional input image from a client who requests the manufacture of a three-dimensional shape model. As for the input of the two-dimensional input image, for example, the client accesses the information processing apparatus 100 as the server apparatus from his / her computer and inputs the two-dimensional input image to the input unit 10 via the communication unit 70 described above. Is done. Alternatively, it is performed by the user of the information processing apparatus 100 inputting a two-dimensional input image obtained from the requester via mail, a recording medium, or the like to the input unit 10. The two-dimensional input image input to the input unit 10 is stored in the storage unit 50.

  In the present embodiment, the two-dimensional input image input by the client is a plurality of photographs, the first is a “face photo of the pet viewed from the front direction”, and the second is the “pet viewed from the front direction”. Suppose that the third photograph is the “entire photo of the pet seen from the side direction” and the fourth photograph is the “entire photo of the pet seen from the rear direction”. All of these “overall pictures of pets” are sitting. In addition, the client has been informed that the crossing angle of both front legs in the sitting posture is characteristic. The angle of intersection of both forelimbs is made clear from the second “Overall photo of the pet as seen from the front”. Therefore, weighting information indicating that the photograph includes a characteristic portion is added to the second “entire photo of the pet viewed from the front direction” requested by the client. Similar weighting information is also added to the first “face photo of the pet as seen from the front”, which generally has a large effect on the similarity of the three-dimensional shape model.

(Three-dimensional data input step S312)
In the three-dimensional data input step, the modeler creates three-dimensional data based on the two-dimensional input image from the client, and the modeler or the user of the information processing apparatus 100 inputs the three-dimensional data to the input unit 10.

  For example, the modeler can access the information processing apparatus 100 from his computer and check the two-dimensional input image stored in the storage unit 50 via the communication unit 70. The modeler creates three-dimensional data based on the two-dimensional input image. The modeler again accesses the information processing apparatus 100 from its own computer, and stores the three-dimensional data in the storage unit 50 via the communication unit 70. Alternatively, the user of the information processing apparatus 100 inputs the three-dimensional data obtained from the modeler via mail, a recording medium, or the like to the input unit 10 and stores it in the storage unit 50.

(Image generation step S320)
As shown in FIG. 3, when the input step S310 is completed, the image generation step S320 can be started. The start timing of the image generation step S320 may be determined by the user of the information processing apparatus 100 or a modeler.

  In the image generation step S320, based on the three-dimensional data input to the information processing apparatus 100 in the input step S310, a two-dimensional created image showing the appearance of the three-dimensional model arranged in the virtual space viewed from a given viewpoint. Generate.

  In the present embodiment, the two-dimensional input image input by the client is a photograph of four pets. For example, the first sheet is a “pet face photograph viewed from the front direction”, and the second sheet is “front direction”. It is assumed that the whole picture of the pet viewed from the side, the third picture is the whole picture of the pet seen from the side, and the fourth picture is the whole picture of the pet seen from the back.

  First, the image generation unit 20 estimates the camera position from the input face photograph, and adjusts the positions of the three-dimensional model and the projection center in the virtual space based on the estimated camera position information. In the present embodiment, the image generation unit 20 displays the projection image of the three-dimensional model on the projection surface while rotating the three-dimensional model arranged in the virtual space one by one, and is input by the requester. A two-dimensional created image group showing an appearance close to a pet face photograph is generated.

  Note that in this image generation step S320, the step of estimating the camera position is omitted. For example, the image generation unit 20 rotates the three-dimensional model 360 degrees one by one to generate 360 two-dimensionally created image groups, In the extraction step S330, which will be described later, a configuration in which a high similarity is extracted from a group of 360 two-dimensionally created images may be used.

  In this image generation step S320, the image generation unit 20 does not acquire a plurality of two-dimensional created images, but acquires one two-dimensional created image from the estimated camera position, and in an extraction step S330 described later. The similarity between the one-dimensional created image and the two-dimensional input image may be calculated, and the calculated similarity may be output in the output step S360 described later.

  In the present embodiment, the image generation unit 20 generates a first two-dimensional created image group for the first “face photo of the pet viewed from the front direction”, and generates the second “front direction” The second two-dimensional created image group is generated for the “total photo of the pet viewed from the side”, and the third two-dimensional generated image is generated for the third “total photo of the pet viewed from the side”. A group is generated, and a fourth two-dimensional created image group is generated for the fourth “entire photo of the pet viewed from the back direction”. The method for generating each two-dimensionally created image group is as described above in (Image generating unit 20).

(Extraction step S330)
In the extraction step S330, the extraction unit 22 included in the control unit 40 calculates the similarity between the two-dimensional creation image group and the two-dimensional input image, and the two-dimensional creation image with the highest similarity is calculated as the two-dimensional creation image. Extract from the group.

  In the present embodiment, the first two-dimensional created image having the highest similarity to the first photograph is extracted from the first two-dimensional created image group, and the second two-dimensional created image group The second two-dimensional created image having the highest similarity with the second photo is extracted, and the third two-dimensional created image having the highest similarity with the third photo is extracted from the third two-dimensional created image group. A two-dimensional created image is extracted, and a fourth two-dimensional created image having the highest similarity to the fourth photograph is extracted from the fourth two-dimensional created image group.

  Next, in S331, it is determined whether or not to divide the area. In the present embodiment, it is selected to divide the region, and the flow proceeds to the dividing step S340.

(Division step S340)
In the present embodiment, since the flow proceeds to the division step S340, the division unit 26 illustrated in FIG. 1 divides the two-dimensional created image extracted by the extraction unit 22 into a plurality of regions. For example, the dividing unit 26 uses a triangle that connects the first two-dimensionally created image determined to have the highest degree of similarity to the first “face photo of the pet viewed from the front” to connect both eyes and the mouth of the pet. The area is divided into a “front triangle area” that is an area and a “front other face area” that is a face area other than the triangle area. Further, the dividing unit 26 converts the second two-dimensional created image determined to have the highest degree of similarity with the second “entire photo of the pet viewed from the front direction”, “front body region”, and “front Divide into “forefoot areas”. In addition, the dividing unit 26 converts the third two-dimensional created image determined to have the highest degree of similarity to the third “entire photo of the pet viewed from the side direction”, “side head region”, “ It is divided into a “side body region”, a “side front foot region”, and a “side rear foot region”. Further, the dividing unit 26 converts the fourth two-dimensional created image determined to have the highest similarity to the fourth “entire photo of the pet viewed from the back direction” into “back head region” and “ It is divided into “back fuselage area”.

(Calculation step S350)
In calculation step S350, the calculation unit 24 calculates an average value of the similarities between the plurality of two-dimensional created images extracted by the extraction unit 22 and the plurality of two-dimensional input images.

  FIG. 4 is a diagram illustrating a concept of calculation steps in the information processing method according to the present embodiment.

  The calculation unit 24 includes a triangular area connecting the eyes and the mouth of the pet recognized from the “first face photograph of the pet as viewed from the front”, and the triangular area in the first two-dimensional created image. A comparison is made with a certain “frontal triangular area”, and the degree of coincidence is displayed as a score. In the present embodiment, as shown in FIG. 4, the calculation unit 24 evaluates that the degree of coincidence of each angle is 80% and there are 80 similarities.

  The calculation unit 24 compares the first photograph “a pet face photograph viewed from the front direction” with the “front other face area” in the first two-dimensionally created image. Here, since the degree of similarity has already been determined in the triangular area connecting the eyes and mouth of the pet, the degree of similarity is determined for the face area other than the triangular area. For example, the arrangement of the mouth and ears in the entire face, the shape of the mouth, the shape of the ears, the ratio of the length of a specific part of the face to the length of another specific part, the color of the entire face, the presence / absence / shape of the pattern, etc. Judgment is made. If the similarity of the arrangement of the mouth and ears in the entire face is calculated, for example, the triangle that connects the mouth and both ears of the pet face in the first two-dimensional created image, and the pet face in the first photograph If the triangle connecting the mouth and both ears is overlapped and the area of the triangle in the first two-dimensional created image is 100%, the ratio of the area of the overlapped portion is calculated as the similarity. Good. If the similarity of the shape of the mouth (or ear) is to be calculated, the pet's mouth (or ear) in the first two-dimensional created image and the pet's mouth (or ear) in the first picture are overlaid. In addition, when the area of the mouth (or ear) in the first two-dimensional created image is 100%, the ratio of the area of the overlapped portion may be calculated as the similarity. If the similarity of the ratio between the length of a specific part on the face and the length of another specific part is calculated, the ratio of the ratio in the first two-dimensional created image and the ratio in the first photograph is larger Alternatively, a value obtained by dividing a smaller value may be calculated as the similarity. In the present embodiment, the calculation unit 24 evaluated that there are 70 similarities as shown in FIG.

  As described in the input step S310, the face region viewed from the front direction is generally an important region for determining the similarity of the three-dimensional shape model, and thus is the first photograph. Weighting information is added to “a pet face photograph viewed from the front”. Therefore, as shown in FIG. 4, weighting is performed on the “front triangular area” and the “front other face area” for comparison with the “face photo of the pet seen from the front direction”.

  The computing unit 24 compares the body recognized from the second photograph “entire photo of the pet viewed from the front direction” with the “front body region” in the second two-dimensionally created image. Here, for example, similarity determination is performed on the shape and aspect ratio of the body, the color of the entire body, the presence / absence / shape of the pattern, and the like. If the similarity of the shape of the torso is calculated, for example, the torso of the pet in the second two-dimensional created image and the torso of the pet in the second photograph are superimposed, When the area of the trunk is 100%, the ratio of the overlapped area may be calculated as the similarity. In the present embodiment, the calculation unit 24 evaluated that there are 80 similarities as shown in FIG.

  The calculation unit 24 compares the forefoot recognized from the “entire photo of the pet viewed from the front direction” that is the second photograph with the “front forefoot region” in the second two-dimensionally created image. Here, for example, the degree of similarity is determined with respect to the position of both front legs with respect to the body region, the angle of intersection of both front legs, the color of the entire front legs, the presence / absence / shape of the pattern, and the like. If the similarity between the position of both forefoot and the intersection angle of both forefoot is calculated, for example, a triangle connecting the base of both forefoot of the pet and the intersection of both feet in the second two-dimensional created image, The area of the overlapped portion when the area connecting the triangles in the second two-dimensional created image is 100% by superimposing the triangles connecting the bases of both forefoot of the pet and the intersection of both legs in the photo of May be calculated as the degree of similarity. In the present embodiment, the calculation unit 24 evaluated that there are 60 similarities as shown in FIG.

  In addition, as explained in the input step S310, the client has been informed that there is a characteristic in the crossing angle of both front legs in the sitting posture, and includes the “pet viewed from the front direction” including the crossing angle information of the front legs. Weighting information is added to the “whole picture”. Both the “front body region” and the “front forefoot region” to be compared with the second photograph may be weighted. However, in the present embodiment, from the contents of communication from the requester, “ Only the “intersection angle of both forefoot” is regarded as a feature, and weighting is not performed on the “front body region”, but only weighting is performed on the “front forefoot region”.

  The computing unit 24 compares the head recognized from the third photograph “the entire picture of the pet viewed from the side direction” with the “side head region” in the third two-dimensional created image. Here, for example, similarity determination is performed on the shape of the head, the position of the eyes, ears, nose, mouth in the head region, the position of the head region relative to the body region, the color of the entire head, the presence / absence / shape of the pattern, etc. Is called. If the similarity between the positions of the eyes, ears, nose, and mouth in the head region is calculated, for example, the quadrilateral connecting the pet's eyes, ears, nose, and mouth in the third two-dimensionally created image and the third piece The ratio of the area of the overlapped part when the area connecting the eyes, ears, nose, and mouth of the pet in the photo is overlapped and the area of the rectangle in the third two-dimensional created image is 100%. The similarity may be calculated. In the present embodiment, the calculation unit 24 evaluated that there are 90 similarities as shown in FIG.

  The calculation unit 24 compares the body recognized from the third photograph “entire pet photograph viewed from the side direction” with the “side body area” in the third two-dimensionally created image. Here, for example, similarity determination is performed on the shape of the body, the color of the entire body, the presence / absence / shape of the pattern, and the like. If the similarity of the shape of the torso is to be calculated, for example, the pet torso in the third two-dimensionally created image is superimposed on the torso of the pet in the third photograph, and in the third two-dimensionally created image When the area of the trunk is 100%, the ratio of the overlapped area may be calculated as the similarity. In the present embodiment, the calculation unit 24 evaluated that there are 90 similarities as shown in FIG.

  The computing unit 24 compares the forefoot recognized from the third photograph “entire photo of the pet viewed from the side direction” and the “side forefoot region” in the third two-dimensionally created image. Here, for example, similarity determination is performed on the shape of the forefoot, the position of the forefoot with respect to the body region, the color of the entire forefoot, the presence / absence / shape of the pattern, and the like. If the similarity of the shape of the forefoot is to be calculated, for example, the pet's forefoot in the third two-dimensional created image and the pet's forefoot in the third photograph are superimposed, When the area of the forefoot is 100%, the ratio of the area of the overlapped portion may be calculated as the similarity. If the similarity of the position of the forefoot with respect to the torso area is calculated, for example, the position of the base of the forefoot of the pet in the third two-dimensional created image matches the position of the forefoot of the pet in the third photograph As such, the ratio of the area of the overlapped portion may be calculated as the degree of similarity when both the body regions are overlapped and the area of the body in the third two-dimensional created image is 100%. In the present embodiment, the calculation unit 24 evaluated that there are 90 similarities as shown in FIG.

  The computing unit 24 compares the hind paw recognized from the third photograph “entire photo of the pet viewed from the side direction” with the “side hind paw region” in the third two-dimensional created image. Here, for example, similarity determination is performed on the shape of the hind legs, the position of the hind legs with respect to the body region, the color of the whole hind legs, the presence / absence / shape of the pattern, and the like. If the similarity of the shape of the hind paws is calculated, for example, the pet hind paw in the third two-dimensional created image is superimposed on the hind paw of the pet in the third photograph, and the third two-dimensional When the area of the hind legs in the created image is 100%, the ratio of the area of the overlapped portion may be calculated as the similarity. If the similarity of the position of the hind paw with respect to the torso region is calculated, for example, the position of the base of the hind paw of the pet in the third two-dimensional created image and the base of the hind paw of the pet in the third photograph The third two-dimensionally created image and the body region of the third photograph are overlapped so that the positions coincide with each other, and the body area in the third two-dimensionally created image is set to 100%. The proportion of the area of the part may be calculated as the similarity. In the present embodiment, the calculation unit 24 evaluated that there are 90 similarities as shown in FIG.

  The calculation unit 24 compares the head recognized from the “fourth photograph,“ entire photo of the pet as viewed from the back direction ””, and the “back head region” in the fourth two-dimensionally created image. Here, for example, similarity determination is performed on the shape of the head, the position of the head region relative to the body region, the color of the entire head, the presence / absence / shape of the pattern, and the like. If the similarity of the shape of the head is to be calculated, for example, the pet's head in the fourth two-dimensional created image and the pet's head in the fourth photograph are overlaid, and the fourth two-dimensional When the area of the head in the created image is 100%, the ratio of the area of the overlapped portion may be calculated as the similarity. If the similarity of the position of the head with respect to the body region is calculated, for example, the position of the base of the pet's neck in the fourth two-dimensional created image, and the position of the base of the pet's neck in the fourth photograph Of the 4th two-dimensionally created image and the body area of the fourth photograph so that they match, and the area of the torso in the 4th two-dimensionally created image is 100%. The area ratio may be calculated as the similarity. In the present embodiment, the calculation unit 24 evaluated that there are 80 similarities as shown in FIG.

  The calculation unit 24 compares the body recognized from the “fourth photograph,“ the entire picture of the pet viewed from the back direction ”” and the “back body area” in the fourth two-dimensionally created image. Here, for example, similarity determination is performed on the shape of the body, the color of the entire body, the presence / absence / shape of the pattern, and the like. If the similarity of the shape of the torso is calculated, for example, the pet torso in the fourth two-dimensional created image is superimposed on the torso of the pet in the fourth photograph, and the fourth two-dimensional created image When the area of the trunk is 100%, the ratio of the overlapped area may be calculated as the similarity. In the present embodiment, the calculation unit 24 evaluated that there are 80 similarities as shown in FIG.

  In this calculation step S350, an average value of scores related to the similarity calculated in each region is further calculated. Here, the average value when there is no weighting information is 81 points, which is a value obtained by simply dividing the total of 10 areas by 10. However, in the present embodiment, the “front triangular area”, “front other face area”, and “front forefoot area” are weighted, and these points are multiplied by 2 to calculate the total with other areas. , 78.46154, which is a value obtained by dividing the total by 13, is a calculation result in consideration of weighting. A calculation result in consideration of this weighting is a weighted average value.

  In this embodiment, the weighted average value is lower than the simple average value. This is the "front and other face areas" which is important in determining the similarity of the three-dimensional shape model, and the crossing angle information of both forefoot in the sitting posture in which the client has been informed that it is a characteristic part. This means that the weighted average value weighted for these two regions is lower than the simple average value not considering weighting because the score of the “front forefoot region” is low.

(Output step S360)
In the output step S360, the output unit 30 outputs the determination result determined in the calculation step S350. The modeler can access the information processing apparatus 100 from his computer and check the output result of the output unit 30 via the communication unit 70.

  In the present embodiment, 78.46154 points, which are weighted average values, are output as calculated in the calculation step S350.

  Furthermore, in the present embodiment, the output unit 30 outputs a display of a region whose similarity is lower than a predetermined value different from the display of other regions. Here, it is assumed that there are 80 acceptable lines through which the three-dimensional data can be submitted to the client. In the present embodiment, as shown in FIG. 4, since the scores of “front and other face areas” and “front forefoot areas” are less than 80 points that are the acceptable lines, the output unit 30 has three-dimensional data. Above, the two regions are colored in red, or the boundary line between the regions is displayed with a red line and a thick line. Note that the output unit 30 may perform a plurality of coloring according to the score of similarity, for example, blue for an area of 90 points or more and yellow for an area of 80 points or more and less than 90 points. .

  Note that if the area division is not selected in S331, the flow proceeds to S332.

  In S332, it is determined whether or not there is one two-dimensional input image input in the two-dimensional input image input step S311. In the present embodiment, since the two-dimensional input image described above includes four photographs, the flow proceeds to calculation step S350.

  If the region division is not selected in S331, in calculation step S350, the average value of the similarity between each photograph and each two-dimensionally created image determined to have the highest similarity with each photograph is calculated. For example, as shown in FIG. 3, the similarity between the first photograph and the first two-dimensional created image is 75 points, and the similarity between the second photograph and the second two-dimensional created image is 70 points. If the similarity between the third photograph and the third two-dimensional created image is 90 points, and the similarity between the fourth photograph and the fourth two-dimensional created image is 80 points, the computing unit 24 The average value 78.75 is calculated. In the output step S360, the 78.75 points are output.

  If the region division is not selected in S331 and the number of two-dimensional input images input in the two-dimensional input image input step S311 is one, the flow proceeds from S332 to S360. Then, the degree of similarity between the two-dimensional created image extracted in S330 and the two-dimensional input image is output as it is in S360.

(Confirmation step S370)
In the confirmation step S370, the modeler confirms whether or not the three-dimensional data created by the modeler has reached a predetermined value, for example, a passing line where the three-dimensional data can be submitted to the client. In the present embodiment that has undergone the division step S340, there are 80 passing lines that can submit 3D data to the client, whereas the weighted average value indicating the similarity of the 3D data created by the modeler is Since it is 78.46154, the modeler recognizes that the three-dimensional data created by the modeler does not satisfy the above requirement.

  By adopting such an information processing method, it becomes possible for the modeler to recognize that it is necessary to appropriately correct the three-dimensional data until the output result exceeds a predetermined value.

  Further, in the present embodiment, in the output step S360, the output unit 30 performs red coloring on the “front other face area” and the “front forefoot area” that do not reach 80 points that are the pass line, or the area. Since the boundary line is displayed with a red line and a thick line, the modeler can immediately recognize that the two areas are areas to be corrected, and the work efficiency of the modeler can be improved.

  In the present embodiment where the score of the 3D data created by the modeler has not reached the pass line, the process returns to the 3D data input step S312 and the modeler corrects the 3D data and inputs the 3D data again. In the present embodiment, the modeler mainly corrects the “front other face area” and the “front forefoot area”, which have a particularly low similarity evaluation, and the corrected 3D data is input in the three-dimensional data input step S312. The dimension data is input to the input unit 10.

  After that, when the output unit 30 outputs a calculated value exceeding the pass line in the output step S360 through the above-described image generation step S320, extraction step S330, calculation step S350, etc., this flow ends and the requester receives the three-dimensional data. Will be sent. By such an information processing method, the quality of the three-dimensional data sent to the requester can be ensured, and as a result, the number of times the requester makes a correction request can be reduced. That is, it is possible to obtain three-dimensional data having a high degree of similarity with respect to the image information specified by the requester without taking the trouble of the requester.

DESCRIPTION OF SYMBOLS 10 Input part, 20 Image generation part, 22 Extraction part, 24 Calculation part, 26 Dividing part, 30 Output part, 40 Control part, 50 Storage part, 70 Communication part, 100 Information processing apparatus, S310 Input step, S311 Two-dimensional input Image input step, S312 Three-dimensional data input step, S320 Image generation step, S330 extraction step, S340 division step, S350 calculation step, S360 output step, S370 confirmation step.

Claims (11)

An input unit for receiving input of 3D data and a 2D input image related to the 3D model;
Based on the three-dimensional data, an image generation unit that generates one or a plurality of two-dimensional creation images showing an appearance of the three-dimensional model arranged in a virtual space when viewed from a given viewpoint;
An output unit that outputs a similarity between the two-dimensional created image and the two-dimensional input image;
Information processing device. The image generation unit generates a plurality of the two-dimensional creation images,
The output unit outputs the highest similarity among the similarities between each of the plurality of two-dimensional created images and the two-dimensional input image.
The information processing apparatus according to claim 1. The image processing apparatus further includes an extraction unit that calculates a similarity between each of the plurality of two-dimensional created images and the two-dimensional input image, and extracts a two-dimensional created image having the highest similarity from the plurality of two-dimensional created images. ,
The information processing apparatus according to claim 1 or 2. The input unit receives input of a plurality of two-dimensional input images,
The extraction unit extracts, for each of the plurality of two-dimensional input images, a two-dimensional creation image having the highest similarity from the plurality of two-dimensional creation images according to the number of the plurality of two-dimensional input images. ,
A calculation unit that calculates an average value of the similarities between the plurality of two-dimensional created images extracted by the extraction unit and the plurality of two-dimensional input images;
The output unit outputs the average value calculated by the calculation unit.
The information processing apparatus according to claim 3. The average value includes a weighted average value weighted with respect to a two-dimensional input image including a feature portion among the plurality of two-dimensional input images.
The information processing apparatus according to claim 4. Weighting information about whether or not a feature part is included is added to the two-dimensional input image in which the input part receives input.
The information processing apparatus according to claim 5. A division unit that divides the two-dimensionally created image extracted by the extraction unit into a plurality of regions;
The computing unit calculates a similarity between the plurality of regions and the two-dimensional input image;
The output unit makes a display of a region whose similarity is lower than a predetermined value out of the plurality of regions different from the display of other regions.
The information processing apparatus according to claim 3. The output unit displays a color in an area where the similarity is lower than a predetermined value.
The information processing apparatus according to claim 7. The two-dimensional input image is a photograph;
The information processing apparatus according to claim 1. An input step for receiving input of three-dimensional data relating to the three-dimensional model and a two-dimensional input image;
An image generation step of generating a two-dimensional creation image showing an appearance of the three-dimensional model arranged in a virtual space viewed from a given viewpoint based on the three-dimensional data;
Outputting a similarity between the two-dimensional created image and the two-dimensional input image,
Information processing method. On the computer,
An input step for receiving input of three-dimensional data relating to the three-dimensional model and a two-dimensional input image;
An image generation step of generating a two-dimensional creation image showing an appearance of the three-dimensional model arranged in a virtual space viewed from a given viewpoint based on the three-dimensional data;
An output step of outputting a similarity between the two-dimensional created image and the two-dimensional input image;
A program for running

Images