JP6894685B2 - Information processing equipment, information processing methods and programs - Google Patents

Description

The present invention relates to an information processing device, an information processing method and a program.

The following Patent Document 1 discloses a technique for generating a three-dimensional shape model using a plurality of image information or the like designated by a client.

Japanese Unexamined Patent Publication No. 2016-4466

However, in the conventional information processing device, it is necessary to make the client take time and effort. That is, according to the above-mentioned conventional configuration, in order to ensure that the quality of the completed 3D shape model, specifically, the image information specified by the client and the 3D shape model are similar, in advance. It was necessary to communicate with the requester, such as sending 3D data to the requester and receiving a correction request if they were not similar, which caused the requester to take time and effort.

The present disclosure has been made in view of the above problems, and the purpose of the present disclosure is to obtain three-dimensional data having a high degree of similarity to the image information specified by the requester without causing the requester to take time and effort. That is.

(1) The information processing apparatus according to the present disclosure includes an input unit that receives input of 3D data and a 2D input image related to the 3D model, and the 3D model arranged in a virtual space based on the 3D data. Includes an image generation unit that generates a two-dimensionally created image showing the appearance seen from a given viewpoint, and an output unit that outputs the similarity between the two-dimensionally created image and the two-dimensional input image.

(2) In the information processing apparatus of (1) above, the image generation unit generates a plurality of two-dimensional created images showing the appearance of the three-dimensional model arranged in the virtual space as viewed from a plurality of viewpoints. The output unit may output the highest degree of similarity between each of the plurality of two-dimensional created images and the two-dimensional input image.

(3) In the information processing apparatus of (1) or (2), the similarity between each of the plurality of two-dimensional created images and the two-dimensional input image is calculated, and the two-dimensional created image having the highest degree of similarity is obtained. An extraction unit for extracting from the plurality of two-dimensionally created images may be further provided.

(4) In the information processing apparatus of (3) above, the input unit receives inputs of a plurality of two-dimensional input images, and the extraction unit receives the plurality of two-dimensional inputs for each of the plurality of two-dimensional input images. The two-dimensional created image having the highest degree of similarity is extracted from the created images according to the number of the plurality of two-dimensional input images, and the plurality of two-dimensional created images extracted by the extraction unit and the plurality of two-dimensional images are extracted. The output unit may further include a calculation unit that calculates the average value of each similarity with the input image, and the output unit may output the average value calculated by the calculation unit.

(5) In the information processing apparatus of (4) above, the average value may include a weighted average value weighted with respect to the two-dimensional input image including the feature portion among the plurality of two-dimensional input images.

(6) In the information processing apparatus of (5) above, weighting information as to whether or not a feature portion is included may be added to the two-dimensional input image that the input unit receives an input.

(7) In the information processing apparatus of (3) to (6), the two-dimensionally created image extracted by the extraction unit is further divided into a plurality of regions, and the calculation unit includes the plurality of division units. The similarity with the two-dimensional input image with respect to the region is calculated, and the output unit makes the display of the region having the similarity lower than the predetermined value different from the display of the other regions among the plurality of regions. May be good.

(8) In the information processing apparatus of (7) above, the output unit may display a region in which the similarity is lower than a predetermined value by coloring.

(9) In the information processing devices (1) to (8), the two-dimensional input image may be a photograph.

(10) The information processing method according to the present disclosure includes an input step for receiving input of 3D data and a 2D input image related to the 3D model, and the 3D model arranged in a virtual space based on the 3D data. Includes an image generation step of generating a two-dimensionally created image showing the appearance seen from a given viewpoint, and an output step of outputting the similarity between the two-dimensionally created image and the two-dimensional input image.

(11) The program according to the present disclosure includes an input step for receiving input of 3D data and a 2D input image related to a 3D model to a computer, and the 3D arranged in a virtual space based on the 3D data. An image generation step of generating a two-dimensional created image showing the appearance of the model as viewed from a given viewpoint and an output step of outputting the similarity between the two-dimensional created image and the two-dimensional input image are executed.

FIG. 1 is a schematic view of an information processing device according to the present embodiment. FIG. 2 is a flowchart of an information processing method according to the present embodiment. FIG. 3 is a diagram showing a concept of calculation steps in the information processing method according to the present embodiment. FIG. 4 is a diagram showing a concept of calculation steps in the information processing method according to the present embodiment.

[First Embodiment]
The embodiments of the present disclosure will be described below with reference to the drawings.

FIG. 1 is a schematic view of an information processing device according to the present embodiment.

As shown in FIG. 1, the information processing device 100 is a server device, and controls the requester's computer, the communication unit 70 that transmits and receives data to and from the computer of the modeler that creates three-dimensional data, and the entire information processing device 100. A control unit 40 for information processing and a storage unit 50 for storing a program or the like used for the control are included. In the present disclosure, the three-dimensional data includes, for example, three-dimensional CAD (computer-aided design) data, three-dimensional CG (computer graphics) data, and the like.

(Communication unit 70)
The communication unit 70 transmits / receives data to / from the computer of the client and the modeler by wire or wirelessly. The communication unit 70 communicates directly or indirectly with an external device using, for example, a LAN (Local Area Network), Wi-Fi (Wireless Fidelity, registered trademark), a telephone line, a broadband network, or the like.

(Control unit 40)
The control unit 40 controls the overall operation of the information processing device 100 according to various programs stored in the storage unit 50. The control unit 40 is realized by an electronic circuit such as a CPU (Central Processing Unit) or a microprocessor. The control unit 40 may include a ROM (Read Only Memory) for storing programs to be used, calculation parameters, and the like, and a RAM (Random Access Memory) for temporarily storing parameters and the like that change as appropriate.

The control unit 40 includes an input unit 10 that accepts input of 3D data and a 2D input image, an image generation unit 20 that generates a 2D created image based on the 3D data, and an output unit 30 that outputs similarity. And include. Further, in the present embodiment, the control unit 40 extracts a two-dimensional created image having the highest degree of similarity to the two-dimensional input image from the plurality of two-dimensional created images, and a plurality of two-dimensional created images. A division unit 26 for dividing into the regions of the above, and a calculation unit 24 for calculating the average value of the degree of similarity between the plurality of two-dimensional created images and the plurality of two-dimensional input images.

(Input unit 10)
The input unit 10 accepts the input of the two-dimensional input image from the requester who requests the production of the three-dimensional shape model. Here, the two-dimensional input image is, for example, a photograph of the client's pet. In this embodiment, an example in which the client requests the production of a three-dimensional shape model of a pet will be described. The input of the two-dimensional input image is performed by the requester accessing the information processing device 100, which is a server device, and inputting the two-dimensional input image to the input unit 10 via the communication unit 70 described above. It is also possible for the user of the information processing apparatus 100 to directly input the two-dimensional input image obtained from the requester via an e-mail, a recording medium, or the like to the input unit 10 without going through the communication unit 70. .. The two-dimensional input image input to the input unit 10 is stored in the storage unit 50.

The input unit 10 receives input of 3D data indicating a 3D model created by the modeler based on the 2D input image from the client. The input of the three-dimensional data is performed by the modeler accessing the information processing device 100, which is a server device, and inputting the three-dimensional data to the input unit 10 via the communication unit 70 described above. It is also possible for the user of the information processing apparatus 100 to directly input the three-dimensional data obtained from the modeler via an e-mail, a recording medium, or the like into the input unit 10 without going through the communication unit 70. The three-dimensional data input to the input unit 10 is stored in the storage unit 50.

(Image generation unit 20)
The image generation unit 20 generates a two-dimensional image that shows the appearance of the three-dimensional model arranged in the virtual space as viewed from a given viewpoint, based on the three-dimensional data created by the modeler.

As a first specific example, a case where the two-dimensional input image input by the client is one photograph and is a face photograph of a pet seen from the front direction will be described. The image generation unit 20 estimates the camera position from the input facial photograph. As a method of estimating the camera position, for example, the image generation unit 20 extracts edge information in the projected image of the 3D model projected on the projection plane arranged between the projection center of the virtual space and the 3D model. By doing so, the position where the contour shape of the pet in the two-dimensional input image input by the client and the contour shape of the three-dimensional model in the projected image have the highest ratio of matching is estimated as the camera position. Next, the position of the 3D model with respect to the projection center in the virtual space is adjusted based on the estimated camera position information. Then, the image generation unit 20 acquires a projected image of the three-dimensional model projected on the projection plane arranged between the projection center of the virtual space and the three-dimensional model as a two-dimensional created image. Further, the image generation unit 20 acquires a plurality of two-dimensional created images projected on the projection surface while rotating the three-dimensional model arranged in the virtual space, for example, once. In this specific example, as an appearance close to the pet's face photograph input by the client, a plurality of two-dimensional created images showing the appearance of the pet's face viewed from a plurality of angles in the vicinity of the front surface are generated.

As a second specific example, the two-dimensional input image input by the client is a plurality of photographs, for example, the first photograph is "a photo of a pet's face seen from the front direction" and the second picture is "from the front direction". When the third image is the "overall photograph of the pet seen from the side" and the fourth image is the "overall photograph of the pet seen from the back", the image generation unit 20 is as follows. A plurality of two-dimensionally created image groups are generated as described above.

The image generation unit 20 adjusts the positional relationship between the projection center and the three-dimensional model in the virtual space based on the camera position information estimated from the first "face photograph of the pet seen from the front direction". For example, in the virtual space, the face area of the three-dimensional model is arranged at a position close to the viewpoint of the camera, which is the center of projection. Then, the image generation unit 20 rotates the three-dimensional model by 1 degree from the front direction to generate a first two-dimensional created image group showing the appearance of the pet's face when viewed from a plurality of angles in the vicinity of the front.

The image generation unit 20 adjusts the positional relationship between the projection center and the three-dimensional model in the virtual space based on the camera position information estimated from the second “overall photograph of the pet viewed from the front direction”. For example, in the virtual space, the viewpoint of the camera, which is the center of projection, is adjusted to a position that captures the entire front surface of the three-dimensional model. Then, the image generation unit 20 rotates the three-dimensional model by 1 degree from the front direction to generate a second two-dimensional created image group showing the overall appearance of the pet when viewed from a plurality of angles in the vicinity of the front.

The image generation unit 20 adjusts the positional relationship between the projection center and the three-dimensional model in the virtual space based on the camera position information estimated from the third "overall photograph of the pet viewed from the side direction". For example, in the virtual space, the viewpoint of the camera, which is the center of projection, is adjusted to a position that captures the entire side surface of the three-dimensional model. Then, the image generation unit 20 rotates the three-dimensional model by one degree from the side surface direction to generate a third two-dimensionally created image group showing the overall appearance of the pet when viewed from a plurality of angles in the vicinity of the side surface.

The image generation unit 20 adjusts the positional relationship between the projection center and the three-dimensional model in the virtual space based on the camera position information estimated from the fourth "overall photograph of the pet seen from the back direction". For example, in the virtual space, the viewpoint of the camera, which is the center of projection, is adjusted to a position that captures the entire back surface of the three-dimensional model. Then, the image generation unit 20 rotates the three-dimensional model by 1 degree from the back surface direction to generate a fourth two-dimensionally created image group showing the overall appearance of the pet as viewed from a plurality of angles in the vicinity of the back surface.

In the present embodiment, an example in which the camera position is estimated and then the three-dimensional model is rotated to acquire a plurality of two-dimensional created images has been described, but the step of estimating the camera position is omitted and the three-dimensional model is obtained. The model is rotated 360 degrees one by one to generate 360 two-dimensional created image groups, and in the processing of the extraction unit 22 described later, a configuration having a high degree of similarity is extracted from the 360 two-dimensional created image groups. I do not care.

Further, in the present embodiment, an example in which the camera position is estimated and then the three-dimensional model is rotated to acquire a plurality of two-dimensional created images has been described. However, one two-dimensional created image is obtained from the estimated camera position. May be configured to output the similarity of this one two-dimensionally created image in the processing of the output unit 30 described later.

(Extractor 22)
When a plurality of two-dimensional created image groups are acquired in the image generation unit 20, the extraction unit 22 calculates the similarity between the two-dimensional created image group and the two-dimensional input image, and the two-dimensional created image having the highest degree of similarity is calculated. Is extracted from the two-dimensionally created image group. When the image generation unit 20 has acquired one 2D image group, the extraction unit 22 only calculates the degree of similarity between the 2D image group and the 2D input image.

A specific example will be described when a plurality of two-dimensionally created image groups are acquired in the image generation unit 20. For all the two-dimensionally created images of the first two-dimensionally created image group generated by the image generation unit 20, the degree of similarity with the first photograph, "the face photograph of the pet seen from the front direction" is calculated. Here, the similarity is, for example, the size of the pet's eyes, the length of the ears, the length of the nose, the distance between the eyes, the angle of each vertex in the triangle connecting the positions of both eyes and the position of the nose, and the like. Calculate by using the degree of matching as a judgment item. As a method of calculating the similarity, for example, it is possible to calculate using a classification algorithm such as machine learning including reinforcement learning and deep learning, and the similarity is calculated by a simpler algorithm. You may. As a method of calculating the similarity with a simple algorithm, for example, if the similarity of the size of the pet's eyes is calculated, the area of the pet's eyes in the two-dimensional image and the area of the pet's eyes in the photograph are used. Of these, the value obtained by dividing the larger one by the smaller one may be calculated as the similarity. If you want to calculate the similarity of pet ear or nose length, start with the pet ear in the two-dimensional image, or the pet ear in the nose length and the pet ear in the photograph, or the largest of the nose lengths. The value obtained by dividing the smaller one may be calculated as the similarity. If the similarity between the eyes is calculated, for example, the value obtained by dividing the distance between the pet's eyes in the two-dimensional image and the distance between the pet's eyes in the photograph by dividing the larger one from the smaller one is calculated as the similarity. You may. If the similarity of the triangle connecting the positions of both eyes and the position of the nose is to be calculated, for example, the triangle in the two-dimensional image and the triangle in the photograph are superimposed, and the area of the triangle in the two-dimensional image is 100. The ratio of the area of the overlapped portion in the case of% may be calculated as the degree of similarity. Then, the first two-dimensionally created image having the highest degree of similarity is extracted by the extraction unit 22 from the first two-dimensionally created image group.

For all the two-dimensionally created images of the second two-dimensionally created image group generated by the image generation unit 20, the degree of similarity with the second photograph, "the whole photograph of the pet seen from the front direction" is calculated. Then, the second two-dimensionally created image having the highest degree of similarity is extracted by the extraction unit 22 from the second two-dimensionally created image group.

For all the two-dimensionally created images of the third two-dimensionally created image group generated by the image generation unit 20, the degree of similarity with the third photograph, "the whole picture of the pet seen from the side direction" is calculated. Then, from the third two-dimensional created image group, the third two-dimensional created image having the highest degree of similarity is extracted by the extraction unit 22.

For all the two-dimensionally created images of the fourth two-dimensionally created image group generated by the image generation unit 20, the degree of similarity with the fourth photograph, "the whole picture of the pet seen from the back direction" is calculated. Then, the fourth two-dimensionally created image having the highest degree of similarity is extracted by the extraction unit 22 from the fourth two-dimensionally created image group.

(Calculation unit 24)
The calculation unit 24 calculates the average value of the degree of similarity between the plurality of two-dimensional created images extracted by the extraction unit 22 and the plurality of two-dimensional input images.

In the present embodiment, as shown in FIG. 3, the similarity between the first photograph and the first two-dimensional created image is 75 points, and the similarity between the second photograph and the second two-dimensional created image is 75 points. Assuming that the degree is 70 points, the similarity between the third photograph and the third two-dimensional image is 90 points, and the similarity between the fourth photograph and the fourth two-dimensional image is 80 points. The average value of 78.75 is calculated.

The average value may be a weighted average value weighted with respect to the two-dimensional input image including the feature portion among the plurality of two-dimensional input images. That is, when the calculation unit 24 calculates the average value of the similarity between the plurality of two-dimensional created images and the plurality of two-dimensional input images, the plurality of two-dimensional input images include the feature unit. It may be configured to calculate the average value by weighting. For example, whether or not the shape of the characteristic part of the pet, such as the pet's face seen from the front and the skeleton shape when the pet takes a unique pose, resembles the pet itself in the finally created three-dimensional shape model. It may be important in determining whether or not. In such a case, weighting information is added to the photograph of the pet's face in the front direction and the photograph in which the skeleton shape of the entire posing pet can be recognized. By doing so, the calculated value of the similarity of the photograph including the feature part has an influence on the calculated value of the overall similarity, and the calculated value of the similarity of other photographs becomes the calculated value of the overall similarity. It can be greater than the degree of influence.

It should be noted that the weighting information regarding whether or not the feature portion is included may be added to the two-dimensional input image when the input unit 10 accepts the input of the two-dimensional input image.

(Output unit 30)
When a plurality of two-dimensional created image groups are acquired by the image generation unit 20, the output unit 30 outputs the similarity of the two-dimensional created images determined by the extraction unit 22 to have the highest similarity. When the two-dimensional input image is a plurality of two-dimensional input images, the average value calculated by the calculation unit 24 is output. The modeler can confirm the output result of the output unit 30 via the communication unit 70. When one image group of two-dimensionally created images is acquired by the image generation unit 20, the output unit 30 outputs the similarity of the two-dimensionally created images calculated by the extraction unit 22.

With such a configuration, if the output result does not exceed the predetermined value, the modeler can be urged to appropriately modify the three-dimensional data until the output result exceeds the predetermined value. It will be possible. As a result, the quality of the three-dimensional data sent to the requester can be guaranteed, and the number of times the requester makes a correction request can be reduced, so that the time and effort can be reduced for the requester.

(Division 26)
Further, the control unit 40 may be configured to further include the division unit 26 as shown in FIG. The division unit 26 divides the two-dimensionally created image extracted by the extraction unit 22 into a plurality of regions. For example, the dividing portion 26 divides into a triangular region connecting both eyes and the nose of the pet in the two-dimensional image, other face regions, forefoot regions, hindfoot regions, side surface regions of the torso, back regions of the torso, and the like. With such a configuration, in the output unit 30 described later, it is possible to make the display of a region having a similarity lower than a predetermined value different from the display of another region.

Further, the control unit 40 includes the division unit 26, the calculation unit 24 calculates the similarity of each region divided by the division unit 26, and the output unit 30 displays the region whose similarity is lower than a predetermined value. Is output differently from the display of other areas, it is easy for the modeler to recognize the points to be corrected, and the work efficiency of the modeler can be improved. The algorithm for calculating the similarity by the calculation unit 24 may be the same as the algorithm for calculating the similarity by the extraction unit 22.

As a specific example in which the output unit 30 makes the display of a region having a similarity lower than a predetermined value different from the display of another region, for example, a region having a similarity lower than a predetermined value is colored in red or the like. Alternatively, the boundary line of the area is displayed with a red line or a thick line, and so on. Alternatively, the output unit 30 sets the output unit 30 to, for example, blue in the high-scoring area, yellow in the area reaching the predetermined value but not reaching the high-scoring value, and the predetermined value, depending on the score of the similarity. Multiple colors such as red may be applied to the unreachable area.

(Memory unit 50)
The storage unit 50 records and reproduces data on a predetermined recording medium. The storage unit 50 is realized as, for example, an HDD (Hard Disc Drive). Various types of recording media such as a solid-state memory such as a flash memory, a memory card with a built-in solid-state memory, an optical disk, a magneto-optical disk, and a hologram memory can be considered, and the storage unit 50 can perform recording / playback according to the recording medium to be adopted. It may be configured. The storage unit 50 stores a program used by the control unit 40.

The storage unit 50 can store the two-dimensional input image and the three-dimensional data input to the input unit 10 according to the instruction of the control unit 40, and the storage unit 50 can store the two-dimensional input image and the three-dimensional data according to the instruction of the control unit 40. It is possible to have the control unit 40 obtain the input image and the three-dimensional data.

(Information processing method)
Hereinafter, the information processing method will be described with reference to FIG.

FIG. 2 is a flowchart showing an example of the price estimation method according to the present embodiment.

(Input step S310)
As shown in FIG. 2, in the input step S310, the input unit 10 includes a two-dimensional input image input step S311 that accepts the input of the two-dimensional input image and a three-dimensional data input step S312 that accepts the input of the three-dimensional data. Including.

(Two-dimensional input image input step S311)
In the two-dimensional input image input step, the input unit 10 receives a two-dimensional input image from a requester who requests the production of a three-dimensional shape model. To input the two-dimensional input image, for example, the client accesses the information processing device 100, which is a server device, from his / her own computer, and inputs the two-dimensional input image to the input unit 10 via the communication unit 70 described above. Is done by. Alternatively, the user of the information processing apparatus 100 inputs a two-dimensional input image obtained from the requester via an e-mail, a recording medium, or the like into the input unit 10. The two-dimensional input image input to the input unit 10 is stored in the storage unit 50.

In the present embodiment, the two-dimensional input images input by the client are a plurality of photographs, the first is a "frontal view of the pet's face" and the second is the "frontward view of the pet". It is assumed that the third photo is the "whole photo of the pet seen from the side" and the fourth is the "whole photo of the pet seen from the back". All of these "whole pictures of pets" are in a sitting position. In addition, the client has informed that the crossing angle of both front legs in the sitting posture is characteristic. The crossing angle of both front legs is clarified from the second "overall photograph of the pet seen from the front". Therefore, weighting information to the effect that the photograph includes a feature portion is added to the second "overall photograph of the pet seen from the front direction" requested by the client. Further, the same weighting information is added to the first "face photograph of the pet seen from the front direction", which generally has a great influence on the similarity of the three-dimensional shape model.

(3D data input step S312)
In the three-dimensional data input step, the modeler creates three-dimensional data based on the two-dimensional input image from the client, and the modeler or the user of the information processing apparatus 100 inputs the three-dimensional data to the input unit 10.

The modeler can access the information processing device 100 from his / her own computer, for example, and confirm the two-dimensional input image stored in the storage unit 50 via the communication unit 70. The modeler creates 3D data based on this 2D input image. The modeler accesses the information processing device 100 from his / her own computer again, and stores the three-dimensional data in the storage unit 50 via the communication unit 70. Alternatively, the user of the information processing apparatus 100 inputs the three-dimensional data obtained from the modeler via mail, a recording medium, or the like into the input unit 10, and stores the three-dimensional data in the storage unit 50.

(Image generation step S320)
As shown in FIG. 3, when the input step S310 is completed, the image generation step S320 can be started. The start timing of the image generation step S320 may be determined by the user of the information processing apparatus 100 or by the modeler.

In the image generation step S320, a two-dimensional created image showing the appearance of the three-dimensional model arranged in the virtual space as viewed from a given viewpoint is generated based on the three-dimensional data input to the information processing apparatus 100 in the input step S310. Generate.

In the present embodiment, the two-dimensional input image input by the client is a photograph of four pets, for example, the first image is a "face photograph of the pet seen from the front direction" and the second image is a "front direction". It is assumed that the third piece is the "whole picture of the pet seen from the side" and the fourth piece is the "whole picture of the pet seen from the back side".

First, the image generation unit 20 estimates the camera position from the input facial photograph, and adjusts the positions of the three-dimensional model and the projection center in the virtual space based on the estimated camera position information. Then, in the present embodiment, the image generation unit 20 displays the projected image of the three-dimensional model on the projection surface while rotating the three-dimensional model arranged in the virtual space by one degree, and is input by the client. Generates a two-dimensional created image group showing an appearance close to that of a pet's face photograph.

In this image generation step S320, the step of estimating the camera position is omitted, and for example, the image generation unit 20 rotates the three-dimensional model by 360 degrees once to generate 360 two-dimensional created image groups. In the extraction step S330, which will be described later, a configuration in which images having a high degree of similarity may be extracted from a group of 360 two-dimensional images may be used.

In the image generation step S320, the image generation unit 20 does not acquire a plurality of two-dimensional created images, but acquires one two-dimensional created image from the estimated camera position, and in the extraction step S330 described later. The similarity between the one two-dimensional created image and the two-dimensional input image may be calculated, and the calculated similarity may be output in the output step S360 described later.

In the present embodiment, the image generation unit 20 generates a first two-dimensional image group for the first "frontal view of the pet's face photograph" and generates the second "frontal direction". A second two-dimensional image group is generated for the "whole photo of the pet seen from the side", and a third two-dimensional image is generated for the third "whole photo of the pet seen from the side". A group is generated, and a fourth two-dimensional image group is generated for the fourth "overall photograph of the pet seen from the back direction". The method of generating each two-dimensionally created image group is as described above in the above (image generation unit 20).

(Extraction step S330)
In the extraction step S330, the extraction unit 22 included in the control unit 40 calculates the similarity between the two-dimensional created image group and the two-dimensional input image, and the two-dimensional created image having the highest similarity is the two-dimensional created image. Extract from the group.

In the present embodiment, the first two-dimensional created image having the highest degree of similarity to the first photograph is extracted from the first two-dimensional created image group, and the second two-dimensional created image group is included. From, the second two-dimensionally created image having the highest degree of similarity with the second photograph is extracted, and from the third two-dimensionally created image group, the third image having the highest degree of similarity with the third photograph is taken. The two-dimensionally created image is extracted, and the fourth two-dimensionally created image having the highest degree of similarity to the fourth photograph is extracted from the fourth two-dimensionally created image group.

Next, in S331, it is determined whether or not to divide the area. In the present embodiment, the area division is selected, and the flow proceeds to the division step S340.

(Division step S340)
In the present embodiment, since the flow proceeds to the division step S340, the division unit 26 shown in FIG. 1 divides the two-dimensional created image extracted by the extraction unit 22 into a plurality of regions. For example, the dividing portion 26 creates a first two-dimensional image that is judged to have the highest degree of similarity to the first "face photograph of the pet seen from the front", and is a triangle connecting both eyes and the mouth of the pet. It is divided into a "front triangular area" which is an area and a "front other face area" which is a face area other than the triangular area. In addition, the division portion 26 sets the second two-dimensionally created image, which is judged to have the highest degree of similarity to the second "overall photograph of the pet seen from the front direction", into the "front fuselage region" and the "front". Divide into "forefoot area". In addition, the division portion 26 sets the third two-dimensionally created image, which is judged to have the highest degree of similarity to the third "whole photograph of the pet viewed from the side", as the "side head region" and ". It is divided into a "side body area", a "side forefoot area", and a "side hindfoot area". Further, the division portion 26 sets the fourth two-dimensionally created image, which is judged to have the highest degree of similarity to the fourth "overall photograph of the pet seen from the back direction", as the "back head region" and ". Divide into "back fuselage area".

(Calculation step S350)
In the calculation step S350, the calculation unit 24 calculates the average value of the degree of similarity between the plurality of two-dimensional created images extracted by the extraction unit 22 and the plurality of two-dimensional input images.

FIG. 4 is a diagram showing a concept of calculation steps in the information processing method according to the present embodiment.

The calculation unit 24 includes a triangular region connecting both eyes and the mouth of the pet recognized from the first photograph "a photo of the pet's face viewed from the front" and a triangular region in the first two-dimensionally created image. A comparison is made with a certain "front triangular area", and the degree of matching is displayed as a score. In the present embodiment, as shown in FIG. 4, the calculation unit 24 evaluated that the degree of coincidence of each angle was 80% and that there was a degree of similarity of 80 points.

The calculation unit 24 compares the first photograph, the "face photograph of the pet seen from the front direction", with the "front and other face areas" in the first two-dimensionally created image. Here, since the similarity has already been determined in the triangular region connecting both eyes and the mouth of the pet, the similarity is determined here for the face region other than the triangular region. For example, the arrangement of the mouth and ears on the entire face, the shape of the mouth, the shape of the ears, the ratio of the length of a specific part on the face to the length of another specific part, the color of the entire face, the presence / absence / shape of a pattern, etc. Judgment is made. If you want to calculate the similarity of the arrangement of the mouth and ears on the entire face, for example, the triangle connecting the mouth and both ears of the pet's face in the first two-dimensional image and the pet's face in the first photograph. Even if the ratio of the area of the overlapped part is calculated as the degree of similarity when the area of the triangle in the first two-dimensional image is 100% by superimposing the triangle connecting the mouth and both ears. Good. If the similarity of the shape of the mouth (or ear) is to be calculated, the pet's mouth (or ear) in the first two-dimensional image and the pet's mouth (or ear) in the first photograph are overlapped. In addition, when the area of the mouth (or ear) in the first two-dimensional image is 100%, the ratio of the area of the overlapped portion may be calculated as the degree of similarity. If you want to calculate the similarity between the length of a specific part of the face and the length of another specific part, the larger one of the ratio in the first two-dimensional image and the ratio in the first photograph. The value obtained by dividing the smaller one from the above may be calculated as the similarity. In the present embodiment, the calculation unit 24 evaluated that there are 70 points of similarity as shown in FIG.

As described in the input step S310, the face region viewed from the front direction is generally an important region for determining the similarity of the three-dimensional shape model, and thus is the first photograph. Weighting information is added to the "photograph of the pet's face viewed from the front". Therefore, as shown in FIG. 4, weighting is performed on the "front triangular region" and the "front other face region" to be compared with the "pet face photograph viewed from the front direction".

The calculation unit 24 compares the fuselage recognized from the second photograph "the entire photograph of the pet seen from the front direction" with the "front fuselage region" in the second two-dimensionally created image. Here, for example, the similarity is determined with respect to the shape and aspect ratio of the body, the color of the entire body, the presence / absence / shape of a pattern, and the like. If the similarity of the shapes of the torso is to be calculated, for example, the torso of the pet in the second two-dimensional created image and the torso of the pet in the second photograph are superimposed, and in the second two-dimensional created image. When the area of the body is 100%, the ratio of the areas of the overlapped portions may be calculated as the degree of similarity. In the present embodiment, the calculation unit 24 evaluated that there is a degree of similarity of 80 points as shown in FIG.

The calculation unit 24 compares the forefoot recognized from the second photograph, the "whole photograph of the pet viewed from the front", with the "front forefoot region" in the second two-dimensional image. Here, for example, the degree of similarity is determined with respect to the positions of both forefoot with respect to the torso region, the intersection angle of both forefoot, the color of the entire forefoot, the presence / absence / shape of the pattern, and the like. If you want to calculate the similarity between the positions of both forefoot and the intersection angle of both forefoot, for example, the triangle connecting the base of both forefoot and the intersection of both forefoot of the pet in the second two-dimensional image, and the second piece. The area of the overlapped part when the triangle connecting the bases of both forefoot and the intersection of both feet of the pet in the photograph is overlapped and the area of the triangle in the second two-dimensional image is set to 100%. The ratio of may be calculated as the degree of similarity. In the present embodiment, the calculation unit 24 evaluated that there are 60 points of similarity as shown in FIG.

As explained in the input step S310, the client has been informed that the crossing angle of both forefoot in the sitting posture is characteristic, and the "pet seen from the front direction" including the crossing angle information of the forefoot. Weighting information is added to the "whole picture of". Both the "front torso area" and the "front forefoot area" to be compared with this second photograph may be weighted, but in the present embodiment, from the content of the communication from the above client, " Only the "intersection angle of both forefoot" is regarded as a feature part, and weighting is performed only in the "front forefoot region" without weighting in the "front torso region".

The calculation unit 24 compares the head recognized from the third photograph, the “whole photograph of the pet seen from the side direction”, with the “side head region” in the third two-dimensionally created image. Here, for example, the similarity is judged for the shape of the head, the positions of the eyes, ears, nose, and mouth in the head area, the position of the head area with respect to the body area, the color of the entire head, and the presence / absence / shape of the pattern. Be told. If you want to calculate the similarity of the positions of the eyes, ears, nose, and mouth in the head area, for example, the square connecting the eyes, ears, nose, and mouth of the pet in the third two-dimensional image, and the third image. The ratio of the area of the overlapped part when the area of the quadrangle connecting the eyes, ears, nose, and mouth of the pet in the above photograph is overlapped and the area of the quadrangle in the third two-dimensional image is 100%. It may be calculated as the degree of similarity. In the present embodiment, the calculation unit 24 evaluated that there is a degree of similarity of 90 points as shown in FIG.

The calculation unit 24 compares the body recognized from the third photograph, the “whole photograph of the pet viewed from the side”, with the “side body area” in the third two-dimensionally created image. Here, for example, the similarity is determined with respect to the shape of the body, the color of the entire body, the presence / absence / shape of a pattern, and the like. If the similarity of the shapes of the torso is to be calculated, for example, the torso of the pet in the third two-dimensional image and the torso of the pet in the third photograph are superimposed, and in the third two-dimensional image. When the area of the body is 100%, the ratio of the areas of the overlapped portions may be calculated as the degree of similarity. In the present embodiment, the calculation unit 24 evaluated that there is a degree of similarity of 90 points as shown in FIG.

The calculation unit 24 compares the forefoot recognized from the third photograph, the “whole photograph of the pet viewed from the side”, with the “side forefoot region” in the third two-dimensional image. Here, for example, the similarity is determined with respect to the shape of the forefoot, the position of the forefoot with respect to the torso region, the color of the entire forefoot, the presence / absence / shape of the pattern, and the like. If the similarity of the shapes of the forefoot is to be calculated, for example, the forefoot of the pet in the third two-dimensional image and the forefoot of the pet in the third photograph are overlapped to form the third two-dimensional image. When the area of the forefoot is 100%, the ratio of the area of the overlapped portion may be calculated as the degree of similarity. If the similarity of the position of the forefoot with respect to the body area is calculated, for example, the position of the base of the pet's forefoot in the third two-dimensional image and the position of the base of the pet's forefoot in the third photograph match. As such, when both body regions are overlapped and the area of the body in the third two-dimensional image is 100%, the ratio of the area of the overlapped portion may be calculated as the degree of similarity. In the present embodiment, the calculation unit 24 evaluated that there is a degree of similarity of 90 points as shown in FIG.

The calculation unit 24 compares the hind paw recognized from the third photograph, the “whole photograph of the pet seen from the side direction”, with the “side hind paw region” in the third two-dimensionally created image. Here, for example, the similarity is determined with respect to the shape of the hind paw, the position of the hind paw with respect to the body region, the color of the entire hind paw, the presence / absence / shape of the pattern, and the like. If the similarity of the shape of the hind paw is to be calculated, for example, the hind paw of the pet in the third two-dimensional image and the hind paw of the pet in the third photograph are overlapped to form the third two-dimensional image. When the area of the hind legs in the created image is 100%, the ratio of the areas of the overlapped portions may be calculated as the degree of similarity. If you want to calculate the similarity of the position of the hind legs to the torso region, for example, the position of the base of the hind legs of the pet in the third two-dimensional image and the base of the hind legs of the pet in the third photograph. The body areas of the third two-dimensional image and the third photograph were overlapped so as to match the positions, and the areas of the body in the third two-dimensional image were set to 100%. The ratio of the area of the portion may be calculated as the degree of similarity. In the present embodiment, the calculation unit 24 evaluated that there is a degree of similarity of 90 points as shown in FIG.

The calculation unit 24 compares the head recognized from the fourth photograph, the "whole photograph of the pet seen from the back direction", with the "back head area" in the fourth two-dimensionally created image. Here, for example, the similarity is determined with respect to the shape of the head, the position of the head region with respect to the body region, the color of the entire head, the presence / absence / shape of a pattern, and the like. If the similarity of the shape of the head is to be calculated, for example, the head of the pet in the fourth two-dimensional image and the head of the pet in the fourth photograph are overlapped to form the fourth two-dimensional image. When the area of the head in the created image is 100%, the ratio of the area of the overlapped portion may be calculated as the degree of similarity. If the similarity of the position of the head with respect to the body region is to be calculated, for example, the position of the base of the pet's neck in the fourth two-dimensional image and the position of the base of the pet's neck in the fourth photograph. When the body area of the 4th 2D image and the body area of the 4th photograph are overlapped and the area of the body in the 4th 2D image is 100%, the overlapped portion The ratio of the area may be calculated as the degree of similarity. In the present embodiment, the calculation unit 24 evaluated that there is a degree of similarity of 80 points as shown in FIG.

The calculation unit 24 compares the fuselage recognized from the fourth photograph, the “whole photograph of the pet seen from the rearward direction”, with the “back fuselage region” in the fourth two-dimensionally created image. Here, for example, the similarity is determined with respect to the shape of the body, the color of the entire body, the presence / absence / shape of a pattern, and the like. If the similarity of the shapes of the torso is to be calculated, for example, the torso of the pet in the fourth two-dimensional image and the torso of the pet in the fourth photograph are superimposed, and in the fourth two-dimensional image. When the area of the body is 100%, the ratio of the areas of the overlapped portions may be calculated as the degree of similarity. In the present embodiment, the calculation unit 24 evaluated that there is a degree of similarity of 80 points as shown in FIG.

In this calculation step S350, the average value of the points related to the similarity calculated in each area is further calculated. Here, as the average value when there is no weighting information, 81 points, which is a value obtained by simply dividing the total of 10 regions by 10, is the calculation result. However, in the present embodiment, the "front triangular region", "front other face region", and "front forefoot region" are weighted, and these points are multiplied by 2 to calculate the total with other regions. , 78.46154, which is the value obtained by dividing the total by 13, is the calculation result in consideration of weighting. The calculation result considering this weighting is the weighted average value.

In the present embodiment, the weighted average value is lower than the simple average value. This is the "front and other face areas" that are important in determining the similarity of the 3D shape model, and the crossing angle information of both forefoot in the sitting posture that the client has informed that it is a feature part. Since the score of the included "front forefoot region" was low, it means that the weighted average value for these two regions was lower than the simple average value without considering the weighting.

(Output step S360)
In the output step S360, the output unit 30 outputs the determination result determined in the calculation step S350. The modeler can access the information processing device 100 from his / her own computer and check the output result of the output unit 30 via the communication unit 70.

In the present embodiment, 78.46154 points, which are weighted average values, are output as calculated in the above calculation step S350.

Further, in the present embodiment, the output unit 30 outputs a display of a region having a similarity lower than a predetermined value different from the display of another region. Here, it is assumed that the pass line at which the three-dimensional data can be submitted to the client is 80 points. In the present embodiment, as shown in FIG. 4, since the scores of the "front other face area" and the "front forefoot area" are lower than the 80 points which are the pass lines, the output unit 30 has three-dimensional data. In the above, the above two areas are colored in red, or the boundary line of the area is indicated by a red line and a thick line. The output unit 30 may perform a plurality of colorings, for example, blue in a region of 90 points or more and yellow in a region of 80 points or more and less than 90 points, depending on the score of similarity. ..

If the area division is not selected in S331, the flow proceeds to S332.

In S332, it is determined whether or not the two-dimensional input image input in the two-dimensional input image input step S311 is one. In the present embodiment, since the above-mentioned two-dimensional input image includes four photographs, the flow proceeds to the calculation step S350.

When the area division is not selected in S331, in the calculation step S350, the average value of the similarity between each photograph and each two-dimensionally created image determined to have the highest similarity to each photograph is calculated. For example, as shown in FIG. 3, the similarity between the first photograph and the first two-dimensional created image is 75 points, and the similarity between the second photograph and the second two-dimensional created image is 70 points. Assuming that the similarity between the third photograph and the third two-dimensional created image is 90 points, and the similarity between the fourth photograph and the fourth two-dimensional created image is 80 points, the calculation unit 24 The average value of 78.75 is calculated. In the output step S360, these 78.75 points are output.

If the area division is not selected in S331 and the number of two-dimensional input images input in the two-dimensional input image input step S311 is one, the flow proceeds from S332 to the output step of S360. Then, the similarity between the two-dimensional created image extracted in S330 and the two-dimensional input image is output as it is in S360.

(Confirmation step S370)
In the confirmation step S370, the modeler confirms whether or not the three-dimensional data created by the modeler has reached a predetermined value, for example, a pass line capable of submitting the three-dimensional data to the requester. In the present embodiment that has undergone the division step S340, the pass line at which the 3D data can be submitted to the requester is 80 points, whereas the weighted average value indicating the similarity of the 3D data created by the modeler is. Since it is 78.46154, the modeler recognizes that the three-dimensional data created by the modeler does not meet the above requirements.

By using such an information processing method, the modeler can recognize that it is necessary to appropriately modify the three-dimensional data until the output result exceeds a predetermined value.

Further, in the present embodiment, in the output step S360, the output unit 30 colors the "front other face area" and the "front forefoot area" that have not reached the passing line of 80 points in red, or the area concerned. Since the boundary line of is displayed as a red line and a thick line, the modeler can immediately recognize that the two areas are the areas to be corrected, and the work efficiency of the modeler can be improved.

In the present embodiment in which the score of the three-dimensional data created by the modeler has not reached the pass line, the process returns to the three-dimensional data input step S312, the modeler corrects the three-dimensional data, and inputs the three-dimensional data again. In the present embodiment, the modeler mainly modifies the "front and other face regions" and the "front forefoot region", which have a particularly low evaluation of similarity, and has corrected 3 in the three-dimensional data input step S312. The dimensional data is input to the input unit 10.

After that, when the output unit 30 outputs a calculated value exceeding the pass line in the output step S360 through the above-mentioned image generation step S320, extraction step S330, calculation step S350, etc., this flow ends and the three-dimensional data is sent to the requester. Will be sent. By such an information processing method, the quality of the three-dimensional data sent to the requester can be ensured, and as a result, the number of times the requester makes a correction request can be reduced. That is, it is possible to obtain three-dimensional data having a high degree of similarity to the image information specified by the requester without causing the requester to take time and effort.

10 input unit, 20 image generation unit, 22 extraction unit, 24 calculation unit, 26 division unit, 30 output unit, 40 control unit, 50 storage unit, 70 communication unit, 100 information processing device, S310 input step, S311 2D input Image input step, S312 3D data input step, S320 image generation step, S330 extraction step, S340 division step, S350 calculation step, S360 output step, S370 confirmation step.

Claims (9)

An input unit that accepts input of 3D data related to a 3D model and a plurality of 2D input images,
An image generation unit that generates a plurality of two-dimensional created images showing the appearance of the three-dimensional model arranged in the virtual space as viewed from a given viewpoint based on the three-dimensional data.
For each of the plurality of two-dimensional input images, an extraction unit that extracts the two-dimensional created image having the highest degree of similarity from the plurality of two-dimensional created images according to the number of the plurality of two-dimensional input images.
A calculation unit that calculates a value of 1 for evaluation of the three-dimensional data from the similarity between the plurality of two-dimensional created images extracted by the extraction unit and the plurality of two-dimensional input images.
Includes an output unit that outputs the value of 1 calculated by the calculation unit.
Information processing device. The calculation unit calculates the average value of the respective similarity as a value of 1 for the evaluation of the three-dimensional data.
The output unit outputs the average value calculated by the calculation unit.
The information processing device according to claim 1. The average value includes a weighted average value weighted with respect to the two-dimensional input image including the feature portion among the plurality of two-dimensional input images.
The information processing device according to claim 2. Weighting information as to whether or not a feature portion is included is added to the two-dimensional input image in which the input unit receives an input.
The information processing device according to claim 3. A division unit that divides the two-dimensionally created image extracted by the extraction unit into a plurality of regions, and a division unit.
Further including a calculation unit for calculating the similarity with the two-dimensional input image for the plurality of regions.
The output unit makes the display of a region having a similarity lower than a predetermined value among the plurality of regions different from the display of other regions.
The information processing device according to claim 1. The output unit displays by coloring a region where the similarity is lower than a predetermined value.
The information processing device according to claim 5. The two-dimensional input image is a photograph.
The information processing device according to any one of claims 1 to 6. An input step that accepts input of 3D data related to a 3D model and a plurality of 2D input images,
Based on the 3D data, an image generation step of generating a plurality of 2D created images showing the appearance of the 3D model arranged in the virtual space as viewed from a given viewpoint, and an image generation step.
For each of the plurality of two-dimensional input images, an extraction step of extracting the two-dimensional created image having the highest degree of similarity from the plurality of two-dimensional created images according to the number of the plurality of two-dimensional input images.
A calculation step for calculating a value of 1 for evaluation of the three-dimensional data from the similarity between the plurality of two-dimensional created images extracted by the extraction unit and the plurality of two-dimensional input images.
Includes an output step that outputs the value of 1 calculated by the calculation unit.
Information processing method. On the computer
An input step that accepts input of 3D data related to a 3D model and a plurality of 2D input images,
Based on the 3D data, an image generation step of generating a plurality of 2D created images showing the appearance of the 3D model arranged in the virtual space as viewed from a given viewpoint, and an image generation step.
For each of the plurality of two-dimensional input images, an extraction step of extracting the two-dimensional created image having the highest degree of similarity from the plurality of two-dimensional created images according to the number of the plurality of two-dimensional input images.
A calculation step for calculating a value of 1 for evaluation of the three-dimensional data from the similarity between the plurality of two-dimensional created images extracted by the extraction unit and the plurality of two-dimensional input images.
An output step that outputs the value of 1 calculated by the calculation unit, and
A program to execute.

Images