CN116543128A

CN116543128A - Method and device for generating three-dimensional grid model of object and electronic equipment

Info

Publication number: CN116543128A
Application number: CN202310457029.1A
Authority: CN
Inventors: 邬宏; 费义云
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2023-04-21
Filing date: 2023-04-21
Publication date: 2023-08-04

Abstract

The embodiment of the application provides a method, a device and an electronic device for generating a three-dimensional grid model of an object. Therefore, the original image of the object required by the method for generating the three-dimensional grid model of the object is easy to acquire, and the three-dimensional grid model of the object can be generated through the processing after the original image of the object is acquired, so that the generation efficiency of the three-dimensional grid model of the object is improved. In addition, the target size ratio of the target image contour is obtained, and the three-dimensional grid model of the object is generated according to the image area of the object in the original image and the target image contour, and the target size ratio of the target image contour is referred to, so that the matching rate of the three-dimensional grid model of the object and the actual object is improved.

Description

Method and device for generating three-dimensional grid model of object and electronic equipment

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for generating a three-dimensional mesh model of an object, and an electronic device.

Background

Currently, an object or subject may be represented using a two-dimensional image or a three-dimensional mesh model. Three-dimensional mesh models are often widely used in 3D (3D) application scenarios in many fields. However, in the prior art, the difficulty of acquiring a three-dimensional grid model is higher than that of acquiring a two-dimensional image. In addition, the three-dimensional grid model generation requirements in part of the fields are high, a large number of input images and other generation factors are required to be obtained, the three-dimensional grid model generation process is complicated, and the matching degree between the obtained three-dimensional grid model of the object and the actual shape of the object is low.

Therefore, how to improve the generation efficiency and the matching rate of the three-dimensional grid model of the object is a technical problem to be solved.

Disclosure of Invention

The embodiment of the application provides a method for generating a three-dimensional grid model of an object, so as to improve the generation efficiency and the matching rate of the three-dimensional grid model of the object. The embodiment of the application also relates to a device for generating the three-dimensional grid model of the object, electronic equipment and a computer storage medium. The embodiment of the application also relates to a method and a device for obtaining the three-dimensional grid model of the object, electronic equipment and a computer storage medium.

The embodiment of the application provides a method for generating a three-dimensional grid model of an object, which comprises the following steps: acquiring an original image of an object; carrying out matting processing on an original image of the object to obtain an image area of the object in the original image; determining a target image contour of the image area and a target size ratio of the target image contour based on the image area; and generating a three-dimensional grid model of the object according to the image area of the object in the original image, the target image contour and the target size ratio, wherein the three-dimensional grid model of the object is a three-dimensional geometric figure of the object.

Optionally, the determining, based on the image area, a target image contour of the image area and a target size ratio of the target image contour includes: carrying out image contour extraction processing on the image area to obtain image contour boundary points corresponding to the image area; fitting the image contour boundary points to obtain a fitted target image contour; and determining the pixel length ratio of the target image contour according to the image contour boundary points, and determining the target size ratio of the target image contour according to the pixel length ratio and the actual size ratio of the target image contour.

Optionally, the obtained image area of the object in the original image includes a preliminary image border of the image area; the method further comprises the steps of: and removing the preliminary image frame, and constructing a target image frame corresponding to the image area in a three-dimensional modeling mode.

Optionally, the performing image contour extraction processing on the image area to obtain an image contour boundary point corresponding to the image area includes: determining a connected chain code for describing the boundary points of the image outline; judging whether pixel points used for representing the image contour boundary in the image area are boundary points or not based on the determined connected chain codes; and if so, taking the pixel point as an image contour boundary point corresponding to the image area.

Optionally, the fitting processing is performed on the image contour boundary points to obtain a target image contour after the fitting processing, including: acquiring a curve formed by connecting all boundary points of the image contour; acquiring a first straight line formed by connecting a first boundary point and a second boundary point from the curve; acquiring a first vertical distance between a first target boundary point in the curve and the first straight line; judging whether the first vertical distance is smaller than or equal to a preset distance threshold value; and if the first vertical distance is smaller than or equal to a preset distance threshold value, taking the first straight line as the target image contour after the fitting processing.

Optionally, the method further comprises: if the first vertical distance is greater than a preset distance threshold, segmenting the first straight line by using the first target boundary point to obtain a second straight line formed by connecting the first boundary point and the first target boundary point and a third straight line formed by connecting the first target boundary point and the second boundary point; acquiring a second vertical distance between a second target boundary point in the curve and the second straight line and a third vertical distance between a third target boundary point in the curve and the third straight line; and if the second vertical distance is smaller than or equal to a preset distance threshold value and the third vertical distance is smaller than or equal to the preset distance threshold value, taking a broken line formed by connecting the second straight line and the third straight line as the target image contour after the fitting processing.

Optionally, the method further comprises: screening and obtaining a target image contour from the obtained multiple candidate image contours; the screening and obtaining the target image contour from the obtained multiple candidate image contours comprises the following steps: and screening and obtaining the target image contour from the obtained multiple candidate image contours according to a preset screening condition.

Optionally, the preset screening conditions include at least one of the following screening conditions: judging whether the number of contour edges contained in the candidate image contours is a preset edge data threshold value or not; judging whether the included angle of the adjacent edges of the candidate image contours is within a preset included angle range value or not; judging whether the ratio between the area of the candidate image contour and the image area is larger than a preset area ratio threshold value or not; and judging whether the shape of the candidate image contour has a concave-convex shape.

Optionally, the method further comprises: adjusting the position distribution areas of a plurality of sub-image areas in the image area and adjusting the position sequence of endpoints corresponding to the distribution of the plurality of sub-image areas in the image area; an image area in which the position distribution area and the position order of the end points are adjusted is obtained.

Optionally, the determining the target size ratio of the target image contour according to the pixel length ratio and the actual size ratio of the target image contour includes: acquiring two coordinate values of the image contour boundary point in a pixel coordinate system; taking a first coordinate value of the two coordinate values as width data and a second coordinate value as height data, and calculating a first pixel length ratio of the image contour boundary point in a pixel coordinate system; taking a first coordinate value of the two coordinate values as height data and a second coordinate value as width data, and calculating a second pixel length ratio of the image contour boundary point in a pixel coordinate system; and respectively judging the similarity between the first pixel length ratio and the actual size ratio and the similarity between the first pixel length ratio and the second pixel length ratio, wherein the pixel length ratio with the similarity being larger than or equal to a preset similarity threshold value is used as the target size ratio of the image contour.

Optionally, the method further comprises: and if the image area comprises a plurality of sub-image areas, determining the target size ratio of the image outlines corresponding to the sub-image areas in the image area by adopting a multi-combination image size matching model.

Optionally, the generating a three-dimensional grid model of the object according to the image area of the object in the original image, the target image contour and the target size ratio includes: determining endpoint position coordinates of the object according to the coordinate size of the image area, the coordinate size of the target image outline and the target size ratio; and carrying out triangulation processing on the endpoint position coordinates of the object to obtain a three-dimensional grid model of the object.

The embodiment of the application also provides a method for obtaining the three-dimensional grid model of the object, which comprises the following steps: obtaining identification information of an object; acquiring an original image of the object based on the identification information of the object; according to the original image, the method is utilized to obtain a three-dimensional grid model of an object containing the original image, wherein the three-dimensional grid model of the object is a three-dimensional geometric figure of the object.

The embodiment of the application also provides electronic equipment, which comprises a processor and a memory; the memory stores a computer program, and the processor executes the method after running the computer program.

The embodiment of the application also provides a computer storage medium, which stores a computer program, and the computer program executes the method after being executed by a processor.

Compared with the prior art, the embodiment of the application has the following advantages:

According to the method, the original image of the object is subjected to matting processing, the image area of the object in the original image is obtained, and the target image contour of the image area and the target size ratio of the target image contour are determined. Based on the image area of the object in the original image, the target image contour and the target size ratio, a three-dimensional mesh model of the object, also referred to as a three-dimensional geometric map of the object, is generated. Therefore, the original image of the object required by the method for generating the three-dimensional grid model of the object is easy to acquire, and the three-dimensional grid model of the object can be generated through the processing after the original image of the object is acquired, so that the generation efficiency of the three-dimensional grid model of the object is improved. In addition, the target size ratio of the target image contour is obtained, and the three-dimensional grid model of the object is generated according to the image area of the object in the original image and the target image contour, and the target size ratio of the target image contour is referred to, so that the matching rate of the three-dimensional grid model of the object and the actual object is improved.

Drawings

Fig. 1 is an application scenario diagram of a method for generating a three-dimensional mesh model of an object according to an embodiment of the present application.

Fig. 2 is a schematic diagram of image information extraction of an object according to an embodiment of the present application.

Fig. 3 is a schematic diagram of an image matting processing procedure provided in an embodiment of the present application.

Fig. 4 is a schematic diagram of a process for removing a frame of an image by using a corrosion operation according to an embodiment of the present application.

Fig. 5 is a schematic diagram of comparison between before and after an image frame removing operation according to an embodiment of the present application.

Fig. 6 is a schematic diagram of a chain code for determining whether an image contour pixel point is a boundary point according to an embodiment of the present application.

Fig. 7 is a schematic diagram of determining whether two pixel points AB are boundary points according to the embodiment of the present application.

Fig. 8 is a schematic diagram of an image region matting result and a contour boundary point extraction result of an object in an original image according to an embodiment of the present application.

Fig. 9 is a schematic diagram of a fitted image contour provided in an embodiment of the present application.

Fig. 10 is a schematic diagram of an image contour boundary point extraction result and an image contour boundary fitting result provided in an embodiment of the present application.

Fig. 11 is a schematic diagram of position adjustment of each sub-image area in an original image of an object according to an embodiment of the present application.

Fig. 12 is a schematic diagram of size matching degree adjustment of each sub-image area in an image area according to an embodiment of the present application.

Fig. 13 is a schematic diagram of a hanging three-dimensional model construction flow provided in an embodiment of the present application.

Fig. 14 is a schematic diagram of an image frame generating manner according to an embodiment of the present application.

Fig. 15 is a flowchart of a method for generating a three-dimensional mesh model of an object according to a first embodiment of the present application.

Fig. 16 is a schematic diagram of a device for generating a three-dimensional mesh model of an object according to a second embodiment of the present application.

Fig. 17 is a flowchart of a method for obtaining a three-dimensional mesh model of another object according to a third embodiment of the present application.

Fig. 18 is a schematic diagram of an apparatus for obtaining a three-dimensional mesh model of another object according to a fourth embodiment of the present application.

Fig. 19 is a schematic view of an electronic device according to a fifth embodiment of the present application.

Fig. 20 is a flowchart of a method for generating a multi-die type dimension according to an embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is, however, susceptible of embodiment in many other ways than those herein described and similar generalizations can be made by those skilled in the art without departing from the spirit of the application and the application is therefore not limited to the specific embodiments disclosed below.

The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. The manner of description used in this application and in the appended claims is for example: "a", "a" and "a" etc. are not limited in number or order, but are used to distinguish the same type of information from each other.

First, concepts related to the present application will be described:

the object is: it may be a single basic object, e.g. a commodity, for constructing a three-dimensional mesh model, or a combination of objects, e.g. a set of commodity combinations, of multiple objects for constructing a three-dimensional mesh model.

Original image of object: may be a two-dimensional image or video frame depicting the object, such as a merchandise hanging, or a combination of a group of merchandise hanging comprised of a plurality of merchandise hanging.

And (3) matting processing: the foreground picture and the background picture in the original image can be separated to obtain an image area or a graphic area of the object in the original image.

Image area: may be an area of the original image for describing main information of the image.

Image profile: may be a boundary contour of an image area of the object in the original image.

Pixel length ratio: it may be a ratio between two values in pixel coordinates that will represent image pixel information.

Triangularization: also called point cloud gridding, i.e. the interconnection of discrete points in a three-dimensional space into a triangular mesh and the representation of the object surface by the triangular mesh.

Sweep technique: one way to change a two-dimensional shape image into a three-dimensional shape image is to focus on sweeping a simple two-dimensional shape image along a complex curve to obtain a three-dimensional shape image.

Ex (stretching): one way to change a two-dimensional shape image into a three-dimensional shape image is to focus on stretching a complex two-dimensional shape image along a straight line to obtain a three-dimensional shape image.

Buffer: the two-dimensional shape image is expanded by the target distance in the target direction.

The embodiment of the application provides a method for generating a three-dimensional grid model of an object. The embodiment of the application also provides a device for generating the three-dimensional grid model of the object, electronic equipment and a computer storage medium. The following examples are described in detail one by one.

In order to facilitate understanding of the methods and systems provided in the embodiments of the present application, a description of the background of the embodiments of the present application is provided before the description of the embodiments of the present application.

In view of the foregoing, those skilled in the art can appreciate the problems existing in the prior art, and the following describes in detail an application scenario of the method for generating a three-dimensional mesh model of an object of the present application. The method for generating the three-dimensional grid model of the object can be applied to various application scenes for generating the three-dimensional grid model of the object according to a single two-dimensional image of the object. For example, a two-dimensional image of a commodity is obtained on a shopping platform, and according to the two-dimensional image of the commodity, a three-dimensional grid model of the commodity, that is, a three-dimensional geometric figure of the commodity, is generated according to the method for generating the three-dimensional grid model of the object provided by the application.

For another example, according to Shan Zhanggua, a three-dimensional mesh model of the hanging, that is, a three-dimensional geometric hanging drawing of the hanging, is generated by using the method for generating a three-dimensional mesh model of the object.

The specific generation method is as follows: the method comprises the steps of obtaining a single Zhang Yuanshi image of a hanging picture, carrying out matting processing on the single Zhang Yuanshi image of the hanging picture to obtain an image area of the hanging picture, in other words, separating main image content of the hanging picture from background content of the hanging picture, and carrying out area buckling processing for describing main content information of the hanging picture in the hanging picture to obtain the image area of the hanging picture. Then, carrying out image contour extraction processing on the hung image area to obtain image contour boundary points of the hung image area; fitting the image contour boundary points to obtain a target image contour after fitting; and determining the pixel length ratio of the target image contour according to the image contour boundary points, and determining the target size ratio of the target image contour according to the pixel length ratio and the actual size ratio of the target image contour. And finally, acquiring a target image contour of the hanging image and a target size ratio of the target image contour, and generating a three-dimensional grid model of the hanging image according to the image area of the hanging image, the target image contour and the target size ratio.

According to the method for generating the hanging three-dimensional grid model, the three-dimensional grid model of the object is generated according to the single Zhang Yuanshi image of the object, so that the generation efficiency of the three-dimensional grid model of the object is improved. In addition, the target size ratio of the target image contour is obtained, and the three-dimensional grid model of the object is generated according to the image area of the object in the original image and the target image contour, and the target size ratio of the target image contour is referred to, so that the matching rate of the three-dimensional grid model of the object and the actual object is improved.

For another example, in the home decoration industry, a three-dimensional design drawing of a home decoration design is generated by using the method for generating a three-dimensional mesh model of the object according to a two-dimensional home decoration design drawing in the home decoration design.

The specific generation method is as follows: the method comprises the steps of obtaining a single two-dimensional home decoration design drawing, carrying out matting processing on the single two-dimensional home decoration design drawing to obtain an image area of the two-dimensional home decoration design drawing, in other words, separating main image content and background content of the two-dimensional home decoration design drawing, and carrying out area buckling processing for describing main content information of the image in the two-dimensional home decoration design drawing to obtain an image area of the two-dimensional home decoration design drawing. Then, carrying out image contour extraction processing on the image area of the two-dimensional home decoration design drawing to obtain image contour boundary points of the image area of the two-dimensional home decoration design drawing; fitting the image contour boundary points to obtain a target image contour after fitting; and determining the pixel length ratio of the target image contour according to the image contour boundary points, and determining the target size ratio of the target image contour according to the pixel length ratio and the actual size ratio of the target image contour. And finally, acquiring a target image contour of the two-dimensional home decoration design drawing and a target size ratio of the target image contour, and generating a three-dimensional grid model corresponding to the two-dimensional home decoration design drawing, namely the three-dimensional design drawing, according to the image area of the two-dimensional home decoration design drawing and the target image contour and the target size ratio.

Compared with the prior art, a great deal of time and economic cost are required for constructing the three-dimensional image of the target object, and the method for generating the three-dimensional grid model of the object according to the single two-dimensional image of the object, which is provided by the embodiment of the application, requires that the input image information is the information of the single two-dimensional image of the object, so that the efficiency of generating the three-dimensional grid model of the object is improved. In addition, a three-dimensional grid model of the object is generated, the image area of the object in the original graph is referred, the target image contour of the image area is referred, and the target size ratio is determined for the target image contour, so that the matching rate between the three-dimensional grid model of the object and the actual object is improved.

The method for constructing the three-dimensional grid model in the prior art generally comprises at least three construction modes:

the first three-dimensional network model construction mode is as follows: the body modeling and color rendering adopts a staged construction mode, and the construction method of the three-dimensional network model is split into two subtasks, namely a body modeling subtask and a color rendering subtask. Modeling each subtask by using one sub-model respectively, and then using the Sota algorithm of each sub-model to respectively make the shape and the color to be the target state in the current state. The Sota (State of the arts, the target state in the current state) may be a model for training in a certain field to reach the target state, and may generally refer to a model for running on some basic comparison object data set to reach a preset condition. Advantages of the first approach include at least one of the following: 1) The training mode is firstly used for training color information, tasks are respectively trained in two stages, and model training difficulty is reduced. 2) The training result of each sub-model obtained by the staged training easily reaches the preset condition.

However, the above-described first three-dimensional network model construction method stores the following drawbacks: 1) In the model training process, various data information including input images, masks, key points and semantic segmentation parameters are required to be input, the input data are more, and the requirement on the input data for model construction is higher. 2) In addition, the model output is respectively output aiming at each type of subtask model, wherein the morphological estimated value output by the morphological model is relatively close. 3) The subtask model is trained in two stages respectively, so that the model training time and the test time are long.

The second three-dimensional network model construction mode is as follows: a neural renderer.

The three-dimensional grid model is constructed by using the nerve renderer, so that end-to-end training learning can be realized, and learning targets are gestures, shapes, camera shooting parameters, color information and the like. The neural renderer can derive the data by optimizing the pixel sampling process. Rendering a three-dimensional image rendering result by using a model constructed by a nerve renderer, projecting the rendering result back to the two-dimensional image by using a projection technology, calculating the difference between the generated rendering result and an original image, and then acquiring key parameters for training the three-dimensional model.

The second three-dimensional grid model construction mode has the advantages that single-stage learning is directly carried out, a model training frame is simple, data required for training the model is reduced, and the model can be trained to an expected state by using a mask under the target condition.

However, the model obtained by the second method described above is determined by at least one of: 1) The data assumes that the training object is a symmetrical object, and the training needs to initialize the template, so that the learning difficulty of model training is obviously enhanced for the template-free, non-rigid and asymmetrical body. 2) Because the second model training is a self-supervision learning model, the training is easy to enter a suboptimal link or a state incapable of converging due to the fact that the second model training has no definite target training state data. 3) The second model training is limited by the influence of object volume and complexity, has weak training effect on complex objects, and cannot effectively learn the detailed information in the training model.

The third three-dimensional grid model training method comprises the following steps: a nerve radiation field.

The neural radiation fields belong to a similar function as the neural renderer, but each still has its own advantages. The working principle of the nerve radiation field is as follows: and learning three-dimensional space coordinates and a two-dimensional view based on the radiation field and the volume density of the view by using the three-dimensional space information and the two-dimensional posture information, and projecting the three-dimensional space coordinates and the two-dimensional view onto RGB color values. The specific implementation method comprises the steps of translating input information into pixel values and voxel densities by using a fixed condition coding and multi-layer perceptron, and then carrying out body state modeling to directly map the two-dimensional view input information into three-dimensional space coordinates.

Advantages of the third model training method may include at least one of the following: 1) The problem of training data scarcity in the training process is solved, and meanwhile, a target model in the current state can be obtained according to single image input information, so that multi-angle reproduction is further carried out. The third model training method is less costly than both methods described above. 2) The neural radiation field learns three-dimensional characteristics by using a hidden learning mode, and compared with a training mode based on a three-dimensional template, the third mode does not need symmetry requirements, and the application range can be expanded to non-rigid body categories, so that the generalization capability is stronger. 3) The nerve radiation field has an interpretability, and shallow layer characteristics generated by model training can be used for obtaining a three-dimensional grid model through visual chemistry after being processed.

Drawbacks of the third model training method include at least one of the following: 1) Only a single image reconstruction process under a single axis can be fitted. 2) The model cannot be used to obtain a reconstructed three-dimensional image based on a given image.

Therefore, the three model construction methods have the situations that the model construction input information is more, so that the model training cost is higher and the training is complex, or the training object is required to be a symmetrical object in the training process, the training learning difficulty is higher, or the model cannot be trained to a target state, or the reconstruction process of a single image under a complete coordinate axis cannot be carried out.

Based on the above-mentioned problem of model training in the prior art, the embodiment of the application provides a method for generating a three-dimensional grid model of an object, which performs matting processing on an original image of the object to obtain an image area of the object in the original image, and determines a target image contour of the image area and a target size ratio of the target image contour. Based on the image area of the object in the original image, the target image contour and the target size ratio, a three-dimensional mesh model of the object, also referred to as a three-dimensional geometric map of the object, is generated. Therefore, the original image of the object required by the method for generating the three-dimensional grid model of the object is easy to acquire, and the three-dimensional grid model of the object can be generated through the processing after the original image of the object is acquired, so that the generation efficiency of the three-dimensional grid model of the object is improved. In addition, the target size ratio of the target image contour is obtained, and the three-dimensional grid model of the object is generated according to the image area of the object in the original image and the target image contour, and the target size ratio of the target image contour is referred to, so that the matching rate of the three-dimensional grid model of the object and the actual object is improved. .

Please refer to fig. 1, which is an application scenario diagram of a method for generating a three-dimensional mesh model of an object according to an embodiment of the present application.

Fig. 1 is a diagram illustrating an example of a method for generating a three-dimensional mesh model for a single hanging commodity.

Step 1: and acquiring the identification information of the single hanging picture commodity.

The identification information of the hanging commodity is used for representing the authentication information of the hanging commodity on the commodity platform, and each hanging commodity has unique identification information.

Step 2: and inquiring and obtaining the commodity main diagram of the single hanging commodity according to the identification information of the single hanging commodity.

Please refer to fig. 2, which is a schematic diagram illustrating image information extraction of an object according to an embodiment of the present application.

According to the identification information of the single hanging commodity, at least one of the following information can be inquired: commodity main picture information, commodity size information, commodity hanging frame type information and hanging commodity style information. The information is key information for constructing the three-dimensional grid model of the image.

Step 3: and carrying out the picture matting processing on the commodity main picture to obtain commodity main body content, namely a commodity content area in the hanging picture.

The commodity main graph adopts a matting technology and can adopt a U2Net network, wherein U2Net is proposed for Salient Object Detetion (SOD), namely a saliency target detection task. The saliency target detection task is very similar to the semantic segmentation task, and the saliency detection task is a classification task, mainly attractive targets or areas in the picture are segmented, so that only two types of data including foreground and background of the picture are provided. The foreground corresponds to the commodity main body in the embodiment of the application, and the background corresponds to the background of the commodity image in the embodiment of the application.

Please refer to fig. 3, which is a schematic diagram illustrating an image matting processing procedure according to an embodiment of the present application. In fig. 3, the first left-hand image is a commodity main image, the second left-hand image is a foreground (Bai Detu) of the commodity main image, and the third left-hand image is a background (mask image, also referred to as a black-matrix-white frame image from which commodity main image information is removed) of the commodity main image. And carrying out the picture matting processing on the main picture of the hanging commodity to obtain the foreground and the background of the hanging commodity.

Step 4: and removing the frame from the commodity main body content.

And 3, carrying out matting processing on the commodity main image to obtain a commodity main image and a background image, wherein in the commodity main image, the situation that the matting is incomplete exists on the border lines of all the sub-areas, so that the border removing operation is carried out on the commodity main image.

According to the embodiment of the application, the frame is removed by adopting an erode (corrosion, erosion, erasure, and the like) corrosion operation mode, particularly, the boundary points of the commodity main body diagram are eliminated, so that the boundary points of the commodity main body diagram shrink inwards, and objects smaller than elements of the structural units are removed.

Fig. 4 is a schematic diagram illustrating a process of removing a frame of an image by using a corrosion operation according to an embodiment of the present application. In fig. 4, by constructing a structural element, then performing an and operation with the target image, specifically, a point with the structural element being 1, and points corresponding to the target image being 1, the pixel of the point is 1, otherwise, is 0. Specifically, as in the example of the structural element in fig. 4, the structural element is a 1 point, the point on the left side of the point corresponds to the adjacent point being 1, and the point on the upper side of the point corresponds to the adjacent point being 1, the pixel representing the point where the structural element is 1. In the above manner, the upper side corresponding to the uppermost 1 row of points in the target image is the point with the structural element 0, and therefore, the uppermost 1 row of points in the target image is the point to be removed. In addition, a 1 point of the structural element adjacent to 0 in the longitudinal axis direction in the target image also belongs to a point to be removed. Therefore, after the target image is subjected to the erosion operation, an image from which the frame of the image is removed, that is, the erosion result diagram in fig. 4 is obtained.

Reference may be made to fig. 5, which is a schematic diagram of the comparison of the mask image obtained by the matting processing in fig. 3 before and after the frame removal operation according to the corrosion operation manner shown in fig. 4. The first diagram on the left side of fig. 5 is a diagram before the mask diagram is subjected to the etching operation, and the second diagram on the left side of fig. 5 is a diagram after the mask diagram is subjected to the etching operation, wherein the gap between every two white areas is wider than the gap of the first diagram.

Step 5: and extracting the hanging outline of the hanging commodity main body content to obtain the outline boundary point of the hanging commodity main body.

The hanging outline extraction in this step may be that a boundary tracking algorithm Suzuki85 is adopted on the matting result to complete the hanging outline extraction, the boundary tracking algorithm converts a binary image into a representation form of the boundary, and the topology structure between the boundaries is extracted.

Before the hanging outline extraction operation, a chain code is first defined. Fig. 6 is a schematic diagram of a chain code for determining whether an image contour pixel point is a boundary point according to an embodiment of the present application.

The chain code is used to describe a boundary or curve using the start point coordinates and the boundary point direction code. The chain codes typically include 4-way chain codes and 8-way chain codes. Wherein, the 4-communication chain code indicates that when the pixels in the upper, lower, left and right directions are connected with the central pixel, the pixel belongs to a communication domain, namely a boundary or region; correspondingly, an 8-way chain code indicates that 8-way pixels are connected to a center pixel.

Fig. 7 is a schematic diagram of determining whether two pixel points AB are boundary points according to the embodiment of the present application. In fig. 7, the type of the connected chain code is defined first, the analysis is performed by using the 4 connected chain code, the pixels in the up, down, left and right directions of the pixel point a and the pixel point B are connected with the pixel point a, and belong to one connected domain, and then neither the pixel point a nor the pixel point B is a boundary point. And (3) analyzing by using an 8-communication chain code, wherein all the 8-direction pixel points adjacent to the pixel point A are connected with the similar point A, belong to a communication domain, and are not boundary points. The 8-direction pixel points adjacent to the B pixel point are not all connected to the B pixel point, and therefore the B pixel point belongs to the boundary point.

After the connected chain code type is determined, judging whether contour pixel points in the matting structure are boundary points according to the determined connected chain code type, and if so, marking the boundary points on the frame position of each white area in the mask graph after extraction. Here, the boundary tracking algorithm Suzuki85 is used to complete the extraction of the hanging outline, please refer to fig. 8, which is a schematic diagram of the image region matting result and the outline boundary point extraction result of the object in the original image provided in the embodiment of the present application. In fig. 8, a plurality of boundary points are extracted from the border of each white area in the mask map, so as to be used for performing shape fitting according to the boundary points in step 6, and obtaining the target hanging outline after fitting processing.

Step 6: and performing shape fitting according to the contour boundary points of the commodity hanging main body to obtain the target hanging contour after fitting treatment.

In the step, fitting treatment is performed on the contour boundary points to obtain the target hanging contour. In one implementation of the embodiment of the present application, the boundary point fitting is performed using the dawster-plck algorithm. Please refer to fig. 9, which is a schematic diagram of a fitted image contour according to an embodiment of the present application.

Fig. 9 depicts the algorithm principle of the douglas-pock algorithm, specifically as follows:

all boundary points in the frame are connected to form a curve. And establishing a first straight line between the first boundary point a and the second boundary point b of the curve, wherein the first straight line is used as a chord of the curve.

Acquiring a first vertical distance d1 between a first target boundary point c and a first straight line ab in the curve; judging whether the first vertical distance d1 is smaller than or equal to a preset distance threshold value, and if so, judging that the first straight line ab is an approximate line of a curve. Corresponding to the embodiment of the application, the first line ab may be taken as the target hanging outline.

If the first vertical distance d1 is greater than the preset distance threshold, segmenting the first line ab with the first target boundary point c to obtain a second line ac and a third line bc.

A second vertical distance d2 between a second target boundary point e and a second straight line ac in the curve is obtained, and a third vertical distance d3 between a third target boundary point f and a third straight line bc in the curve is obtained.

Judging whether the second vertical distance d2 is smaller than or equal to a preset distance threshold value or not, and judging whether the third vertical distance d3 is smaller than or equal to the preset distance threshold value or not; and if the second vertical distance d2 is smaller than or equal to the preset distance threshold value and the third vertical distance d3 is smaller than or equal to the preset distance threshold value, a broken line formed by connecting the second straight line ac and the third straight line bc is used as an approximate line of the curve. Corresponding to the embodiment of the application, the broken line formed by connecting the second straight line ac and the third straight line bc can be used as the target hanging outline.

Please refer to fig. 10, which is a schematic diagram illustrating an image contour boundary point extraction result and an image contour boundary fitting result provided in an embodiment of the present application. The first graph on the left side of fig. 10 is an image contour boundary point extraction result, and the second graph on the left side of fig. 10 is an image contour boundary fitting result obtained after the boundary points are fitted by adopting the above-mentioned dawshare-pock algorithm.

Step 7: and for the condition that a plurality of candidate hanging outlines are obtained through fitting, screening from the candidate hanging outlines to obtain target hanging outlines.

If a plurality of candidate hanging outlines obtained by fitting are needed to be screened, the specific screening method can comprise at least one of the following screening conditions:

judging whether the number of contour edges contained in the candidate image main body contours is a preset edge data threshold value or not;

in this embodiment of the present application, a hanging image is used, so that the candidate image main body contour belongs to a rectangle, and here, it is necessary to determine whether contour edge data included in the candidate image main body contour is 4 edges.

Judging whether the included angle of the adjacent edges of the main contour of the candidate image is within a preset included angle range value or not;

if the included angle of the adjacent edges is a squint angle, the squint angle has perspective phenomenon, but the perspective phenomenon does not exist in the vertical angle of 90 degrees. Therefore, whether the included angle between two adjacent edges of the main contour of the candidate image is 90 degrees is judged, and if the included angle is 90 degrees, no perspective phenomenon exists between the two contour edges.

Judging whether the ratio between the area of the main contour of the candidate image and the area of the image is larger than a preset area ratio threshold value or not;

the interior of the candidate image main body outline is the image main body content, the proportion of the area size of the image main body content in the whole size of the image is larger than a preset proportion threshold value, so that whether the ratio between the areas of the plurality of candidate image main body outlines and the whole area of the image is larger than a preset area ratio threshold value is judged, and if the ratio is larger than the preset area ratio threshold value, the candidate image main body outline is the target image main body outline.

And judging whether the shape of the candidate image main body outline has a concave-convex shape or not.

If each edge of the candidate image main body contour obtains a contour edge, connecting the contour edges to obtain the image main body contour without concave-convex type.

If the candidate image main body contour is formed by one less contour edge, the image main body contour obtained by connection is a concave contour, and if the candidate image main body contour is formed by one more contour edge, the image main body contour obtained by connection is a convex contour. Therefore, it is necessary to determine whether or not the candidate image subject contour has a concave-convex shape, and when the candidate image subject contour does not have a concave-convex shape, the candidate image subject contour is the target image subject contour.

Step 8: and (3) carrying out position adjustment on the commodity hanging pictures in all the areas in the commodity hanging main body content, and sequencing the commodity hanging pictures in all the areas.

The method comprises the step of carrying out position adjustment and endpoint sequencing on commodity hanging pictures in all areas in the main commodity hanging picture content. The position adjustment comprises the problem of the placement positions among hanging pictures in different shapes. The end point ordering includes the order of the positions of the individual end points in a hanging shape to ensure the accuracy of each hanging map.

Specifically, endpoint ordering may be achieved by:

please refer to fig. 11, which is a schematic diagram illustrating a position adjustment of each sub-image area in an image area of an original image of an object according to an embodiment of the present application.

Two hanging pictures are shown in fig. 11, wherein the first hanging picture on the left side is a hanging picture placement image before the hanging picture position is adjusted, and the second hanging picture on the left side is a hanging picture placement image after the hanging picture position is adjusted. The position adjustment of the hanging picture can be realized by the following modes: the location and interrelationship between the four endpoints of each hanging image is determined. First, position adjustment, the mutual position among each hanging image is determined, and the hanging images are ordered from small to large according to the left lower corner coordinate value of each hanging image.

Secondly, end point sequencing, namely determining positions among the four end points of each hanging image, and establishing a coordinate system by taking the left upper corner coordinates (0, 0) of the whole hanging image as the center to acquire the left sides of the four end points of each hanging image. Wherein, the lower right corner coordinate of the hanging picture is (Hi, wi), the left side of the lower left corner of the hanging picture is (Hi, 0), and the upper right corner coordinate of the hanging picture is (0, wi).

Step 9: and matching the hanging size in the main body content of the hanging commodity to obtain the target size of the hanging commodity.

The method comprises the steps of matching the hanging picture sizes in the main body content of the hanging picture commodity, judging that two coordinate values are respectively wide, high and high when the two coordinate values are respectively wide for a single hanging picture through a cross judgment method, obtaining two pixel length ratios, and comparing the pixel length ratios with actual size ratios to obtain the actual size of a hanging picture.

The pixel length ratio may be a ratio of a horizontal axis length to a vertical axis length on a pixel coordinate axis, and the actual size ratio may be a ratio of width data to height data of the hanging image.

Please refer to fig. 12, which is a schematic diagram illustrating a size matching degree adjustment of each sub-image area in the image area according to an embodiment of the present application.

In fig. 12, a single-shot picture corresponding to the part (a) is shown, which includes a plurality of coordinate data, and is illustrated as (0.4, 0.8). With 0.4 as the horizontal axis width data and 0.8 as the vertical axis height data, the first pixel length ratio was obtained to be 0.8/0.4=2. With 0.4 as vertical axis height data and 0.8 as horizontal axis width data, a second pixel length ratio of 0.4/0.8=0.5 is obtained.

As can be seen in the single-wall picture corresponding to part (a) in fig. 12, the ratio of the pixel length of the image to the ratio of the height pixel of the image to the width pixel of the image is 2:1.

As can be seen from the pixel length ratio of the single-hanging image corresponding to the portion (a) in fig. 12, the vertical axis height data is 0.8, and the horizontal axis width data is 0.4.

In fig. 12, (b) and (c) are each of a plurality of hanging drawings. When determining the target pixel length ratio of the multiple hanging images, a multiple-combination mode type size generation method can be adopted to determine the pixel length ratio of the multiple-combination hanging images.

In the hanging drawings corresponding to the part (b) in fig. 12, the vertical axis height data of the three hanging drawings are the same, and the horizontal axis width data of the middle image is wider than the horizontal axis width data of the left and right images based on the coincidence of the left and right drawings.

In order to make the vertical axis height data of the three hanging pictures uniform, it can be determined from the sets of coordinate data provided in part (b) in fig. 12 that the horizontal axis width data of the first sub-image is 0.4 and the vertical axis height data of the first sub-image is 0.6. The horizontal axis width data of the third sub-image is 0.4 and the vertical axis height data of the third sub-image is 0.6. The horizontal axis width data of the second sub-image is 0.8 and the vertical axis height data is 0.6.

The size of the coordinate data provided in the part (c) in fig. 12 is the overall size describing the hanging image assembly, specifically, the overall horizontal axis width data of the hanging image assembly is 1.0 and the overall vertical axis height data is 0.55. Alternatively, the overall horizontal axis width data of the hanging image combination is 1.5, and the overall vertical axis height data is 0.8. Alternatively, the overall horizontal axis width data of the hanging image combination is 2.0, and the overall vertical axis height data is 1.1.

As can be seen from the multi-hanging picture corresponding to the part (c) in fig. 12, the overall pixel length ratio of the graphic assembly composed of 5 sub-images is 2:1.

from the three sets of coordinate data ratios provided in part (c) of fig. 12, the overall horizontal axis width data of the combined hanging image is 2.0, and the overall vertical axis height data is 1.1, which is the ratio of the overall horizontal axis length data of the combined image to the vertical axis height data of the image.

Step 10: and performing size similarity matching on the main body contents of the plurality of groups of hanging painting commodities.

Please refer to fig. 20, which is a flowchart of a method for generating a multi-mold assembly size according to an embodiment of the present application.

Fig. 20 depicts a flow chart for determining the target size ratio by combining multiple sets of images, which are determined in three ways. In the first case, as for the hanging image composed of single hanging images corresponding to the part (a) in fig. 12, the target size ratio of the single hanging image is determined, the pixel length ratio of the image can be determined according to the image, then, the ratio of the plurality of horizontal axis width coordinate data and the vertical axis height coordinate data provided by the part (a) in fig. 12 is determined respectively, whether the ratio belongs to the pixel length ratio range of the single hanging image or not, and the ratio of the hanging image horizontal axis width coordinate data and the vertical axis height coordinate data belonging to the pixel length ratio range of the image is determined to the target size ratio.

In the second case, as for the combination of 3 hanging pictures provided in the part (b) in fig. 12, the 3 hanging pictures are formed by combining 3 individual hanging pictures, and as can be seen from the multiple hanging pictures corresponding to the part (b) in fig. 12, the ratio of the pixel length to the dimension length in the width-height direction of the three individual hanging pictures. The vertical axis height data of the three sub-images are equal and the horizontal axis width data of the first sub-image and the third sub-image are the same. Therefore, after the ratio is obtained, a plurality of size data are generated according to the ratio, whether the ratio requirement is met or not is judged according to the plurality of size data, and then the size data corresponding to the plurality of hanging painting combinations are determined.

In the third case, if the ratio requirement is not satisfied in the finally generated plurality of size data, there is no proper size, and the size ratio is determined from the method of newly performing the second case.

Step 11: obtaining the hanging commodity model.

Please refer to fig. 13, which is a schematic diagram of a hanging three-dimensional model construction flow provided in an embodiment of the present application. In fig. 13, the process of creating a three-dimensional model of a hanging picture includes a hanging picture generation method, a glass generation method, and a frame generation method. The hanging picture generation mode can be realized by the following modes: and determining the position coordinates of the hanging end points according to the coordinate sizes of the hanging commodity drawing and the bounding box sizes of the hanging image, and triangulating the position coordinates of the hanging end points to obtain the three-dimensional grid model of the hanging commodity. And generating a frame position of the hanging picture by the three-dimensional grid model of the hanging picture commodity, and attaching the picture matting result of the hanging picture commodity to the frame position of the hanging picture to obtain a hanging picture commodity image.

The glass forming mode can be realized by the following modes: and determining the coordinate position of the glass endpoint according to the position of the hanging commodity image and the relative position relation between each hanging picture in the hanging commodity image.

The frame scheme generating manner may be implemented in at least three ways, please refer to fig. 14, which is a schematic diagram of an image frame generating manner provided in an embodiment of the present application.

Fig. 14 includes three types of frame generation methods, which are specifically as follows:

in the first frame generation mode, for a simple regular rectangular hanging frame, four cuboid frames are generated in the circumferential direction of a hanging commodity image, and the frame of the rectangular hanging commodity image is obtained.

In the second frame generation mode, for frames containing complex contours, a sweep technology or an ex technology is adopted, and a two-dimensional commodity image is changed into a three-dimensional commodity image through a target path point and a shape point, wherein the sweep technology is used for sweep of a simple shape along a path curve, and the ex technology is used for stretching of the complex shape.

In the third frame generation mode, for the frame generation mode of the polygonal combined hanging outline, the frame shape of the hanging is generally obtained according to the hanging outline buffer, meanwhile, the end point coordinates of each frame of the hanging image are obtained by combining the thickness position information of the hanging image, the upper surface and the lower surface of the hanging image are subjected to triangularization processing, and the side faces are subjected to triangularization processing, so that the frame of the hanging image is obtained.

First embodiment

Fig. 15 is a flowchart of a method for generating a three-dimensional mesh model of an object according to the first embodiment of the present application, and the method for generating a three-dimensional mesh model of an object according to the present embodiment is described in detail below with reference to fig. 15.

As shown in fig. 15, in step S1501, an original image of an object is acquired.

This step is used to acquire an original image of the object, here representing a single two-dimensional image that may be acquired of the commodity. For example, in fig. 2, a commodity main diagram of a single hanging commodity is obtained according to the identification information inquiry of the single hanging commodity. According to the identification information of the single hanging commodity, at least one of the following information can be inquired: commodity main picture information, commodity size information, commodity hanging frame type information and hanging commodity style information. The information is key information for constructing the three-dimensional grid model of the image.

As shown in fig. 15, in step S1502, a matting process is performed on an original image of the object, and an image area of the object in the original image is obtained.

The method is used for carrying out matting processing on the image to obtain an image area of the object in the original image. Specifically, an image area for representing subject content information of an object in an original image may be mentioned. Thereby determining the target image contour of the image area from the image area in a subsequent step.

Wherein, based on the image area, determining the target image contour of the image area and the target size ratio of the target image contour can be realized by the following ways:

carrying out image contour extraction processing on the image area to obtain image contour boundary points corresponding to the image area; fitting the image contour boundary points to obtain a fitted target image contour; and determining the pixel length ratio of the target image contour according to the image contour boundary points, and determining the target size ratio of the target image contour according to the pixel length ratio and the actual size ratio of the target image contour.

The matting processing may adopt a matting technique, for example, a U2Net network, where U2Net is proposed for Salient Object Detetion (SOD), i.e., a salient object detection task. The saliency target detection task is very similar to the semantic segmentation task, and the saliency detection task is a classification task, mainly attractive targets or areas in the picture are segmented, so that only two types of data including foreground and background of the picture are provided. The foreground corresponds to the commodity main body in the embodiment of the application, and the background corresponds to the background of the commodity image in the embodiment of the application.

Carrying out matting processing on the image of the target object, wherein the image area of the obtained object in the original image comprises a preliminary image frame of the image area; therefore, the image matting process further includes: and removing the preliminary image frame, and constructing a target image frame corresponding to the image area in a three-dimensional modeling mode.

The frame obtained by the removal of the matting is in an expansion and corrosion operation mode.

According to the embodiment of the application, the frame is removed by adopting an erode corrosion operation mode, specifically, the boundary points of the commodity main body diagram are removed, so that the boundary points of the commodity main body diagram shrink inwards, and objects smaller than elements of the structural unit are removed.

As shown in fig. 15, in step S1503, a target image contour of the image area and a target size ratio of the target image contour are determined based on the image area.

This step is used to determine a target image contour of the image area and a target size ratio of the target image contour.

The hanging outline extraction can be achieved by adopting a boundary tracking algorithm Suzuki85 on the matting result, wherein the boundary tracking algorithm is used for converting a binary image into a representation form of the boundary and extracting the topological structure among the boundaries.

The image contour extraction processing is performed on the image area to obtain an image contour boundary point corresponding to the image area, which can be realized by the following steps:

determining a connected chain code for describing the boundary points of the image outline; judging whether pixel points used for representing the image contour boundary in the image area are boundary points or not based on the determined connected chain codes; and if so, taking the pixel point as an image contour boundary point corresponding to the image area.

Before the hanging outline extraction operation, a chain code is defined first. Fig. 6 is a schematic diagram of a chain code for determining whether an image contour pixel point is a boundary point according to an embodiment of the present application.

After the connected chain code type is determined, judging whether contour pixel points in the matting structure are boundary points according to the determined connected chain code type, and if so, marking the boundary points on the frame position of each white area in the mask graph after extraction. Here, the boundary tracking algorithm Suzuki85 is used to complete the extraction of the hanging outline, please refer to fig. 8, which is a schematic diagram of the image region matting result and the outline boundary point extraction result of the object in the original image provided in the embodiment of the present application. In fig. 8, a plurality of boundary points are extracted from the border of each white area in the mask map, so as to be used for performing shape fitting according to the boundary points in step S1604, and a target hanging outline after the fitting processing is obtained.

After determining the image contour boundary point of the image area, further comprising: and carrying out fitting treatment on the image contour boundary points to obtain the target image contour after fitting treatment.

The image contour boundary points are subjected to fitting treatment to obtain a target image contour after the fitting treatment, and the fitting treatment can be realized in the following manner:

acquiring a curve formed by connecting all boundary points of the image contour; acquiring a first straight line formed by connecting a first boundary point and a second boundary point from the curve; acquiring a first vertical distance between a first target boundary point in the curve and the first straight line; judging whether the first vertical distance is smaller than or equal to a preset distance threshold value; and if the first vertical distance is smaller than or equal to a preset distance threshold value, taking the first straight line as the target image contour after the fitting processing.

In one implementation of the embodiment of the present application, the boundary point fitting is performed using the dawster-plck algorithm. Please refer to fig. 9, which is a schematic diagram of a fitted image contour according to an embodiment of the present application.

If the first vertical distance d1 is smaller than or equal to the preset distance threshold, it is indicated that the fold line formed by connecting the connecting lines between the boundary points in the curve and the curve belong to the approximate line, and then the fold obtained by connecting the boundary points can be used as the target hanging outline.

The following describes a case where the first vertical distance d1 is greater than the preset distance threshold:

if the first vertical distance is greater than a preset distance threshold, segmenting the first straight line by using the first target boundary point to obtain a second straight line formed by connecting the first boundary point and the first target boundary point and a third straight line formed by connecting the first target boundary point and the second boundary point; acquiring a second vertical distance between a second target boundary point in the curve and the second straight line and a third vertical distance between a third target boundary point in the curve and the third straight line; and if the second vertical distance is smaller than or equal to a preset distance threshold value and the third vertical distance is smaller than or equal to the preset distance threshold value, taking a broken line formed by connecting the second straight line and the third straight line as the main contour of the target image after the fitting processing.

With continued reference to fig. 9, if the first vertical distance d1 is greater than the preset distance threshold, the first line ab is segmented with the first target boundary point c, obtaining the second line ac and the third line bc.

The above description is of the case where the image subject contour boundary points are subjected to fitting processing to obtain a candidate image subject contour as the target image subject contour after the fitting processing.

If the image main body contour boundary points are subjected to fitting processing to obtain a plurality of candidate image main body contours, the method further comprises the following steps: screening and obtaining a target image contour from the obtained multiple candidate image contours; the screening and obtaining the target image contour from the obtained multiple candidate image contours can be realized by the following steps: and screening and obtaining the target image contour from the obtained multiple candidate image contours according to a preset screening condition.

The preset screening conditions comprise at least one of the following screening conditions:

judging whether the number of contour edges contained in the candidate image contours is a preset edge data threshold value or not; judging whether the included angle of the adjacent edges of the candidate image contours is within a preset included angle range value or not; judging whether the ratio between the area of the candidate image contour and the image area is larger than a preset area ratio threshold value or not; and judging whether the shape of the candidate image contour has a concave-convex shape.

And judging whether the number of contour edges contained in the candidate image contours is a preset edge data threshold value or not. In this embodiment of the present application, a hanging image is used, so that the candidate image contour belongs to a rectangle, and here, it is necessary to determine whether contour edge data included in the candidate image contour is 4 edges.

And judging whether the included angle of the adjacent edges of the candidate image contour is within a preset included angle range value. If the included angle of the adjacent edges is a squint angle, the squint angle has perspective phenomenon, but the perspective phenomenon does not exist in the vertical angle of 90 degrees. Therefore, it is determined here whether the angle between two adjacent edges of the candidate image contour is 90 °, and if it is 90 °, no perspective exists between the two contour edges.

And judging whether the ratio between the area of the candidate image contour and the image area is larger than a preset area ratio threshold value. The interior of the candidate image contour is image content, the proportion of the area size of the image content in the whole size of the image is larger than a preset proportion threshold value, so that whether the ratio between the areas of the plurality of candidate image contours and the whole area of the image is larger than a preset area ratio threshold value is judged, and if the ratio is larger than the preset area ratio threshold value, the candidate image contour is the target image contour.

And judging whether the shape of the candidate image contour has a concave-convex shape. If each edge of the candidate image contour obtains a contour edge, connecting the contour edges to obtain an image contour without concave-convex type. If the candidate image contour is formed by one more contour edge, the image contour obtained by connection is a concave contour, and if the candidate image contour is formed by one more contour edge, the image contour obtained by connection is a convex contour. Therefore, it is necessary to determine whether or not the candidate image contour has a concave-convex shape, and when the candidate image contour does not have a concave-convex shape, the candidate image contour is the target image contour.

The embodiment of the application further comprises: adjusting the position distribution areas of a plurality of sub-image areas in the image area and adjusting the position sequence of endpoints corresponding to the distribution of the plurality of sub-image areas in the image area; an image area in which the position distribution area and the position order of the end points are adjusted is obtained.

Please refer to the multi-hanging picture corresponding to the part (c) in fig. 12, which is an image area for splitting a hanging picture into 5 sub-image areas, in a specific image arrangement process, adjusting the position distribution areas of the sub-image areas in the image area, adjusting the position sequences of the endpoints corresponding to the sub-image areas respectively, and then completing the image areas with the position distribution and the endpoint position sequence ordering.

In addition, after determining an image contour boundary point and a target image contour, determining a pixel length ratio of the target image contour according to the image contour boundary point, and determining a target size ratio of the target image contour according to the pixel length ratio and the actual size ratio of the target image contour.

Wherein, the determining the target size ratio of the target image contour according to the pixel length ratio and the actual size ratio of the target image contour may be implemented as follows:

Acquiring two coordinate values of the image contour boundary point in a pixel coordinate system; taking a first coordinate value of the two coordinate values as width data and a second coordinate value as height data, and calculating a first pixel length ratio of the image contour boundary point in a pixel coordinate system; taking a first coordinate value of the two coordinate values as height data and a second coordinate value as width data, and calculating a second pixel length ratio of the image main body contour boundary point in a pixel coordinate system; and respectively judging the similarity between the first pixel length ratio and the actual size ratio and the similarity between the first pixel length ratio and the second pixel length ratio, wherein the pixel length ratio with the similarity being larger than or equal to a preset similarity threshold value is used as the target size ratio of the image contour.

Taking the hanging picture in fig. 12 as an example, the hanging picture size in the main body content of the hanging picture commodity is matched, when the two coordinate values representing the single hanging picture are respectively wide x high and high x wide are judged by the cross judgment method, the obtained two pixel length ratios are compared, and then the pixel length ratio is compared with the actual size ratio, so that the actual size of the hanging picture is obtained.

As can be seen from the single hanging picture corresponding to the part (a) in fig. 12, the ratio of the pixel length of the image is that the ratio of the height pixel of the image to the width pixel of the image is 2:1.

as can be seen from the pixel length ratio of the single hanging picture corresponding to the part (a) in fig. 12, the vertical axis height data is 0.8, and the horizontal axis width data is 0.4.

In addition, if the image area contains a plurality of sub-image areas, a multi-combination image size matching model is adopted to determine the target size ratio of the image outlines corresponding to the sub-image areas in the image area.

For example, the hanging drawings corresponding to the part (b) in fig. 12 and the hanging drawings corresponding to the part (c) in fig. 12 belong to a plurality of hanging drawings. When determining the target pixel length ratio of the multiple hanging images, a multiple-combination mode type size generation method can be adopted to determine the pixel length ratio of the multiple-combination hanging images.

In the drawing corresponding to the portion (b) in fig. 12, the vertical axis height data of the three drawings are the same, and the horizontal axis width data of the middle image is wider than the horizontal axis width data of the left and right images, based on the coincidence of the left and right drawings.

The size of the coordinate data provided in the hanging chart corresponding to the part (c) in fig. 12 is the overall size describing the hanging chart image combination, specifically, the overall horizontal axis width data of the hanging chart image combination is 1.0, and the overall vertical axis height data is 0.55. Alternatively, the overall horizontal axis width data of the hanging image combination is 1.5, and the overall vertical axis height data is 0.8. Alternatively, the overall horizontal axis width data of the hanging image combination is 2.0, and the overall vertical axis height data is 1.1.

As can be seen from the image in the hanging chart corresponding to the portion (c) in fig. 12, the overall pixel length ratio of the graph combination composed of 5 sub-images is 2:1.

as can be seen from the three sets of coordinate data ratios provided in the hanging image corresponding to the portion (c) in fig. 12, the overall horizontal axis width data of the hanging image combination is 2.0, and the overall vertical axis height data is 1.1, which is the ratio of the overall horizontal axis length data of the combined image to the vertical axis height data of the image.

Fig. 20 depicts a flow chart for determining the target size ratio by combining multiple sets of images, which are determined in three ways. In the first case, as for the hanging picture composed of single hanging pictures provided in the part (a) in fig. 12, the target size ratio of the single hanging picture is determined, the pixel length ratio of the image can be determined according to the image, then, the ratio of the plurality of horizontal axis width coordinate data and the vertical axis height coordinate data provided in the part (a) in fig. 12 is determined whether the ratio belongs to the pixel length ratio range of the image, and the ratio of the hanging picture horizontal axis width coordinate data and the vertical axis height coordinate data belonging to the pixel length ratio range of the image is determined.

In the second case, as shown in the hanging picture corresponding to the part (b) in fig. 12, in the hanging picture combination composed of 3 hanging pictures, the 3 hanging pictures are formed by combining 3 individual hanging pictures, and as shown in the hanging picture corresponding to the part (b) in fig. 12, the ratio of the pixel length and the dimension length in the width-height direction of the three individual hanging pictures is known. The vertical axis height data of the three sub-images are equal and the horizontal axis width data of the first sub-image and the third sub-image are the same. Therefore, after the ratio is obtained, a plurality of size data are generated according to the ratio, whether the ratio requirement is met or not is judged according to the plurality of size data, and then the size data corresponding to the plurality of hanging painting combinations are determined.

As shown in fig. 15, in step S1504, a three-dimensional mesh model of the object is generated according to the image area of the object in the original image, the target image contour, and the target size ratio, wherein the three-dimensional mesh model of the object is a three-dimensional geometric map of the object.

The three-dimensional grid model of the object is generated according to the image area of the object in the original image, the target image outline and the target size ratio.

The generating of the three-dimensional grid model of the object according to the image area of the object in the original image, the target image contour and the target size ratio can be realized by the following steps:

determining endpoint position coordinates of the object according to the coordinate size of the image area, the coordinate size of the target image outline and the target size ratio; and carrying out triangulation processing on the endpoint position coordinates of the object to obtain a three-dimensional grid model of the object.

Second embodiment

Fig. 16 is a schematic diagram of a device for generating a three-dimensional mesh model of an object according to a second embodiment of the present application. The apparatus for generating a three-dimensional mesh model of an object provided in this embodiment is described in detail below with reference to fig. 16. The generating device of the three-dimensional mesh model of the object provided in the second embodiment corresponds to the generating method of the three-dimensional mesh model of the object provided in the first embodiment, and the specific description process may refer to the descriptions of the above-mentioned scene embodiment and the method embodiment, which are not repeated herein.

As shown in fig. 16, the apparatus for generating a three-dimensional mesh model of an object includes:

an acquisition unit 1601 for acquiring an original image of an object;

an image region obtaining unit 1602, configured to perform matting processing on an original image of the object, to obtain an image region of the object in the original image;

a determining unit 1603 for determining a target image contour of the image area and a target size ratio of the target image contour based on the image area;

the three-dimensional grid model generating unit 1604 is configured to generate a three-dimensional grid model of the object according to the image area of the object in the original image, the target image contour and the target size ratio, where the three-dimensional grid model of the object is a three-dimensional geometric figure of the object.

Third embodiment

Corresponding to the scene embodiment and the first embodiment of the present application, a third embodiment of the present application provides a method for obtaining a three-dimensional mesh model of an object, as shown in fig. 17, which is a schematic flow chart of the method for obtaining a three-dimensional mesh model of another object provided in the third embodiment of the present application. The method shown in fig. 17 includes steps S1701-S1703.

As shown in fig. 17, in step S1701, identification information of an object is obtained;

the method comprises the steps of acquiring identification information of objects, wherein the identification information of the objects is used for representing identity authentication uniqueness of the objects, and each object corresponds to one piece of identification information. For example, each hanging image corresponds to one piece of identification information, and the original image of the hanging image is queried according to the identification information of the hanging image.

As shown in fig. 17, in step S1702, an original image of the object is obtained based on the identification information of the object;

this step is used to obtain an original image of the object, so that a three-dimensional mesh model of the object, that is, a three-dimensional geometric figure of the object, is obtained from the original image of the object. The specific acquisition method is acquired in step S1703.

Wherein each object corresponds to one piece of identification information, for example, one piece of identification information for each hanging picture.

As shown in fig. 17, in step S1703, according to the original image, a three-dimensional mesh model of an object including the original image is obtained using the method provided by the first embodiment, wherein the three-dimensional mesh model of the object is a three-dimensional geometric figure of the object.

This step is used to obtain a three-dimensional mesh model of an object according to the method for generating a third-order mesh model of an object provided in the first embodiment. The specific method for obtaining the original image of the hanging picture according to the identification information may be obtaining a single Zhang Yuanshi image of the hanging picture. The embodiment of the application adopts a single Zhang Yuanshi image to generate a three-dimensional grid model of the object. Specifically, a single Zhang Yuanshi image of the hanging picture is obtained according to the identification information of the hanging picture, the single Zhang Yuanshi image of the hanging picture is subjected to matting processing to obtain an image area of the hanging picture, in other words, the main image content of the hanging picture is separated from the background content of the hanging picture, and the area for describing the main content information of the hanging picture in the hanging picture is subjected to buckling processing to obtain the image area of the hanging picture.

Then, carrying out image contour extraction processing on the hung image area to obtain image contour boundary points of the hung image area; fitting the image contour boundary points to obtain a target image contour after fitting; and determining the pixel length ratio of the target image contour according to the image contour boundary points, and determining the target size ratio of the target image contour according to the pixel length ratio and the actual size ratio of the target image contour.

And after the target image contour of the hanging image and the target size ratio of the target image contour are obtained, generating a three-dimensional grid model of the hanging image according to the image area of the hanging image, the target image contour and the target size ratio.

In a third embodiment of the present application, identification information of an object is obtained, and after an original image of the object is determined according to the identification information of the object, a three-dimensional mesh model of the object is obtained by using the method for generating a three-dimensional mesh model of the object provided in the first embodiment. According to the method, the three-dimensional grid model of the object is generated according to the single Zhang Yuanshi image of the object, so that the generation efficiency of the three-dimensional grid model of the object is improved. In addition, the target size ratio of the target image contour is obtained, and the three-dimensional grid model of the object is generated according to the image area of the object in the original image and the target image contour, and the target size ratio of the target image contour is referred to, so that the matching rate of the three-dimensional grid model of the object and the actual object is improved.

Fourth embodiment

Fig. 18 is a schematic diagram of an apparatus for obtaining a three-dimensional mesh model of an object according to a fourth embodiment of the present application. The apparatus for obtaining a three-dimensional mesh model of an object provided in this embodiment is described in detail below with reference to fig. 18. The device for obtaining the three-dimensional mesh model of the object provided in the fourth embodiment corresponds to the method for obtaining the three-dimensional mesh model of the object provided in the third embodiment, and the specific description process may refer to the descriptions of the above-mentioned scene embodiment and the method embodiment, which are not repeated herein.

An identification information obtaining unit 1801 for obtaining identification information of an object;

an original image obtaining unit 1802 configured to obtain an original image of the object based on identification information of the object;

a three-dimensional mesh model obtaining unit 1803, configured to obtain, according to the original image, a three-dimensional mesh model of an object including the original image, using the method provided by the first embodiment, where the three-dimensional mesh model of the object is a three-dimensional geometric figure of the object.

Fifth embodiment

The fifth embodiment of the present application also provides an electronic device corresponding to the methods of the first embodiment and the third embodiment of the present application. As shown in fig. 19, fig. 19 is a schematic view of an electronic device according to a fifth embodiment of the present application. The electronic device includes: at least one processor 1901, at least one communication interface 1902, at least one memory 1903, and at least one communication bus 1904; alternatively, the communication interface 1902 may be an interface of a communication module, such as an interface of a GSM module; the processor 1901 may be a processor CPU or a particular integrated circuit ASIC (Application Specific Integrated Circuit) or one or more integrated circuits configured to implement embodiments of the present invention. The memory 1903 may include high-speed RAM memory or may further include non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. In which a program is stored in the memory 1903, and the processor 1901 calls the program stored in the memory 1903 to execute the methods of the first and third embodiments of the present invention.

Sixth embodiment

The sixth embodiment of the present application also provides a computer storage medium corresponding to the methods of the first and third embodiments of the present application. The computer storage medium stores a computer program that is executed by a processor to perform the methods of the first and third embodiments.

While the preferred embodiment has been described, it is not intended to limit the invention thereto, and any person skilled in the art may make variations and modifications without departing from the spirit and scope of the present invention, so that the scope of the present invention shall be defined by the claims of the present application.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

1. Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable Media, as defined herein, does not include non-Transitory computer-readable Media (transmission Media), such as modulated data signals and carrier waves.

2. It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. It should be noted that, in the embodiments of the present application, the use of user data may be involved, and in practical applications, user specific personal data may be used in the schemes described herein within the scope allowed by applicable legal regulations in the country where the applicable legal regulations are met (for example, the user explicitly agrees to the user to actually notify the user, etc.).

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region, and provide corresponding operation entries for the user to select authorization or rejection.

Claims

1. A method for generating a three-dimensional mesh model of an object, comprising:

acquiring an original image of an object;

carrying out matting processing on an original image of the object to obtain an image area of the object in the original image;

determining a target image contour of the image area and a target size ratio of the target image contour based on the image area;

and generating a three-dimensional grid model of the object according to the image area of the object in the original image, the target image contour and the target size ratio, wherein the three-dimensional grid model of the object is a three-dimensional geometric figure of the object.

2. The method of claim 1, wherein the determining a target image contour of the image region and a target size ratio of the target image contour based on the image region comprises:

carrying out image contour extraction processing on the image area to obtain image contour boundary points corresponding to the image area;

fitting the image contour boundary points to obtain a fitted target image contour;

and determining the pixel length ratio of the target image contour according to the image contour boundary points, and determining the target size ratio of the target image contour according to the pixel length ratio and the actual size ratio of the target image contour.

3. The method of claim 1, wherein the image region of the object in the original image obtained comprises a preliminary image border of the image region;

the method further comprises the steps of:

and removing the preliminary image frame, and constructing a target image frame corresponding to the image area in a three-dimensional modeling mode.

4. The method according to claim 2, wherein the performing image contour extraction processing on the image area to obtain an image contour boundary point corresponding to the image area includes:

determining a connected chain code for describing the boundary points of the image outline;

judging whether pixel points used for representing the image contour boundary in the image area are boundary points or not based on the determined connected chain codes;

and if so, taking the pixel point as an image contour boundary point corresponding to the image area.

5. The method according to claim 2, wherein the fitting the image contour boundary points to obtain a target image contour after the fitting process includes:

acquiring a curve formed by connecting all boundary points of the image contour;

acquiring a first straight line formed by connecting a first boundary point and a second boundary point from the curve;

Acquiring a first vertical distance between a first target boundary point in the curve and the first straight line;

judging whether the first vertical distance is smaller than or equal to a preset distance threshold value;

and if the first vertical distance is smaller than or equal to a preset distance threshold value, taking the first straight line as the target image contour after the fitting processing.

6. The method as recited in claim 5, further comprising:

if the first vertical distance is greater than a preset distance threshold, segmenting the first straight line through the first target boundary point to obtain a second straight line formed by connecting the first boundary point and the first target boundary point and a third straight line formed by connecting the first target boundary point and the second boundary point;

acquiring a second vertical distance between a second target boundary point in the curve and the second straight line and a third vertical distance between a third target boundary point in the curve and the third straight line;

and if the second vertical distance is smaller than or equal to a preset distance threshold value and the third vertical distance is smaller than or equal to the preset distance threshold value, taking a broken line formed by connecting the second straight line and the third straight line as the target image contour after the fitting processing.

7. The method as recited in claim 1, further comprising: screening and obtaining a target image contour from the obtained multiple candidate image contours;

the screening and obtaining the target image contour from the obtained multiple candidate image contours comprises the following steps:

and screening and obtaining the target image contour from the obtained multiple candidate image contours according to a preset screening condition.

8. The method of claim 7, wherein the predetermined screening conditions include at least one of the following:

judging whether the number of contour edges contained in the candidate image contours is a preset edge data threshold value or not;

judging whether the included angle of the adjacent edges of the candidate image contours is within a preset included angle range value or not;

judging whether the ratio between the area of the candidate image contour and the image area is larger than a preset area ratio threshold value or not;

and judging whether the shape of the candidate image contour has a concave-convex shape.

9. The method as recited in claim 1, further comprising:

adjusting the position distribution areas of a plurality of sub-image areas in the image area and adjusting the position sequence of endpoints corresponding to the distribution of the plurality of sub-image areas in the image area;

An image area in which the position distribution area and the position order of the end points are adjusted is obtained.

10. The method of claim 2, wherein said determining a target size ratio of said target image contour from a pixel length ratio and an actual size ratio of said target image contour comprises:

acquiring two coordinate values of the image contour boundary point in a pixel coordinate system;

taking a first coordinate value of the two coordinate values as width data and a second coordinate value as height data, and calculating a first pixel length ratio of the image contour boundary point in a pixel coordinate system;

taking a first coordinate value of the two coordinate values as height data and a second coordinate value as width data, and calculating a second pixel length ratio of the image main body contour boundary point in a pixel coordinate system;

and respectively judging the similarity between the first pixel length ratio and the actual size ratio and the similarity between the first pixel length ratio and the second pixel length ratio, wherein the pixel length ratio with the similarity being larger than or equal to a preset similarity threshold value is used as the target size ratio of the image contour.

11. The method as recited in claim 1, further comprising:

And if the image area comprises a plurality of sub-image areas, determining the target size ratio of the image outlines corresponding to the sub-image areas in the image area by adopting a multi-combination image size matching model.

12. The method of claim 1, wherein the generating a three-dimensional mesh model of the object from the image region of the object in the original image, the target image contour, and the target size ratio comprises:

determining endpoint position coordinates of the object according to the coordinate size of the image area, the coordinate size of the target image outline and the target size ratio;

and carrying out triangulation processing on the endpoint position coordinates of the object to obtain a three-dimensional grid model of the object.

13. A method of obtaining a three-dimensional mesh model of an object, comprising:

obtaining identification information of an object;

acquiring an original image of the object based on the identification information of the object;

from the original image, a three-dimensional mesh model of an object comprising the original image is obtained using the method of claim 1, wherein the three-dimensional mesh model of the object is a three-dimensional geometric figure of the object.

14. An electronic device comprising a processor and a memory;

the memory having stored therein a computer program which, when executed by the processor, performs the method of any of claims 1-13.

15. A computer storage medium, characterized in that the computer storage medium stores a computer program which, when executed by a processor, performs the method of any of claims 1-13.