KR102547323B1 - Object segmentation device and method formed through 3d scanning - Google Patents

Abstract

An apparatus for segmenting an object formed through 3D scanning according to an embodiment of the present invention includes a memory, at least one processor communicating with the memory, an external device performing a 3D scanning function, and a communication module communicating with a user terminal. The processor receives a scan object, which is a scan object, and a texture image representing a texture of the scan object from the external device, and receives the texture through supervised-learning Based on the image, a segment representing a desired part of the object is extracted, the extracted segment is masked with a predetermined specific color, and the texture image including the scan object and the masked segment is extracted. 3D modeling data in which the object is three-dimensionally modeled is formed, and through comparison with attribute values for the specific color masked based on mesh data including attribute values of the 3D modeling data, the 3D data for segments can be extracted.

Description

Object segmentation device and method formed through 3D scanning {OBJECT SEGMENTATION DEVICE AND METHOD FORMED THROUGH 3D SCANNING}

The present invention relates to an apparatus for segmenting an object, and more particularly, to a method for segmenting an object formed through 3D scanning.

Recently, various shapes have been created using 3D printing technology. In particular, in relation to the field of nail art, the popularity of nail tips is increasing.

Unlike conventional methods in which nail tips are often manually molded and cut, 3D printing technology can be used to produce user-customized nail tips.

The 3D scanning process captures data points of the surface shape of an object to create a digital model, which is then analyzed and processed by the apparatus and method of the present invention. Software algorithms in the 3D scanning process use mathematical calculations and data analysis techniques to identify and isolate individual segments of an object, which can be further processed or modified as needed.

Conventionally, there has been a problem in that it may be difficult to accurately acquire a desired part of a complex object having an irregular shape or feature through manual segmentation technology. Accordingly, there has been a demand for an ability to accurately and efficiently divide a region desired by a user.

That is, in order to manufacture a user-customized nail tip, a segmentation technology for clearly acquiring only the human fingernail portion through 3D scanning and a 3D printing technology for manufacturing a user-customized nail tip for the acquired human fingernail portion are required. .

Registered Patent No. 10-2259509 (2021.05.27.)

An object of the embodiments disclosed in the present disclosure is to provide an effect capable of accurately segmenting a desired part of an object through 3D scanning, artificial intelligence learning, and 3D modeling.

An object of the embodiment disclosed in the present disclosure is to provide an effect of providing a user-customized nail tip through 3D printing technology using data on a segmented portion of an object.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

An apparatus for segmenting an object formed through 3D scanning according to an aspect of the present disclosure for achieving the above technical problem is,

Memory,

at least one processor in communication with the memory; and

Including a communication module for communicating with an external device and a user terminal that performs a 3D scan function,

the processor,

Receiving a scan object, which is a scan object, and a texture image representing a texture of the scan object from the external device;

Extracting a segment representing a desired part of an object based on the texture image through supervised-learning,

Masking the extracted segments with a predetermined specific color,

Forming 3D modeling data in which the object is three-dimensionally modeled based on the texture image including the scan object and the masked segments;

3D data for the segment may be extracted through comparison with attribute values of the specific color masked based on mesh data including attribute values of the 3D modeling data.

At this time, the texture of the scan object,

It may be 3-dimensionally implemented by constructing a point cloud of the object based on at least one texture image.

In addition, in relation to the supervised learning,

Labeling the object data including the part to be segmented and the segment part data as training data,

Batch is made to the learning model based on the labeled training data,

When the texture image is input from the user and the segment is extracted,

Based on a pre-stored ground truth set, testing and verification of the extraction of the segment is performed,

generate feedback data based on the testing and verification;

Based on the feedback data, parameters of the learning model may be tuned to perform supervised-learning of the segmentation of the object.

In addition, extracting 3D data for the segment,

Assigning color information for the texture image to the mesh data as a vertex color value,

Extracting RGB values for each polygon area of the mesh data formed based on the vertex color values;

Comparing RGB values for each polygon area of the mesh data with RGB values for the specific color masked,

When the RGB values of the polygon area match the RGB values of the specific color, it may be performed by extracting the polygon area with matching RGB values.

On the other hand, in the segmentation device of an object formed through 3D scanning according to one aspect of the present disclosure for achieving the above-described technical problem,

The masking is

An area of the object excluding only the portion related to the segment,

The 3D modeling data,

It is formed based on the texture image including an area excluding the scan object and the masked portion related to the segment,

Extracting 3D data for the segment,

It may be performed by excluding the masked area through comparison with attribute values for the specific color masked based on mesh data including attribute values of the 3D modeling data.

Meanwhile, in the present invention, the scan object means data on a human hand,

The segment may refer to a region of a fingernail portion of the human hand.

On the other hand, in the present invention, the 3D modeling data,

It may be compiled with one of .stl, .fbx, .dae, .3ds, .abc, .usd and .obj extensions and stored in the memory.

In addition to this, a computer program stored in a computer readable recording medium for execution to implement the present disclosure may be further provided.

In addition to this, a computer readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

According to the above-described problem solving means of the present disclosure, an effect of accurately segmenting a desired part of an object through 3D scanning, artificial intelligence learning, and 3D modeling is provided.

According to the above-described problem solving means of the present disclosure, an effect of providing a user-customized nail tip is provided through 3D printing technology using data on a segmented portion of an object.

According to the above-mentioned problem solving means of the present disclosure, an effect capable of providing high accuracy and consistency in manufacturing a nail tip using 3D printing technology is provided.

According to the above-mentioned problem solving means of the present disclosure, using 3D printing technology can provide advantages in terms of speed and efficiency of manufacturing nail tips, and simplifies the production process compared to traditional methods that may require a significant amount of manual work and time. and has the effect of reducing the amount of waste generated.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram schematically illustrating a segmentation system of an object formed through 3D scanning according to an embodiment.
2 is a diagram schematically illustrating components of an apparatus for segmenting an object formed through 3D scanning according to an exemplary embodiment.
3 is a flowchart illustrating a segmentation of an object formed through 3D scanning according to an exemplary embodiment.
4A is a diagram schematically illustrating a scan object obtained from an external device performing a 3D scan function according to an exemplary embodiment.
4B is a diagram schematically illustrating a texture image obtained from an external device performing a 3D scan function according to an exemplary embodiment.
5 is a diagram illustrating that a segmentation operation of an object is learned through a deep learning model according to an embodiment.
6 is a diagram illustrating that 3D modeling data is formed by combining a scan object and a masked texture image according to an exemplary embodiment.
7A and 7B are diagrams schematically illustrating a process of extracting a segment from a specific color extraction or removal algorithm according to an embodiment.
8 is a diagram schematically showing the structure of mesh data for 3D modeling data according to an embodiment.
9 is a diagram illustrating that a user-customized nail tip can be manufactured using extracted nail data according to an embodiment.

Like reference numbers designate like elements throughout this disclosure. The present disclosure does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the present disclosure belongs is omitted. The term 'unit, module, member, or block' used in the specification may be implemented as software or hardware, and according to embodiments, a plurality of 'units, modules, members, or blocks' may be implemented as one component, It is also possible that one 'part, module, member, block' includes a plurality of components. Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case of being directly connected but also the case of being indirectly connected, and indirect connection includes being connected through a wireless communication network. do.

In addition, when a certain component is said to "include", this means that it may further include other components without excluding other components unless otherwise stated.

Throughout the specification, when a member is said to be located “on” another member, this includes not only a case where a member is in contact with another member, but also a case where another member exists between the two members.

Terms such as first and second are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

Expressions in the singular number include plural expressions unless the context clearly dictates otherwise.

In each step, the identification code is used for convenience of description, and the identification code does not explain the order of each step, and each step may be performed in a different order from the specified order unless a specific order is clearly described in context. there is.

Hereinafter, the working principle and embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a diagram schematically illustrating a segmentation system of an object formed through 3D scanning according to an embodiment.

Referring to FIG. 1, the segmentation system of an object formed through 3D scanning includes an object segmentation device 100 formed through 3D scanning, an external device 200 performing a 3D scanning function, and a user in order to perform the operation of the present invention. It can be configured by the terminal 300.

Referring to FIG. 1 , an apparatus 100 for segmenting an object formed through 3D scanning may communicate with an external device 200 performing a 3D scanning function and a user terminal 300 .

In this specification, the 'device 100 for segmenting an object formed through 3D scanning according to the present disclosure' includes various devices capable of providing results to users by performing calculation processing. For example, the apparatus 100 for segmenting an object formed through 3D scanning according to the present disclosure may include a computer, a server device, and a portable terminal, or may be in any one form.

The device (server) 100 is a server that processes information by communicating with the external device 200 and the user terminal 300, and includes an application server, a computing server, a database server, a file server, a game server, a mail server, May include proxy servers and web servers.

In this specification, the external device 200 that performs the 3D scan function may be any one of all external devices (input devices) that perform the 3-scan function, including a 3D scanner.

An input device such as the external device 200 performing a 3D scan function is for inputting image information (or signals), audio information (or signals), data, or information input from a user, and includes at least one camera, It may include at least one of at least one microphone and a user input unit. In the external device 200 performing the 3D scan function, image data may be analyzed and processed as a user's control command.

A camera processes an image frame such as a still image or a moving image obtained by an image sensor in a photographing mode. The processed image frame may be displayed on a display unit or stored in a memory.

On the other hand, when there are a plurality of cameras, they can be arranged to form a matrix structure, and a plurality of image information having various angles or focal points can be input through the cameras forming the matrix structure, and the cameras form a three-dimensional image. It may be arranged in a stereo structure so as to obtain left and right images for realizing a stereoscopic image.

Meanwhile, a 3D scanner may refer to a device that captures the shape, texture, and color of an object and creates a three-dimensional digital model thereof. 3D scanners can work by emitting light or lasers at objects that are reflected off the scanner's sensors. The 3D scanner's sensor records the time it takes for light to return to the subject and can calculate the distance to the object's surface. A scanner can scan an object from different angles to build a point cloud of data representing the shape of the object.

Meanwhile, after the 3D scanner has collected the point cloud, software can be used to process the data and create a 3D model through 3D modeling. The software can also apply texture and color information to the model to provide a true representation of the object. The resulting digital models can be used for a variety of purposes, such as 3D printing, reverse engineering or virtual reality.

3D scanners may include various types of 3D scanners, including handheld scanners, desktop scanners, and structured light scanners. Each type has its own strengths and limitations, and your choice of 3D scanner may depend on your specific application and requirements.

The sensing unit of the external device 200 performing the 3D scan function senses at least one of information in the external device 200, surrounding environment information surrounding the external device 200, and user information, and generates a sensing signal corresponding thereto, , information on the sensing signal may be transmitted to the apparatus 100. Based on these sensing signals, the controller of the device 100 uses data received from the external device 200 that performs the 3D scan function to process data, functions, or operations related to the application program installed in the device 100. can be performed.

In addition, the external device 200 performing the 3D scan function may directly communicate with the user terminal 300 through wired communication and/or wireless communication according to its own communication module.

As described above, the sensing unit of the external device 200 includes a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and a gravity sensor ( G-sensor), gyroscope sensor, motion sensor, RGB sensor, infrared sensor, finger scan sensor, ultrasonic sensor, optical sensor Sensor (optical sensor (eg, camera), microphone, environmental sensor (eg, including at least one of a barometer, hygrometer, thermometer, radiation detection sensor, heat detection sensor, gas detection sensor), chemical sensor (eg For example, at least one of a healthcare sensor, a biometric sensor, and the like). Meanwhile, the device 100 may receive information sensed by at least two or more of these sensors, and may combine and utilize them.

In addition, the user terminal 300 may include both a computer and a portable user terminal, or may be of any one type.

Here, the computer may include, for example, a laptop computer, a desktop computer, a laptop computer, a tablet PC, a slate PC, and the like equipped with a web browser.

The portable terminal is, for example, a wireless communication device that ensures portability and mobility, and includes a Personal Communication System (PCS), a Global System for Mobile communications (GSM), a Personal Digital Cellular (PDC), a Personal Handyphone System (PHS), and a PDA. (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, smart phone ) and wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted-devices (HMDs). can include

2 is a diagram schematically illustrating components of an apparatus for segmenting an object formed through 3D scanning according to an exemplary embodiment.

Referring to FIG. 2 , an apparatus 100 for segmenting an object formed through 3D scanning according to the present disclosure may include a memory 110, a processor 120, and a communication module 130 (or communication unit).

The components shown in FIG. 2 are not essential in implementing the segmentation apparatus 100 of an object formed through 3D scanning according to the present disclosure, so that the segmentation apparatus 100 of an object formed through 3D scanning described herein may have more or fewer components than those listed above.

The memory 110 may store data supporting various functions of the present control device 100 and programs for operation of the processor 120, and input/output data (eg, music files, still images). , videos, etc.), and can store a plurality of application programs (application programs or applications) driven by the control device 100, data for operation of the control device 100, and commands. At least some of these application programs may be downloaded from an external server through wireless communication.

The memory 110 may be a flash memory type, a hard disk type, a solid state disk type, a silicon disk drive type, or a multimedia card micro type. micro type), card type memory (eg SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable EEPROM (EEPROM) It may include a storage medium of at least one type of a programmable read-only memory (PROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory 110 is separated from the apparatus 100, but may be a database connected by wire or wirelessly.

The controller of the control device 100 stores data for an algorithm or a program that reproduces the algorithm for controlling the operation of the components in the device, and the memory 110 communicates with the memory 110 to store data in the memory. It may be implemented as at least one processor 120 that performs the above-described operation using stored data. In this case, the memory 110 and the processor 120 may be implemented as separate chips. Alternatively, the memory 110 and the processor 120 may be implemented as a single chip.

In addition, the processor 120 may control any one or a combination of a plurality of the components described above in order to implement various embodiments according to the present disclosure described in FIGS. 3 to 9 below on the device 100. can

Among the components, the communication module 130 may include one or more components enabling communication with the external device 200 performing the 3D scan function and the user terminal 300, for example, broadcasting It may include at least one of a receiving module, a wired communication module, a wireless communication module, a short-distance communication module, and a location information module.

The wireless communication module of the present device 100 includes, in addition to a WiFi module and a wireless broadband module, GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), It may include a wireless communication module supporting various wireless communication schemes such as universal mobile telecommunications system (UMTS), time division multiple access (TDMA), long term evolution (LTE), 4G, 5G, and 6G.

The wireless communication module of the present device 100 may include a wireless communication interface including an antenna and a transmitter for transmitting mobile communication signals. In addition, the wireless communication module may further include a signal modulation module that modulates the digital control signal output from the control unit through the wireless communication interface into an analog type wireless signal under the control of the processor 120 .

The wireless communication module of the device 100 may include a wireless communication interface including an antenna and a receiver for receiving mobile communication signals. In addition, the wireless communication module may further include a signal demodulation module for demodulating an analog type of wireless signal received through a wireless communication interface into a digital control signal.

The wired communication module of the device 100 includes not only various wired communication modules such as a local area network (LAN) module, a wide area network (WAN) module, or a value added network (VAN) module. , Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Digital Visual Interface (DVI), recommended standard 232 (RS-232), powerline communications, or plain old telephone service (POTS). can

At least one component may be added or deleted corresponding to the performance of the components shown in FIG. 2 . In addition, it will be easily understood by those skilled in the art that the mutual positions of the components may be changed corresponding to the performance or structure of the system.

Meanwhile, each component shown in FIG. 2 may mean software and/or hardware components such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

3 is a flowchart illustrating a segmentation of an object formed through 3D scanning according to an exemplary embodiment.

Referring to FIG. 3 , S310 may refer to a step of receiving a scan object, which is a scan object, and a texture image representing a texture of the scan object from an external device performing a 3D scan function.

Specifically, the device 100 may receive data on a 'scan object', which means a target of scanning, from a 3D scanner, which is an external device 200 that performs a 3D scanning function.

In addition, the device 100 receives data for a 'texture image', which means an image capable of expressing the texture of a scanned object, from a 3D scanner, which is an external device 200 that performs a 3D scan function. can receive

In the case of a 3D optical scanner, the function of measuring the distance to a subject (object) through a distance measurement sensor and the function of taking a picture of a subject (which may mean a target of scanning) through a lens for acquiring a texture image can be done

Therefore, the device 100 communicates with the 3D scanner, which is the external device 200 that performs the 3D scan function, through the communication module 130 to perform a 'scan object' and A 'texture image' for representing the texture of a subject may be received.

S320 may refer to a step of extracting a segment representing a desired part of an object based on a texture image through supervised learning.

Specifically, the present device 100 can build labeling data as learning data through an artificial intelligence learning module and learn to acquire desired data.

That is, the apparatus 100 takes a texture image as an input value through an artificial intelligence learning module, and uses a 'segment', which can mean a region of a desired part in the texture image, as an output value to perform deep learning (Deep Learning). Segmentation with learning progressed can be performed. Meanwhile, in this specification, segmentation may mean acquiring a desired part of an object (subject) by a user.

A deep learning process of the apparatus 100 will be described later with reference to FIG. 5 .

S330 may mean a step of masking the extracted segments with a predetermined specific color.

Specifically, S330 may refer to a step of masking the segment area with RGB values of a specific color previously determined by the user when a segment desired by the user is obtained from the texture image.

Here, as a method using color information, it may be performed by selecting any one of color expression methods such as cmyk and hex code (#00000) as well as RGB values.

S330, as will be described later, is a step that can be utilized to extract only the segment part from the 3D modeling data in S350, storing the RGB values of a specific color given by the user in the memory 110, and storing the related data in the step of S350. can be utilized

S340 may refer to a step of forming 3D modeling data in which the object is three-dimensionally modeled based on the texture image including the scanned object and the masked segments.

Specifically, the processor 120 of the present device 100 combines a scan object with a texture image including a segment on which a masking process has been performed to perform '3D modeling', which may mean data in which an object (subject) is three-dimensionally modeled. data can be formed.

S350 may refer to a step of extracting 3D data for a segment through comparison with attribute values of a specific color masked based on mesh data including attribute values of 3D modeling data. Here, the mesh data including attribute values of 3D modeling data may include attribute values of 3D modeling data having attribute values of a texture image.

Specifically, the 3D modeling data may have mesh data including unique attribute values of the data. At this time, the color information included in the mesh data of the 3D modeling data and the color information for a specific color given by the user when masking to the segment in step S330 are compared to extract the 3D data of the segment, which is the area desired by the user, from the 3D modeling data can do. This will be described in detail with reference to FIGS. 7A and 7B as will be described later.

Meanwhile, in relation to S310 to S350, according to an embodiment of the present invention, masking is a region excluding only the segment portion from an object, and 3D modeling data is a texture image including a scan object and an area excluding the masked segment portion. Formed based on, extracting the 3D data for the segment is by excluding the masked area through comparison with the attribute value for the specific color masked based on the mesh data including the attribute value of the 3D modeling data. may be performed.

4A is a diagram schematically illustrating a scan object obtained from an external device performing a 3D scan function according to an exemplary embodiment.

In the case of a 3D optical scanner as an external device 200 that performs a 3D scan function, a function for measuring the distance to a subject (object) through a distance measuring sensor and a subject (meaning the target of scanning) through a lens for acquiring a texture image can perform the function of taking a picture of).

Referring to FIG. 4A , according to an embodiment of the present invention, an object may represent a case of a 'human hand'.

That is, the 3D optical scanner itself has a distance measurement sensor, emits light or laser, records the time it takes for the light to return to the object, and calculates the distance to the surface of the 'human hand', which is the object.

In this way, when the distance to the surface of the object 'human hand' is measured, the shape D410 of the object (human hand) may be formed by performing a 3D scan function based on the distance data.

4B is a diagram schematically illustrating a texture image obtained from an external device performing a 3D scan function according to an exemplary embodiment.

In the case of a 3D optical scanner as an external device 200 that performs a 3D scan function, a point cloud of data representing the shape of an object by scanning an object from various angles (meaning a set of data points belonging to a 3D space). ) to express the texture of the object.

Referring to FIG. 4B , according to an embodiment of the present invention, an object may represent a case of 'a person's hand'.

The external device 200 that performs the 3D scan function to represent the texture of the 'human hand', which is an object, may build a point cloud of object data from various angles.

That is, the optical 3D scanner can build a point cloud by performing a function of taking photos of 'human hands' from various angles through its own lens for acquiring texture images.

For example, referring to FIG. 4B , a first texture image D421 photographed at a first angle as a plurality of texture images D420 from a lens for acquiring a texture image in order to express the texture of a 'human hand', which is an object, A second texture image D422 taken from a second angle, a third texture image D423 taken from a third angle, a fourth texture image D424 taken from a fourth angle, and a fifth taken from a fifth angle. A texture image D425 may be acquired.

3D modeling data may be formed by combining a plurality of texture image data such as D421 to D425 with scan object data. Here, the number of texture images may be at least one, as well as five as in D421 to D425.

5 is a diagram illustrating that a segmentation operation of an object is learned through a deep learning model according to an embodiment.

Referring to FIG. 5 , training data may be object data including a portion to be segmented and data related to the segmented portion.

Referring to FIG. 5 , the learning model may perform supervised-learning by labeling the object data including the portion to be segmented and the segment portion data (S510).

The learning model may mean an artificial intelligence model learned based on deep learning, and for example, may mean a model learned using a convolutional neural network (CNN). However, it is not limited to the above learning model, Natural Language Processing (NLP), Random Forest (RF), Support Vector Machine (SVC), eXtra Gradient Boost (XGB), Decision Tree (DC), Knearest Neighbors (KNN), Gaussian At least one algorithm of Naive Bayes (GNB), Stochastic Gradient Descent (SGD), Linear Discriminant Analysis (LDA), Ridge, Lasso, and Elastic net may be included.

Referring to FIG. 5 , when the training data set is learned according to the machine learning algorithm in the learning model, a batch may be made (S520).

Subsequently, when a plurality of texture images D510 are input from the user to the artificial intelligence module (S530), object segmentation is performed based on the learning data transmitted to the artificial intelligence module, and the segments can be output (S540).

Data on the output segment is tested and verified through comparison with a pre-stored ground truth set, and the apparatus 100 may form feedback information based on the test and verification (S550). .

The artificial intelligence learning module may change or tune parameters of the learning model based on the feedback information (S570).

Here, the parameter may mean at least one of weight and/or bias of each layer included in the learning model.

Supervised learning is to learn the mapping between inputs and outputs, and is applied when input and output pairs are given as data. For example, when a computer recognizes a license plate at a parking lot entrance, it may not recognize it correctly if the license plate is dirty. In this case, it is possible to increase the license plate recognition rate by learning various contaminated license plate cases and normal license plates as input and output pairs, respectively.

Meanwhile, referring to FIG. 6 , masking may be performed on the extracted segment or a region excluding the segment (S560).

That is, when a segment, which is an area desired by a user, is extracted by inputting a texture image, the segment may be masked with a specific color previously determined by the user (D521).

In addition, when a segment, which is an area desired by the user, is extracted by inputting a texture image, the remaining area other than the segment may be masked with a specific color previously determined by the user (D522).

6 is a diagram illustrating that 3D modeling data is formed by combining a scan object and a masked texture image according to an exemplary embodiment.

Referring to FIG. 6 , the processor 120 may form 3D modeling data by combining a scan object and a plurality of texture images in which masking is performed on an extracted segment.

In addition, the processor 120 may form 3D modeling data by combining a plurality of texture images on which masking is performed on areas other than the scan object and the extracted segment.

At this time, the 3D modeling file may be compiled in one of the .fbx, .dae, .3ds, .abc, .usd and .obj extensions, which are the storage formats of the 3D modeling file, and stored in the memory 110.

That is, the 3D modeling data may include information about the mesh data included in the data about the scan object, the texture image representing the texture of the object, and the MTL file in which the 3D object data is compiled.

7A and 7B are diagrams schematically illustrating a process of extracting a segment from a specific color extraction or removal algorithm according to an exemplary embodiment.

Referring to FIG. 7A , from 3D modeling data D710 formed by combining a plurality of masked texture images in an area other than a scan object and an extracted segment, extraction or removal of a specific color previously determined by a user and assigned to a segment A segment D720 corresponding to a portion desired by the user may be extracted using the algorithm S700.

Referring to FIG. 7B, extracting 3D data for a segment by performing segmentation on an object gives color information for a texture image as a vertex color to mesh data (S710 ), and extracts RGB values for each polygon area of the mesh data formed based on the vertex color values (S720), and compares the RGB values for each polygon area of the mesh data with the RGB values for the masked specific color (S730 ), and when the RGB values of the polygon area and the RGB values for a specific color match, extracting the polygon area with matching RGB values (S740).

For example, in order to extract 3D data for the segment by segmenting the 'nail' part, which is the area desired by the user, for the object 'human hand', the mesh data included in the scan object data of 'human hand' Color information can be given by applying information about the texture image to (Mesh data). At this time, the mesh data is composed of a multi-polygon area, and a vertex color can be set by assigning color information of a texture image to a vertex of the polygon area.

Based on the vertex color values given in this way, RGB values may also be set for each face of the polygon region constituting the mesh data of the scan object. At this time, the RGB values of the specific color masked by the user in the segment are compared with the RGB values of each polygon area of the mesh data. If the RGB values of the polygon area and the RGB values of the specific color match, the RGB values match. A segmentation operation of an object may be performed by extracting a corresponding polygon area.

Meanwhile, this operation may be applied similarly to a case in which the rest of the texture image is masked except for a segment desired by the user.

8 is a diagram schematically showing the structure of mesh data (mesh data) for 3D modeling data according to an embodiment.

Referring to FIG. 8 , it may mean that a structure (D800) of mesh data (mesh data) including unique attribute values in 3D modeling data is shown.

Referring to FIG. 8 , the numerical values of 1 to 13 may mean a number D810 indicating a vertex of a polygon area constituting mesh data.

8 may be for explaining an improved structure of mesh data included in 3D modeling data.

Mesh data can consist of a collection of vertices, edges, and faces that define the shape and surface of a 3D model of an object.

The traditional mesh data structure can store the position of each vertex D810 in the 3D space, the connection between each vertex to form an edge, and the connection between edges to form a face, but the improved mesh data structure of the present invention (D800 ) may include additional information about each vertex, including vertex color values.

The vertex color value may refer to a set of RGB (Red, Green, Blue) values representing colors of the vertex. This allows for more realistic and detailed 3D models because the color of each vertex can be used to create shading effects and simulate the shape of real objects.

The improved mesh data structure D800 of the present invention may include other attributes associated with each vertex such as surface normals, texture coordinates, or material information in addition to vertex color values. By incorporating this additional information into the mesh data structure, 3D modeling software can create more accurate and detailed models.

The advanced mesh data structure can be used with a variety of 3D modeling software and file formats. Including vertex color values and other properties in the mesh data structure (D800) can improve the visual quality of 3D models and provide users with a more realistic and immersive experience.

That is, the present invention 100 provides an improved mesh data structure for 3D modeling data including vertex color values and other properties associated with each vertex. Through this, more realistic and detailed 3D models can be implemented and the visual quality of 3D modeling software can be improved.

9 is a diagram illustrating that a user-customized nail tip can be manufactured using extracted nail data according to an embodiment.

Referring to FIG. 9 , it can be assumed that the scan object is a 'human hand' and the segment is a 'nail' according to an embodiment of the present invention. When 3D modeling data in which a texture image including a scan object and a masked segment is combined is formed under the control of the processor 120 of the present invention, a user desired color is extracted from the mesh data of the 3D modeling data according to a specific color extraction or removal algorithm. 3D data about a segment (eg, 'nail') that is an area may be output. Accordingly, when Shape (D912) and Length (D913) information is additionally input through the user terminal using the 3D data on the nail as the input data (D910), the nail tip is automatically designed through the nail tip design algorithm. A user-customized nail tip can be manufactured by controlling an external device that performs a 3D printing function so that the tip is automatically designed as 3D data and 3D printed based on the 3D data. In addition, the nail tip may be manufactured using 3D data of the nail tip not only by 3D printing but also by various manufacturing methods.

That is, with respect to the object acquired through 3D scanning technology, the segment is extracted by segmenting the area desired by the user, deep learning can be used to increase the accuracy of the segmentation, and the shape can be obtained based on the 3D data of the extracted segment. and length information are additionally input, a user-customized object may be manufactured according to a predetermined automatic nail tip design algorithm.

Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing instructions executable by a computer. Instructions may be stored in the form of program codes, and when executed by a processor, create program modules to perform operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

Computer-readable recording media include all types of recording media in which instructions that can be decoded by a computer are stored. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, and the like.

As above, the disclosed embodiments have been described with reference to the accompanying drawings. Those skilled in the art to which the present disclosure pertains will understand that the present disclosure may be implemented in a form different from the disclosed embodiments without changing the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

100: Segmentation device for objects formed through 3D scanning
200: External device performing 3D scan function
300: user terminal

Claims (10)

Memory;
at least one processor in communication with the memory; and
Including a communication module that communicates with an external device and a user terminal that performs a 3D scan function,
the processor,
Receiving a scan object, which is a scan object, and a texture image representing a texture of the scan object from the external device;
Extracting a segment representing a desired part of an object based on the texture image through supervised-learning,
Masking the extracted segments with a predetermined specific color,
Forming 3D modeling data in which the object is three-dimensionally modeled based on the texture image including the scan object and the masked segments;
Extracting 3D data for the segment through comparison with attribute values for the specific color masked based on mesh data including attribute values of the 3D modeling data;
The masking is
An area of the object excluding only the portion related to the segment,
The 3D modeling data,
It is formed based on the texture image including an area excluding the scan object and the masked portion related to the segment,
Extracting 3D data for the segment,
Characterized in that it is performed by excluding the masked area through comparison with the attribute value for the specific color masked based on the mesh data including the attribute value of the 3D modeling data, of the object formed through 3D scanning segmentation device. According to claim 1,
The texture of the scan object is
An apparatus for segmenting an object formed through 3D scanning, characterized in that three-dimensionally implemented by constructing a point cloud of the object based on at least one of the texture images. According to claim 1,
Labeling the object data including the part to be segmented and the segment part data as training data,
Batch is made to the learning model based on the labeled training data,
When the texture image is input from the user and the segment is extracted,
Based on a pre-stored ground truth set, testing and verification of the extraction of the segment is performed,
generate feedback data based on the testing and verification;
Based on the feedback data, parameters of the learning model are tuned to perform supervised-learning of the segmentation of the object. According to claim 3,
Extracting 3D data for the segment,
Assigning color information for the texture image to the mesh data as a vertex color value,
Extracting RGB values for each polygon area of the mesh data formed based on the vertex color values;
Comparing RGB values for each polygon area of the mesh data with RGB values for the specific color masked,
When the RGB values of the polygon area and the RGB values of the specific color match, the segmentation device of an object formed through 3D scanning characterized in that it is performed by extracting the polygon area with matching RGB values. delete According to claim 4,
The scan object means data about a human hand,
The segmentation apparatus of an object formed through 3D scanning, characterized in that the segment refers to a region for a nail portion of the human hand. According to claim 4,
The 3D modeling data,
.fbx, .dae, .3ds, .abc, .usd and .obj are compiled with any one of the extensions and stored in the memory, characterized in that, the segmentation device of the object formed through 3D scanning. Receiving a scan object that is a scan object and a texture image representing a texture of the scan object from an external device that performs a 3D scan function;
Extracting a segment representing a desired part of an object based on the texture image through supervised-learning;
Masking the extracted segments with a predetermined color;
forming 3D modeling data in which the object is three-dimensionally modeled based on the texture image including the scan object and the masked segments; and
Extracting 3D data for the segment through comparison with attribute values for the specific color masked based on mesh data including attribute values of the 3D modeling data,
The masking is
An area of the object excluding only the portion related to the segment,
The 3D modeling data,
It is formed based on the texture image including an area excluding the scan object and the masked portion related to the segment,
Extracting 3D data for the segment,
Characterized in that it is performed by excluding the masked area through comparison with the attribute value for the specific color masked based on the mesh data including the attribute value of the 3D modeling data, of the object formed through 3D scanning Segmentation method. According to claim 8,
The step of extracting 3D data for the segment,
assigning color information about the texture image to the mesh data as a vertex color value;
extracting RGB values for each polygon area of the mesh data formed based on the vertex color values;
Comparing the RGB values of each polygon area of the mesh data with the RGB values of the masked specific color; and
and extracting the polygon area having matching RGB values when the RGB values of the polygon area and the RGB values of the specific color match. A computer program stored in a recording medium that executes the method of any one of claims 8 and 9 in combination with hardware.

Images