KR20240009897A - A method for processing image, an electronic apparatus and a computer readable storage medium - Google Patents

Abstract

An image processing method, an electronic device for performing the same, and a computer-readable storage medium for storing the same are provided. The image processing method includes loading scan data corresponding to at least one oral image, calculating based on the scan data, setting a plurality of reference points in the scan data based on the calculation, and and aligning the scan data based on an occlusal plane created based on a plurality of reference points.

Description

IMAGE PROCESSING METHOD, ELECTRONIC DEVICE, AND COMPUTER READABLE STORAGE MEDIUM

The present disclosure relates to image processing methods, electronic devices, and computer-readable storage media.

To treat temporomandibular joint disorders, dental trauma, cavities, gum disease, etc., oral structures (splints, temporary teeth, etc.) that are inserted and placed in the oral cavity can be used. In order to manufacture oral structures that reflect the characteristics of the oral cavity into which they are inserted, oral images obtained by scanning the oral cavity are used in the production of oral structures.

In manufacturing such oral structures, the front direction of the oral cavity must be specified, or in some cases, a detailed anatomical coordinate system for the teeth may be required. For example, by specifying the plane where opposing teeth are placed, margin line extraction, prosthesis insertion direction, etc. can be determined, and the midline of the arch can be recognized to determine left/right teeth, or the anterior and posterior teeth recognized by recognizing the front direction of the mouth. Since it can be separated, it can be of great help in carrying out subsequent work.

However, in the current manufacturing process of oral structures, the front direction of the oral cavity and the occlusal plane are specified through the user's manual method, which takes a lot of time and has low accuracy.

The disclosed embodiments can efficiently and accurately specify the front direction and occlusal plane of the oral cavity through computational processing on scan data.

According to one disclosed embodiment, receiving at least one scan data, extracting an initial occlusal direction of the scan data based on mesh data included in the at least one scan data, and determining the initial occlusal direction generating cusp information of the scan data based on the cusp information, setting a front direction and midline of the scan data based on the cusp information, adjusting the midline by creating a cylinder adjacent to the scan data, An image processing method is provided including setting an occlusal plane of the scan data based on an adjustment operation.

According to another disclosed embodiment, a user interface device, a processor, and a memory storing instructions executable by the processor, wherein the processor executes the instructions to receive at least one scan data, extracting the initial occlusal direction of the scan data based on mesh data included in at least one scan data, generating cusp information of the scan data based on the initial occlusal direction, and generating cusp information based on the cusp information An electronic device is provided that sets the front direction and midline of scan data, creates a cylinder adjacent to the scan data to adjust the midline, and sets an occlusal plane of the scan data based on the adjustment operation.

According to the disclosed embodiments, the time required to manufacture an oral structure can be reduced by automatically specifying the front direction and occlusal plane of the oral cavity and aligning the oral image.

According to the disclosed embodiments, the oral image can be precisely aligned by placing the midline of the dental arch between the central incisors.

FIG. 1 is a diagram for explaining an image processing system including an electronic device according to an embodiment.
FIG. 2 is a block diagram illustrating the configuration of an electronic device, according to one embodiment.
Figure 3 is a flowchart showing an image processing method of an electronic device, according to one embodiment.
4 and 5 are diagrams for explaining scan data, upper jaw scan data, and lower jaw scan data, according to one embodiment.
Figure 6 is a flowchart showing an image processing method of an electronic device, according to one embodiment.
Figures 7 to 9 are diagrams for explaining extracting the initial occlusal surface direction, according to one embodiment.
10 to 12 are diagrams for explaining generating cusp information, according to an embodiment.
Figures 13 and 14 are diagrams for explaining setting the front direction and midline, according to one embodiment.
Figures 15 to 17 are diagrams for explaining fine adjustment of the midline, according to one embodiment.
Figure 18 is a diagram for explaining setting an occlusal plane, according to one embodiment.

Hereinafter, with reference to the attached drawings, various embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. The invention may be implemented in many different forms and is not limited to the embodiments described herein.

In order to clearly explain the present invention, parts that are not relevant to the description are omitted, and identical or similar components are assigned the same reference numerals throughout the specification. The term 'part' (portion) used in the specification may be implemented as software or hardware, and depending on the embodiment, a plurality of 'portions' may be implemented as a single element (unit, element) or as a single 'portion'. It is also possible for 'boo' to contain multiple elements. Hereinafter, the operating principle and embodiments of the present invention will be described with reference to the attached drawings.

In addition, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, so the present invention is not necessarily limited to what is shown. In the drawing, the thickness is enlarged to clearly express various layers and areas. And in the drawings, for convenience of explanation, the thicknesses of some layers and regions are exaggerated.

Additionally, when a part of a layer, membrane, region, plate, etc. is said to be “on” or “on” another part, this includes not only cases where it is “directly above” another part, but also cases where there is another part in between. . Conversely, when a part is said to be “right on top” of another part, it means that there is no other part in between. In addition, being “on” or “on” a reference part means being located above or below the reference part, and does not necessarily mean being located “above” or “on” the direction opposite to gravity. .

In addition, throughout the specification, when a part is said to "include" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary.

In addition, throughout the specification, when referring to “on a plane,” this means when the target portion is viewed from above, and when referring to “in cross section,” this means when a cross section of the target portion is cut vertically and viewed from the side.

Additionally, terms including ordinal numbers, such as 'first' or 'second', used in this specification may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

FIG. 1 is a diagram for explaining an image processing system including an electronic device according to an embodiment, and FIG. 2 is a block diagram showing the configuration of an electronic device according to an embodiment.

Referring to FIGS. 1 and 2 , the image processing system 1 may include a scanner 10 and an electronic device 20 .

In this specification, an 'object' is something that is subject to photography and may include a person, an animal, or a part thereof. For example, the object may include a part of the body (organ or organ, etc.) or a phantom. Additionally, for example, the object may include a plaster model that imitates the oral cavity, a denture such as a denture or denture, a dentiform in the shape of teeth, etc. For example, the subject may have teeth, gingiva, at least some areas of the oral cavity, and/or artificial structures implantable within the oral cavity (e.g., orthodontic devices including brackets and wires, implants, abutments, artificial teeth, inlays, and It may include dental restorations including onlays, orthodontic aids inserted into the oral cavity, etc.), teeth or gingiva to which artificial structures are attached, etc.

The scanner 10 may refer to a device that acquires images related to an object. The scanner 10 may refer to a scanner 10 that acquires oral images related to the oral cavity used for oral treatment. The scanner 10 can acquire at least one of a two-dimensional image and a three-dimensional image. Additionally, the scanner 10 may acquire at least one two-dimensional image of the oral cavity and generate a three-dimensional image (or three-dimensional model) of the oral cavity based on the at least one acquired two-dimensional image. Additionally, the scanner may acquire at least one two-dimensional image of the oral cavity and transmit the at least one two-dimensional image to the electronic device 20.

The electronic device 20 also includes a scanner 10 tooth model or teeth inside the oral cavity, gingiva, and artificial structures that can be inserted into the oral cavity (e.g., orthodontic devices including brackets and wires, implants, artificial teeth, splints, etc.) The surface of at least one of the surfaces (including orthodontic aids inserted into the oral cavity, etc.) can be imaged, and for this purpose, surface information about the object can be acquired as raw data.

The electronic device 20 may generate a 3D image of the oral cavity based on at least one received 2D image. Here, the ‘3D image’ can be created by modeling the object in three dimensions based on the received raw data, so it can be called a ‘3D model’. Additionally, in the present invention, a model or image representing an object in two or three dimensions may be collectively referred to as an ‘image.’

For example, the scanner 10 may be an intraoral scanner that can be inserted into the oral cavity. Depending on the embodiment, the intraoral scanner may be a wired device or a wireless device, and the technical spirit of the present invention is not limited to the type of intraoral scanner.

According to one embodiment, the intraoral scanner may be a hand-held scanner that can be held and carried by hand. The oral scanner is inserted into the oral cavity and scans the teeth in a non-contact manner, so that it can have an image of the oral cavity including at least one tooth, and uses at least one image sensor (for example, an optical camera, etc.) The inside of a patient's mouth can be scanned.

According to one embodiment, the scanner 10 may be a table-type scanner that can be used for dental treatment. A table-type scanner may be a scanner that obtains surface information about an object as raw data by scanning the object using the rotation of a table. A table scanner can scan the surface of an object such as a plaster model or impression model of the oral cavity.

The electronic device 20 can receive raw data from the scanner 10, process the received raw data, and output a three-dimensional image for the raw data. Depending on the embodiment, the output 3D image may be 3D image data including oral structures such as splints for the received raw data. For ease of explanation, a detailed description of the scan data will be described later in FIGS. 4 and 5.

The electronic device 20 is connected to the scanner through a wired or wireless communication network, receives a two-dimensional image obtained by scanning an object from the scanner, and generates, processes, displays, and/or creates an image based on the received two-dimensional image. Or it can be any electronic device capable of transmitting.

The electronic device 20 may store and execute dedicated software to perform at least one operation for receiving, processing, storing, and/or transmitting a 3D image or 2D image of an object. For example, the dedicated software performs processing operations such as region extraction and region setting on the received scan data, and performs data selection, reference point adjustment, and alignment based on the processing operations to create artificial structures including artificial structures such as splints. At least one operation such as generating, storing, and transmitting a 3D image can be performed. The electronic device 20 may be a computing device such as a smart phone, laptop computer, desktop computer, PDA, or tablet PC, but is not limited thereto. Additionally, the electronic device 20 may exist in the form of a server (or server device) for processing oral images.

The electronic device 20 may include a communication unit 21, a processor 22, a user interface device 23, a display 24, a memory 25, and a database 21. However, not all of the illustrated components are essential components. The electronic device 20 may be implemented with more components than the components shown, and the electronic device 20 may be implemented with fewer components than the illustrated components. Below, we will look at the above components.

The communication unit 21 can communicate with an external device. Specifically, the communication unit 21 can be connected to a network by wire or wirelessly to communicate with an external device. Here, the external device may be a scanner 10, a server, a smartphone, a tablet, a PC, etc.

The communication unit 21 may include a communication module that supports one of various wired and wireless communication methods. For example, the communication module may be in the form of a chipset, or may be a sticker/barcode (e.g. a sticker including an NFC tag) containing information necessary for communication. Additionally, the communication module may be a short-distance communication module or a wired communication module.

For example, the communication unit 21 supports wireless LAN, Wi-Fi (Wireless Fidelity), WFD (Wi-Fi Direct), Bluetooth, BLE (Bluetooth Low Energy), Wired Lan, and NFC (Near). It can support at least one of (Field Communication), Zigbee, infrared Data Association (IrDA), 3G, 4G, and 5G.

In one embodiment, the scanner 10 may transmit the acquired raw data to the electronic device 20 through a communication module. Image data acquired from the scanner may be transmitted to the electronic device 20 connected through a wired or wireless communication network.

The processor 22 controls the overall operation of the electronic device 20 and may include at least one processor such as a CPU. The processor 22 may include at least one specialized processor corresponding to each function, or may be an integrated processor.

The processor 22 may receive raw data through the communication unit 21. For example, the processor 22 may receive raw data from the scanner 10 through the communication unit 21. In this case, the processor 22 generates three-dimensional image data (e.g., surface data, mesh data, etc.) that three-dimensionally represents the shape of the surface of the object, based on the received raw data. Hereinafter, the scan data that is the subject of calculation by the electronic device 20 may include 3D image data.

The processor 22 may receive library data from an external device through the communication unit 21. Library data may be data pre-stored in the electronic device 20 or raw data or 3D image data acquired through an external device, but is not limited thereto. Here, the external device may be a camera capable of taking photos or videos, or an electronic device with a camera function. Additionally, the external device may be an oral scanner capable of scanning the inside of the patient's mouth.

The processor 22 may control the user interface device 23 or the display 24 to receive input of certain commands or data from the user.

The processor 22 can execute a program stored in the memory 25, read an image, data, or file stored in the memory 25, or store a new file in the memory 25. The processor 22 may execute instructions stored in the memory 25. The stored program may include, but is not limited to, dedicated software.

The processor 22 may perform calculation operations on mesh data and data included in scan data. For example, the processor 22 calculates the average of the normal vectors of the mesh data constituting the scan data, generates a 3D-Oriented Bounding Box including a plurality of mesh data or a plurality of reference points, or , an operation for creating a 2D-Oriented Bounding Box and/or a Convex Hull algorithm operation for a plurality of mesh data or a plurality of reference points may be performed.

A bounding box refers to a box of the minimum size surrounding a plurality of location data or a specific object. A 3D bounding box may have a rectangular parallelepiped shape, and a 2D bounding box may have a rectangular shape.

The convex Hull algorithm is an algorithm that generates a convex polyhedron containing the points in the set for a finite set of points. Depending on the embodiment, a static calculation algorithm or a dynamic calculation algorithm utilizing triangulation may be used. You can. A convex polyhedron may include multiple vertices and faces.

Processor 22 may extract a vector from the bounding box. The processor 22 may extract a vector having the size of the edge length of the 3D bounding box and the direction of the extension direction, or extract a vector having the size of the length of one side of the 2D bounding box and the direction of the extension direction. For example, the processor 22 may perform an operation of extracting a direction vector for the minimum edge having the minimum length of a 3D bounding box, or extracting a vector for the long side of a 2D bounding box. Additionally, the processor 22 may calculate the average vector of the extracted vectors.

Processor 22 may calculate a complexity representing surface curvature for scan data with respect to a specific direction. As an example, the processor 22 calculates the complexity of the surface curvature using LoG (Laplacian of Gaussian), which combines the Laplacian, which can determine the difference in slope within the scan data, and the Gaussian, which is advantageous for noise removal. You can.

The processor 22 may recognize an object in scan data, extract a partial area, or calculate the area or volume of the recognized object or the extracted area. For example, the processor 22 may use curvature information, cusp information, and anatomical feature shapes of the scan data to recognize the types of teeth or distinguish the spaces between teeth. According to the disclosed embodiment, the recognition operations of the processor 22 are not limited to examples of utilizing the above information, and the recognition operations may be performed through inference of an object recognition artificial intelligence algorithm. Cusp information may include the number and arrangement of cusps that opposing teeth in scan data contact.

The processor 22 may select data based on a specific direction, predetermined value, etc. For example, the processor 22 generates a local maximum point based on the initial occlusal direction from the data in the scan data, or selects a cusp point based on a predetermined value or calculated area among a plurality of local maximum points. You can perform the following actions.

The processor 22 may specify the direction of the scan data based on the reference points and set the midline of the scan data. For example, the processor 22 specifies the front direction of the scan data using the outer cusp of the left first molar, the outer cusp of the right first molar, and the central cusp in the central incisor as reference points, and uses the left and right sides of the scan data as reference points. You can perform movements that set the midline of the split. The midline is a line that passes through the center of the dental arch in the scan data, and the left and right sides of the oral cavity can be distinguished through the midline.

The processor 22 may generate a cylinder fitting into the space between objects through curvature information, etc. For example, the processor 22 may generate a cylinder that is placed in contact with the central incisors in the space between the central incisors.

Processor 22 may perform adjustments to the midline. Specifically, an adjustment operation may be performed using the fitted cylinder with respect to the reference point through which the midline passes, and the point through which the midline passes may be adjusted.

The processor 22 may set a plane based on a plurality of reference points. For example, processor 22 may set an occlusal plane for scan data based on three reference points.

The user interface device 23 may refer to a device that receives data from a user to control the electronic device 20. The display 24 may include an output device for displaying a result image according to the operation of the electronic device 20 or a 3D image output from the electronic device 20.

The user interface device 23 may include, for example, an input device such as a mouse, joystick, operation panel, or touch-sensitive panel that receives user input, and the display 24 may include a display panel that displays a screen. can do.

The database 26 stores data and datasets for learning the artificial intelligence algorithm of the dedicated software, and can provide data for learning upon request from the dedicated software. The artificial intelligence algorithm learns the teeth learning data stored in the database 26 using a deep learning method and can distinguish the characteristics of the data representing the teeth. Meanwhile, the dedicated software extracts maxillary tooth area data and mandibular tooth area data from scan data or recognizes objects according to tooth characteristics, thereby extracting or recognizing them when performing the occlusal plane alignment step, inner surface setting step, and contour designation step, which will be described later. tooth area data can be used.

In the present invention, artificial intelligence (AI) refers to technology that imitates human learning ability, reasoning ability, and perception ability and implements this with a computer, and may include the concepts of machine learning and symbolic logic. Machine learning (ML) can be an algorithmic technology that classifies or learns the characteristics of input data on its own. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the results of the analysis, and makes judgments or predictions based on the results of the learning. Additionally, technologies that mimic the cognitive and judgment functions of the human brain using machine learning algorithms can also be understood as the category of artificial intelligence. For example, it may include technical fields such as verbal understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control.

In the present invention, machine learning may refer to the process of training a neural network model using experience processing data. Machine learning can mean that computer software improves its own data processing capabilities. A neural network model is built by modeling the correlation between data, and the correlation can be expressed by a plurality of parameters. A neural network model extracts and analyzes features from given data to derive correlations between data. Repeating this process to optimize the parameters of the neural network model can be called machine learning.

For example, a neural network model can learn the mapping (correlation) between input and output for data given as input-output pairs. Alternatively, a neural network model can learn relationships by deriving regularities between given data even when only input data is given.

In the present invention, an artificial intelligence learning model, machine learning model, or neural network model may be designed to implement the human brain structure on a computer, and includes a plurality of network nodes that simulate neurons of a human neural network and have weights. can do. A plurality of network nodes may have a connection relationship with each other by simulating the synaptic activity of neurons in which neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes are located in layers of different depths and can exchange data according to convolutional connection relationships.

The database 26 is shown as included in the electronic device 20 in the drawing, but is not limited thereto and is arranged outside the electronic device 20 in the form of a server (or server device) to provide data for learning. and save the learning results.

FIG. 3 is a flowchart showing an image processing method of an electronic device, according to an embodiment, and FIGS. 4 and 5 are diagrams for explaining scan data, upper jaw scan data, and lower jaw scan data, according to an embodiment.

Referring to FIGS. 1 to 5 , the electronic device 20 loads scan data 100 (S100). The electronic device 20 loads scan data 100 generated based on an image received from an external device, including a scanner 10, through the communication unit 21.

The electronic device 20 may process the received image or load scan data 101 previously stored in the processor 22 or the user interface device 23 and display it through the display 24 .

The scan data 100 may be a two-dimensional image of an object, a three-dimensional model representing the object three-dimensionally, or three-dimensional image data, and may specifically be a three-dimensional oral model. The oral images in FIGS. 4 and 5 correspond to the scan data 100 and are a two-dimensional or three-dimensional representation of the object of the scan data 100. As with the scan data 100, which will be described later, the upper jaw profile It may include a pre-preparation image, a maxillary preparation image, a mandibular pre-preparation image, a mandibular preparation image, and an occlusal image including the upper jaw-related image and the mandibular-related image.

In the present invention, preparation (Preparation) may refer to a series of preparation processes that remove part of the enamel and dentin of the tooth to prevent interference between natural teeth and the prosthesis when prosthetics such as crowns and bridges.

“3D oral model” may refer to a model that three-dimensionally models the oral cavity based on raw data acquired through a scanning operation of a scanner. Additionally, “3D oral model” may refer to a three-dimensionally modeled structure based on data obtained by a scanner scanning an object such as a tooth, impression, or artifact. A 3D oral model is created by modeling the internal structure of the oral cavity in 3D, and may be referred to as a 3D scan model, 3D model, or tooth model. For example, the format of the 3D oral model may be one of STL (Standard Triangle Language), OBJ, and polygon file formats, but is not limited to the above example. Additionally, the 3D oral model may include information such as geometric information, color, texture, and material about the 3D shape.

Additionally, “polygon” may refer to a polygon, which is the smallest unit used when expressing the three-dimensional shape of a three-dimensional oral model. For example, the surface of a 3D oral model can be expressed as triangular polygons. For example, a polygon may consist of at least three vertices and one face. Vertices may include information such as position, color, and normal. A mesh may be an object in three-dimensional space created by gathering a plurality of polygons. As the number of polygons representing the 3D oral model increases, the object can be expressed in detail.

The scan data 100 may include at least one of upper jaw scan data 101 and lower jaw scan data 102. Specifically, the scan data 100 may load any one of maxillary pre-preparation data, maxillary preparation data, mandibular pre-preparation data, mandibular preparation data, and occlusion data including the upper and lower jaw-related data.

In the present invention, preparation (Preparation) refers to a series of preparation processes that remove part of the enamel and dentin of the tooth to prevent interference between the natural teeth and the prosthesis when prosthetics such as crowns and bridges. Prep data may be data in which part of the enamel and dentin of the tooth has been removed through the preparation process, and pre-preparation data may be data before part of the enamel and dentin in the tooth have been removed through the preparation process.

The scan data 100 may include a gingival area 200, upper tooth area data 301, and lower tooth area data 302 disposed within the upper jaw scan data 101 and lower jaw scan data 102.

The electronic device 20 may load at least one of upper jaw scan data 101, lower jaw scan data 102, and occlusion data including upper jaw scan data 101 and lower jaw scan data 102.

The electronic device 20 analyzes and sorts the shape of the received scan data 100 (S200). In this step, the occlusal plane and midline for the scan data 100 are set, and the electronic device 20 automatically aligns the scan data 100 with the maxillary scan data 101 or the mandibular scan data 102 according to the occlusal plane. And, the left and right alignment according to the midline can be displayed through the display 24. A detailed description of the corresponding step will be described later in the description of FIGS. 6 to 18.

Additionally, in this step, the user can manually specify a reference point on the scan data 100 to set the front direction and occlusal plane of the scan data 100, and align the scan data 100 along the set occlusal plane. . Illustratively, the user may select some data of the scanned data 100 through the user interface device 23 in the corresponding step, and may align the scanned data 100 using the selected data as a reference point.

The electronic device 20 sets the inner surface of the oral structure for the aligned scan data 100 (S300).

In this step, the electronic device 20 may designate a direction in which the oral structure will be inserted by considering the undercut of the aligned scan data 100. For example, when manufacturing a splint, which is an oral structure, the electronic device 20 calculates the area of the tooth area in the scan data 100 and considers undercut and block-out according to the direction in which the oral structure will be inserted. You can specify the direction in which this will be inserted. By specifying the insertion direction of the oral structure as described above, the insertion efficiency and retention power of the oral structure can be improved.

Based on the inner surface offset distance, surface smoothness, etc. input from the user interface device 23, the electronic device 20 can set the inner surface of the oral structure to be output. The inner surface offset distance may mean the separation distance in the normal direction between the scan data 100 and the inner surface of the oral structure. The surface smoothness may refer to the roughness of the inner surface of the oral structure.

The electronic device 20 specifies the outline of the oral structure for the automatically aligned scan data 100 (S400).

Based on the buccal height, lingual height, etc. input from the user interface device 23, the electronic device 20 may specify the outline of the oral structure to be output. For example, when manufacturing a splint, which is an oral structure, the buccal height is the height of the outer wall of the tooth facing the cheek based on the lower surface of the tooth area. For example, the buccal height is the tooth area of the maxillary scan data 101. It may refer to the height formed along the outer wall of the tooth based on the bottom surface of the tooth. The higher the buccal height, the closer the buccal contour is to the gingiva. The lingual height is the height of the inner wall of the tooth facing the tongue based on the lower surface of the tooth area. For example, the lingual height is the height formed along the inner wall of the tooth based on the bottom surface of the tooth area of the maxillary scan data 101. It can mean. The higher the lingual height, the closer the lingual contour formed is to the gingiva.

The electronic device 20 sets the outer surface of the oral structure for the aligned scan data 100 (S500).

Based on the thickness, surface smoothness, etc. input from the user interface device 23, the electronic device 20 may designate the outer surface of the oral structure to be output. The electronic device 20 may form a three-dimensional image of the oral structure by setting the thickness of the oral structure in the occlusal direction based on the predetermined occlusal thickness. For example, when manufacturing a splint, which is an oral structure, the thickness may refer to the thickness in the buccal/lingual direction from the inner surface of the oral structure. The surface smoothness may refer to the roughness of the outer surface of the oral structure. The predetermined occlusal thickness may mean the maximum thickness value at which the oral structure extends in the occlusal direction.

The electronic device 20 generates three-dimensional image data including the oral structure through the information set and specified in steps S300 to S500 (S600). The generated three-dimensional image of the oral structure may be transmitted to an external device through the communication unit 21 and output as an oral structure. The external device may be a 3D print, but is not limited to the above example depending on the embodiment.

Depending on the embodiment, the electronic device 20 may perform steps S200 to S500 at once without intermediate input from the user. The electronic device 20 loads the scan data 100 (S100), receives input of the inner offset distance, surface smoothness, buccal height, lingual height, thickness, etc. from the user, and performs steps S200 to S500 without intermediate user intervention. can be performed automatically to generate 3D image data (S600).

Depending on the embodiment, the electronic device 20 automatically performs steps S200 to S500 without intermediate user intervention by utilizing the inner offset distance, surface smoothness, buccal height, lingual height, thickness, etc. stored in the memory 25. can do.

Depending on the embodiment, the electronic device 20 may automatically perform steps S200 to S500 without intermediate user intervention, thereby reducing the time required to manufacture the oral structure.

Depending on the embodiment, the electronic device 20 may generate three-dimensional image data for the oral structure through the inference operation of an artificial intelligence algorithm without any additional input other than the loaded scan data 100 (S600). The artificial intelligence algorithm performs learning about the oral structure corresponding to a plurality of scan data before the inference operation, and performs the inference operation in relation to the three-dimensional image data of the oral structure suitable for the loaded scan data 100. .

Depending on the embodiment, the electronic device 20 uses user input regarding the occlusion state or the minimum distance between arches (Distance to Antagonist) between the upper and lower scan data 101 and the lower jaw scan data 102 in the scan data 100. , the occlusal state or the minimum distance between arches can be adjusted in each step of steps S300 to S500. During the adjustment operation, the electronic device 20 may perform a calculation operation on the scan data 100 and display the occlusal state or the minimum distance between arches.

Through the above adjustment and calculation operations, the user can manufacture an oral structure considering the distance and space within the patient's oral cavity.

Hereinafter, step S200 will be described with reference to FIGS. 6 to 18.

Figure 6 is a flowchart showing an image processing method of an electronic device, according to one embodiment.

Referring to FIGS. 1 to 6 , the electronic device 20 extracts the initial occlusal surface direction for the scan data 100 (S210).

With additional reference to FIGS. 7 and 8 , scan data 100 may include a plurality of meshes in a polygonal shape. As shown in FIG. 8, the mesh included in the scan data 100 may have a triangular shape, but the shape is an example for ease of explanation and may be a square, pentagon, etc.

According to one embodiment, the electronic device 20 may extract a normal vector (n_1-n_M) for each mesh from the maxillary scan data 101 including M meshes.

The electronic device 20 extracts the average normal vector (N_avg) based on the extracted normal vector (n_1-n_M), and sets the direction of the extracted average normal vector (N_avg) to the initial occlusal surface, as shown in Equation 1 below. It can be set to direction (N_occ0).

In Equation 1, the is the unit vector of the average normal vector (N_avg) for the M meshes, and inside is the unit vector of the normal vectors (n_1-n_M) of the M meshes, and inside is the area of the M meshes.

In FIGS. 7 and 8 , extraction of the initial occlusal direction (N_occ0) is explained using the maxillary scan data 101 as an example, but it may vary depending on the embodiment and is not limited to the above example. Hereinafter, the description of the upper jaw scan data 101 in FIGS. 9 to 18 may also be applied to other scan data 100 including the lower jaw scan data 102, and for ease of explanation, the description of the other scan data 100 will be provided. The explanation will focus on differences from the explanation in the upper jaw scan data 101.

Referring additionally to FIG. 9, according to one embodiment, the electronic device 20 creates a three-dimensional bounding box (Oriented) with respect to the maxillary scan data 101 including mesh data. Bounding Box, OBB), extract the extension direction of the minimum edge (L1) with the minimum length of the 3D bounding box (OBB) and the minimum edge vector (N_OBB) with the size of the minimum length, and extract the minimum edge vector (N_OBB) can be set as the initial occlusal direction (N_occ0).

According to an embodiment, the electronic device 20 performs a convex Hull algorithm on the scan data to generate a convex polyhedron including vertices in the scan data, and generates a three-dimensional bounding box for the generated convex polyhedron, as shown in FIG. 9 As shown, the extension direction of the minimum edge having the minimum length of the 3D bounding box and the minimum edge vector having the size of the minimum length can be extracted, and the minimum edge vector (N_OBB) can be set as the initial occlusal direction. By creating a 3D bounding box for a convex polyhedron, the efficiency of setting the initial occlusal direction can be increased by reducing the amount of computational processing required to create the 3D bounding box.

According to the embodiment, the electronic device 20 may generate a 3D bounding box for scanned data including mesh data, and calculate the complexity of the surface curvature of the scanned data in the normal direction of each side of the 3D bounding box. there is. The electronic device 20 may set the initial occlusal direction to be parallel to the direction of extension of the minimum edge having the minimum length of the three-dimensional bounding box, and to proceed from a surface with low complexity of surface curvature to a surface with high complexity. In the case of scan data containing a closed-curved tooth model, it may be efficient to set the initial occlusal direction by generating a 3D bounding box and calculating the complexity of surface curvature instead of using the average normal vector.

Next, the electronic device 20 generates cusp information (S220).

With additional reference to FIGS. 10 to 12 , the electronic device 20 utilizes curvature information, cusp information, and anatomical feature shapes in the maxillary scan data 101 to generate maxillary scan data 101 as shown in FIG. 10 . ) Depending on the curved shape, it can be separated into multiple image groups (G1-GN).

The electronic device 20 may generate a plurality of local maximum points (LM_1-LM_N) for the maxillary scan data 102 based on the initial occlusal direction (N_occ0). The plurality of maximal points (LM_1-LM_N) may correspond to the highest placed vertices based on the initial occlusal direction (N_occ0) in each of the plurality of image groups (G1-GN) constituting the maxillary scan data 102. there is.

The electronic device 20 is based on the heights of the plurality of local maximum points (LM_1-LM_N) with respect to the initial occlusal direction (N_occ0) and the plurality of areas (S1-SN) with respect to the plurality of image groups (G1-GN), Cusp_1-cusp_x can be selected from a plurality of local maxima (LM_1-LM_N). The electronic device 20 may select cusp points (cusp_1-cusp_x) by erasing part of the plurality of local maximum points (LM_1-LM_N) in the form of an erasure method, based on a predetermined height and a predetermined area.

The cusp information may include the number and arrangement of cusp points (cusp_1-cusp_x) where opposing teeth in the maxillary scan data 101 contact. The electronic device 20 may extract the posterior teeth (Post) from the maxillary scan data 101 by considering the arrangement and number of cusp points (cusp_1-cusp_x) through cusp information. The posterior teeth (Post) may include cusps arranged in two rows along the arch.

Depending on the embodiment, the electronic device 20 recognizes each tooth in the upper jaw scan data 101 through an object recognition artificial intelligence algorithm at the corresponding stage, recognizes the tooth numbers of the recognized teeth, and selects the teeth according to the tooth number. Cusp information can be generated by considering anatomical features, etc.

The electronic device 20 sets the front direction and midline using the cusp information of the scan data 100 (S230).

Referring additionally to FIGS. 13 to 15, the electronic device 20 uses the extracted posterior teeth (Post) to create a left 2D-Oriented Bounding Box (Box2_L) and a right 2D bounding box (Box2_R) spaced apart from each other. ) can be created. The left two-dimensional bounding box (Box2_L) may include a cusp placed on the left posterior tooth (Post) in the maxillary scan data 101. The right two-dimensional bounding box (Box2_R) may include a cusp placed on the right posterior tooth (Post) in the maxillary scan data 101.

The left two-dimensional bounding box (Box2_L) and the right two-dimensional bounding box (Box2_R) can be included in the posterior region (Post), and the left two-dimensional bounding box (Box2_L) and right two-dimensional bounding box (Box2_R) can be located on the left and right centered on the anterior teeth. It can be placed spaced apart.

The electronic device 20 extracts the left medial vector (V2_L) for the left medial long side arranged in the lingual direction from the left two-dimensional bounding box (Box2_L), and the left medial vector (V2_L) arranged in the lingual direction from the right two-dimensional bounding box (Box2_R). The right medial vector (V2_R) for the right medial long side can be extracted. The left inner vector (V2_L) has the length of the left inner long side as its size and may have a direction in the extension direction of the left inner long side. The right inner vector (V2_R) has the length of the right inner long side as its size and may have a direction in the extension direction of the right inner long side.

The electronic device 20 may generate an average vector (Vav) of the extracted left inner vector (V2_L) and right inner vector (V2_R), and set the front direction of the scan data 100 to the direction of the average vector (Vav). there is.

According to one embodiment, the electronic device 20 determines a first reference point (P1) and a second reference point (P2) based on the arrangement of the left two-dimensional bounding box (Box2_L) and the right two-dimensional bounding box (Box2_R). You can set it. The first reference point (P1) may be the cusp closest to the first reference line (BL1), which is a predetermined distance (L) away from the short side of the left two-dimensional bounding box (Box2_L). The second reference point (P2) may be the vertex closest to the first reference line (BL2), which is a predetermined distance (L) away from the short side of the right two-dimensional bounding box (Box2_R). The predetermined distance (L) is 10 mm to 20 mm, but is not limited thereto.

According to the embodiment, the electronic device 20 recognizes the first reference point (P1) through object recognition for cusp information and maxillary scan data 101 in the left two-dimensional bounding box (Box2_L) and the right two-dimensional bounding box (Box2_R). ) and the second reference point (P2) can be set.

According to the embodiment, the electronic device 20 may recognize and set the maxillary left first molar (Mo1_L) and the maxillary right first molar (Mo1_R) through an object recognition artificial intelligence algorithm for the maxillary scan data 101. there is.

The electronic device 20 selects a cusp disposed in the buccal direction among cusp points within the first molars (Mo1_L, Mo1_R) as the outer cusp, and selects the outer cusp closest to the anterior teeth among the selected outer cusps as the first reference point (P1). ) and the second reference point (P2).

The electronic device 20 sets the midpoint of the first reference point (P1) and the second reference point (P2) as the midpoint (CMo), and moves the point moved from the midpoint (CMo) by the average vector (Vav) to the adjacent reference point. It can be designated as (PP). The electronic device 20 may set the cusp of the mandible scan data 102 closest to the adjacent reference point PP as the third reference point P3.

The electronic device 20 may set an extension line that passes through the third reference point P3 and extends in the direction of the average vector Vav as the midline Cc.

The electronic device 20 adjusts the midline Cc to the adjusted midline Cc' (S240).

With additional reference to FIGS. 16 and 17 , the electronic device 20 may analyze the front shape of the upper jaw scan data 101 and adjust the midline Cc to the adjustment midline Cc'. The front shape of the maxillary scan data 101 can be expressed in the opposite direction of the average vector (Vav).

The electronic device 20 may set the adjacent area PR including the third reference point P3. Depending on the embodiment, the adjacent area PR may be an area that meets the maxillary scan data 101 by rotating the midline Cc within a certain range based on the midpoint CMo. The certain range is -10 degrees to 10 degrees, but it is not limited to the above numerical range and the numerical range may vary depending on the embodiment.

The electronic device 20 acquires curvature information in the maxillary scan data 101 in the adjacent region (PR) and recognizes the interproximal area between central incisors by grouping vertices in the maxillary scan data 101 with curvature information in a certain range. can do. The electronic device 20 may perform a cylinder fitting operation to create a central cylinder Cy extending adjacent to the maxillary scan data 101 along the recognized interproximal area between the central incisors. The central cylinder Cy may be in contact with the maxillary scan data 101 .

The electronic device 20 may set the central reference point Pc by projecting the third reference point P3 onto the central cylinder Cy in a direction perpendicular to the direction in which the central cylinder Cy extends. The electronic device 20 sets the extension line that passes through the central reference point (Pc) and extends in the direction of the average vector (Vav) as the adjustment midline (Cc'), and sets the center line (Cc) by setting the adjustment midline (Cc'). It can be adjusted. Through the corresponding adjustment operation, the electronic device 20 can precisely align the maxillary scan data 101 by placing the adjustment midline (Cc') between the central incisors.

According to the embodiment, the electronic device 20 recognizes the interdental area between the central incisors in the maxillary scan data 101 through an object recognition artificial intelligence algorithm, and determines the center of the interdental area between the central incisors as the location of the third reference point (P3). By adjusting to the reference point (Pc), the midline (Cc) can be adjusted to the adjusted midline (Cc').

The electronic device 20 sets the occlusal plane (OccP) (S250).

With additional reference to FIG. 18 , the electronic device 20 may set the occlusal plane (OccP) based on the first reference point (P1), the second reference point (P2), and the central reference point (Pc). In addition to setting the occlusal plane (OccP) by setting a plurality of reference points (P1, P2, Pc), depending on the embodiment, the electronic device 20 may use the least squares error for cusp information about the cusp points (cusp_1-cusp_x). The occlusal plane (OccP) can be established in a variety of ways, including calculating the plane equation by regression to the law.

According to one embodiment, the electronic device 20 sets a plurality of reference points (P1, P2, Pc) and then selects the plurality of reference points (P1, P2, Pc) based on user input through the input of the user interface device 23. The position can be adjusted, and the occlusal plane (OccP) can be set based on the adjusted plurality of reference points (P1, P2, Pc).

According to one embodiment, the electronic device 20 does not require any separate user input other than loading the scan data 100 during operation S200, thereby reducing the time required to easily align the scan data 100 and produce the splint. It can be reduced.

Additionally, according to one embodiment, the electronic device 20 may precisely align the scan data 100 by automatically placing the midline between the central incisors.

The image processing method according to an embodiment of the present disclosure may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Additionally, an embodiment of the present disclosure may be a computer-readable recording medium on which one or more programs including instructions for executing an image processing method are recorded.

The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the present invention or may be known and usable by those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

Here, the device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and this term is used when data is semi-permanently stored in the storage medium. There is no distinction between temporary storage and temporary storage. For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored.

According to one embodiment, methods according to various embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smartphones) or online. In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) is stored on a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server. It can be temporarily stored or created temporarily.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also possible. It falls within the scope of rights.

Claims (20)

Receiving at least one oral image;
Computing based on scan data corresponding to the at least one oral image;
Based on the calculation step, setting a plurality of reference points in the scan data; and
Aligning the scan data based on an occlusal plane created based on the plurality of reference points;
Image processing method including According to paragraph 1,
Further comprising adjusting the positions of the plurality of reference points through external input,
An image processing method wherein the occlusal plane is generated based on the adjusted positions of the plurality of reference points. According to paragraph 1,
The step of calculating based on the scan data is,
Extracting the initial occlusal direction of the scan data based on mesh data included in the at least one scan data,
Generate cusp information of the scan data,
Setting the front direction and midline of the scan data based on the cusp information,
An image processing method comprising adjusting the midline based on a front surface shape of the scan data in the front direction. According to paragraph 3,
Extracting the initial occlusal direction is,
An image processing method comprising processing normal vectors of the mesh data to obtain an average normal vector. According to paragraph 3,
Extracting the initial occlusal direction is,
Create a 3D-Oriented Bounding Box containing the mesh data,
An image processing method comprising extracting a vector for the minimum edge with the minimum length from the three-dimensional bounding box. According to paragraph 3,
Generating the cusp information includes:
Generating a plurality of local maximum points in the scan data based on the initial occlusal direction,
An image processing method comprising selecting a vertex point among the plurality of maxima points. According to paragraph 3,
Generating the cusp information includes:
Through an artificial intelligence algorithm, the tooth numbers of a plurality of teeth in the scan data are recognized,
An image processing method comprising generating the cusp information for the scan data based on characteristics of the tooth number. According to paragraph 3,
Setting the front direction and the midline of the scan data includes,
Based on the cusp information, extracting the posterior teeth from the scan data,
An image processing method comprising generating first and second 2D-Oriented Bounding Boxes included in the posterior teeth and spaced apart from each other. According to clause 8,
The step of setting the front direction is,
Extracting first and second inner vectors for first and second inner long sides disposed adjacent to each other in each of the first and second two-dimensional bounding boxes,
An image processing method comprising generating an average vector of the first inner vector and the second inner vector. According to clause 9,
The step of setting the midline is,
Setting a first reference point in the first two-dimensional bounding box and a second reference point in the second two-dimensional bounding box,
An image processing method comprising setting a third reference point based on the average vector extending from the midpoint of the first reference point and the second reference point. According to clause 10,
Adjusting the midline is:
Establishing an adjacent area including the third reference point,
Creating a central cylinder by performing cylinder fitting based on curvature information of the scan data in the adjacent area,
An image processing method comprising setting a central reference point by projecting the third reference point onto the central cylinder in a direction perpendicular to the direction in which the central cylinder extends. user interface device;
processor and
Includes a memory that stores instructions executable by the processor,
The processor executes the instructions,
Receive at least one oral image,
Calculate based on scan data corresponding to the at least one oral image,
Based on the operation of the scan data, set a plurality of reference points in the scan data,
An electronic device that aligns the scan data based on an occlusal plane created based on the plurality of reference points. According to clause 12,
The processor adjusts the positions of a plurality of reference points through external input,
The electronic device wherein the occlusal plane is generated based on the adjusted positions of the plurality of reference points. According to clause 12,
Calculating based on the scan data is,
Extracting the initial occlusal direction of the scan data based on mesh data included in the at least one scan data,
Generate cusp information of the scan data,
Setting the front direction and midline of the scan data based on the cusp information,
and adjusting the midline based on the front surface shape of the scan data in the front direction. According to clause 14,
Extracting the initial occlusal direction is,
An electronic device comprising: processing normal vectors of the mesh data to obtain an average normal vector. According to clause 14,
Extracting the initial occlusal direction is,
Create a 3D-Oriented Bounding Box containing the mesh data,
An electronic device comprising extracting a vector for the minimum edge with the minimum length from the three-dimensional bounding box. According to clause 14,
Generating the cusp information includes:
Generating a plurality of local maximum points in the scan data based on the initial occlusal direction,
An electronic device comprising selecting a vertex point among the plurality of maxima points. According to clause 14,
Generating the cusp information includes:
Through an artificial intelligence algorithm, the tooth numbers of a plurality of teeth in the scan data are recognized,
An electronic device comprising generating the cusp information for the scan data based on characteristics of the tooth number. According to clause 14,
Setting the front direction and the midline of the scan data includes,
Based on the cusp information, extracting the posterior teeth from the scan data,
An electronic device including generating first and second 2D-Oriented Bounding Boxes included in the posterior teeth and spaced apart from each other. A computer-readable storage medium containing instructions readable by a computer,
The above commands cause the computer to:
Receiving at least one oral image;
Computing based on scan data corresponding to the at least one oral image;
Based on the calculation step, setting a plurality of reference points in the scan data; and
performing a step of aligning the scan data based on an occlusal plane created based on the plurality of reference points.
A computer-readable storage medium.