JP7260699B1 - DATA PROCESSING DEVICE, DATA PROCESSING METHOD, AND DATA PROCESSING PROGRAM - Google Patents

Abstract

A data processing device, method, and program for easily and appropriately extracting three-dimensional data of a predetermined object in the oral cavity are provided. In a data processing system, a data processing device 1 includes an input unit to which three-dimensional data including position information of each point group representing the surface of at least one object is input, and a cubic data input from the input unit. A two-dimensional data generation unit that processes original data and generates two-dimensional data from three-dimensional data, an identification unit that identifies a predetermined object among at least one object based on the two-dimensional data, and the identified predetermined object By extracting the position information of each point group indicating the surface of the removal unit that extracts the unnecessary three-dimensional data, and every time three-dimensional data for one scan is input, the removal unit extracts three-dimensional data for one scan a computing unit including a combining unit that acquires original data and combines the accumulated three-dimensional data for multiple scans to generate combined three-dimensional data for multiple scans. [Selection drawing] Fig. 9

Description

The present disclosure relates to a data processing device, a data processing method, and a data processing program for processing data including position information of at least one object in the oral cavity.

2. Description of the Related Art Conventionally, in the field of dentistry, a technique of acquiring three-dimensional data of an object such as a tooth by scanning the inside of the oral cavity with a three-dimensional scanner is known. For example, Patent Literature 1 discloses a method of recording the shape of a tooth by imaging the tooth using a three-dimensional scanner.

JP-A-2000-74635

According to the method disclosed in Patent Document 1, a user can measure teeth in the oral cavity using a three-dimensional image generated based on three-dimensional data acquired by a three-dimensional scanner. While confirming the three-dimensional image showing the tooth, the type of tooth can be identified based on one's own knowledge. However, there is a demand for a technique that can appropriately extract three-dimensional data of a predetermined intraoral object such as a tooth without the user identifying the predetermined intraoral object.

The present disclosure has been made to solve such problems, and aims to provide a technique capable of easily and appropriately extracting three-dimensional data of a predetermined object in the oral cavity.

According to one example of the present disclosure, a data processing apparatus is provided for processing data including position information of at least one object within an oral cavity. The data processing device includes an input unit to which three-dimensional data including position information of each point group indicating the surface of at least one object is input, and a calculation unit that processes the three-dimensional data input from the input unit. . The computing unit generates two-dimensional data from the three-dimensional data, and based on the two-dimensional data, at least one of at least one object, an insertion object to be inserted into the oral cavity, a tongue, lips, and a mucous membrane. extracting predetermined three-dimensional data including position information of each point group indicating the surface of the identified predetermined object; extracting predetermined three-dimensional data from the three-dimensional data input from the input unit; Remove data.

According to one example of the present disclosure, a computer data processing method is provided for processing data including position information of at least one object in an oral cavity. The data processing method includes the steps of inputting three-dimensional data including position information of each point cloud representing the surface of at least one object, and processing the input three-dimensional data. The step of processing includes the step of generating two-dimensional data from three-dimensional data, and based on at least one object, at least one of an insertion object to be inserted into the oral cavity, tongue, lips, and mucous membrane based on the two-dimensional data. identifying a predetermined object including one; extracting predetermined three-dimensional data including position information of each of a point cloud indicating the surface of the identified predetermined object; from the input three-dimensional data; and removing predetermined three-dimensional data.

According to one example of the present disclosure, a data processing program is provided for processing data including position information of at least one object within an oral cavity. The data processing program causes the computer to perform steps of inputting three-dimensional data including position information of each point cloud representing the surface of at least one object, and processing the input three-dimensional data. The step of processing includes the step of generating two-dimensional data from three-dimensional data, and based on at least one object, at least one of an insertion object to be inserted into the oral cavity, tongue, lips, and mucous membrane based on the two-dimensional data. identifying a predetermined object including one; extracting predetermined three-dimensional data including position information of each of a point cloud indicating the surface of the identified predetermined object; from the input three-dimensional data; and removing predetermined three-dimensional data.

According to the present disclosure, a user can easily and appropriately extract three-dimensional data of a predetermined intraoral object.

1 is a diagram showing an application example of a data processing system and a data processing device according to a first embodiment; FIG. 2 is a block diagram showing the hardware configuration of the data processing device according to Embodiment 1; FIG. 1 is a diagram showing the configuration of a three-dimensional scanner according to Embodiment 1; FIG. FIG. 4 is a diagram for explaining an example in which the three-dimensional scanner according to Embodiment 1 acquires three-dimensional data based on the confocal method; FIG. 4 is a diagram for explaining a scanning method using a three-dimensional scanner; 4A and 4B are diagrams showing an object in each scan range acquired by the three-dimensional scanner according to Embodiment 1; FIG. It is a figure which shows a mode that an object is scanned using a three-dimensional scanner. It is a figure which shows a mode that an object is scanned using a three-dimensional scanner. 2 is a block diagram showing the functional configuration of the data processing device according to Embodiment 1; FIG. 4 is a diagram for explaining three-dimensional data input to the data processing device according to Embodiment 1; FIG. FIG. 4 is a diagram for explaining learning data used when performing machine learning on the estimation model according to Embodiment 1; FIG. FIG. 4 is a diagram for explaining a correspondence relationship between each of a plurality of objects and a correct label; FIG. 4 is a diagram for explaining combined data after unnecessary object removal generated by the data processing apparatus according to Embodiment 1; FIG. 4 is a flowchart for explaining an example of processing executed by the data processing device according to Embodiment 1; FIG. 10 is a diagram for explaining an example in which a three-dimensional scanner according to Embodiment 2 acquires three-dimensional data based on triangulation; FIG. 10 is a diagram showing a two-dimensional image viewed from an arbitrary viewpoint based on three-dimensional data acquired by the three-dimensional scanner according to Embodiment 2;

<Embodiment 1>
Embodiment 1 of the present disclosure will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are given the same reference numerals, and the description thereof will not be repeated.

[Application example]
An application example of the data processing system 10 and the data processing device 1 according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an application example of a data processing system 10 and a data processing device 1 according to the first embodiment.

As shown in FIG. 1, the user can acquire three-dimensional data of multiple objects in the oral cavity by scanning the oral cavity of a subject using the three-dimensional scanner 2 . A “user” includes an operator such as a dentist, a dental assistant, a teacher or student of a dental university, a dental technician, an engineer of a manufacturer, an operator of a manufacturing plant, etc., using the three-dimensional scanner 2 to scan an object such as a tooth. Any person who acquires three-dimensional data may be employed. The “subject” may be any person who can be scanned by the three-dimensional scanner 2, such as a patient at a dental clinic or a subject at a dental university. The “object” may be any object that can be scanned by the three-dimensional scanner 2, such as teeth in the oral cavity of the subject.

A data processing system 10 includes a data processing device 1 and a three-dimensional scanner 2 . A display 3 , a keyboard 4 and a mouse 5 are connected to the data processing device 1 .

The three-dimensional scanner 2 is an imaging device that takes an image of the intraoral cavity, and acquires three-dimensional data of an object using a built-in three-dimensional camera. Specifically, the three-dimensional scanner 2 scans the intraoral area to obtain positional information (vertical direction, horizontal direction, height coordinates of each axis of the direction) are acquired using an optical sensor or the like. That is, the three-dimensional data is position data including position information of each position of the point group forming the surface of the object.

Since the measurement range that the three-dimensional scanner 2 can measure at one time is limited, when the user wants to acquire three-dimensional data of the entire dental row (dental arch) in the oral cavity, the user moves the three-dimensional scanner 2 along the tooth row. The intraoral area is scanned multiple times by moving the intraoral area using the

The data processing device 1 generates two-dimensional image data corresponding to a two-dimensional image viewed from an arbitrary viewpoint based on the three-dimensional data acquired by the three-dimensional scanner 2, and displays the generated two-dimensional image on the display 3. , the user can see a two-dimensional projection of the surface of the object viewed from a specific direction.

Furthermore, the data processing device 1 outputs the three-dimensional data to the dental laboratory. In the dental laboratory, a dental technician prepares a tooth model such as a prosthesis based on the three-dimensional data acquired from the data processing device 1 . If an automatic manufacturing device capable of automatically manufacturing a tooth model, such as a milling machine or a 3D printer, is installed in the dental clinic, the data processing device 1 may output the three-dimensional data to the automatic manufacturing device. .

[Hardware Configuration of Data Processor]
A hardware configuration of the data processing device 1 according to the first embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing the hardware configuration of the data processing device 1 according to the first embodiment. Data processing apparatus 1 may be realized by a general-purpose computer, or may be realized by a computer dedicated to data processing system 10, for example.

As shown in FIG. 2, the data processing apparatus 1 includes, as main hardware elements, a calculation unit 11, a storage unit 12, a scanner interface 13, a communication unit 14, a display interface 15, and a peripheral device interface 16. , and a reading unit 17 .

The calculation unit 11 is an example of a computer and is a calculation entity (calculation device) that executes various processes by executing various programs. The calculation unit 11 is configured by, for example, a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), and the like. Note that the calculation unit 11 may be configured by at least one of a CPU, an FPGA, and a GPU, or may be configured by all of a CPU and an FPGA, an FPGA and a GPU, a CPU and a GPU, or a CPU, an FPGA, and a GPU. good too. Further, the arithmetic unit 11 may be configured with processing circuitry. Note that the arithmetic unit 11 may be configured with one chip, or may be configured with a plurality of chips. Furthermore, all or part of the functions of the computing unit 11 may be provided in a server device (for example, a cloud-type server device) not shown.

Storage unit 12 includes a volatile storage area (for example, a working area) that temporarily stores program codes, work memory, and the like when operation unit 11 executes an arbitrary program. For example, the storage unit 12 is composed of a volatile memory device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory). Furthermore, the storage unit 12 includes a non-volatile storage area. For example, the storage unit 12 is composed of a nonvolatile memory device such as a ROM (Read Only Memory), a hard disk, or an SSD (Solid State Drive).

In this embodiment, an example in which the volatile storage area and the nonvolatile storage area are included in the same storage unit 12 is shown. It may be contained in different storage units. For example, the calculation unit 11 may include a volatile storage area, and the storage unit 12 may include a nonvolatile storage area. The data processing device 1 may include a microcomputer including an arithmetic section 11 and a storage section 12 .

Storage unit 12 stores data processing program 121 and estimation model 122 . The data processing program 121 generates two-dimensional data from the three-dimensional data acquired by the three-dimensional scanner 2 by the calculation unit 11, and based on the two-dimensional data and the estimation model 122, identifies objects in the oral cavity. process is described.

Estimation model 122 includes neural network 1221 and parameters 1222 used by neural network 1221 . The estimation model 122 uses learning data including two-dimensional data including position information of each of a plurality of objects in the oral cavity and correct labels indicating the types of each of the plurality of objects, based on the two-dimensional data. Machine learning is being done to infer the type of object.

Specifically, in the learning phase, when two-dimensional data including position information of each object in the oral cavity is input to the estimation model 122, the neural network 1221 extracts features of each object based on the two-dimensional data. , estimate the type of each object based on the extracted features. Based on the estimated type of each object and the correct label indicating the type of each object associated with the two-dimensional data, the estimation model 122 does not update the parameter 1222 if both match. If they do not match, the parameter 1222 is optimized by updating the parameter 1222 so that both match. In this way, the estimation model 122 is machine-learned by optimizing the parameters 1222 based on learning data including two-dimensional data as input data and types of objects as correct data. Thereby, the estimation model 122 can estimate each of the multiple objects in the oral cavity based on the two-dimensional data of each of the multiple objects in the oral cavity.

Note that the estimated model 122 optimized by learning such an estimated model 122 is particularly called a "learned model". That is, the pre-learning estimation model 122 and the trained estimation model 122 are collectively referred to as "estimation model", while the trained estimation model 122 is particularly referred to as "learned model".

Estimation model 122 includes a program for execution of estimation processing and learning processing by calculation unit 11 . In the first embodiment, programs that perform image-specific processing include, for example, U-Net, SegNet, ENet, ErfNet, VoxNet, 3D ShapeNets, 3D U-Net, Multi-View CNN, RotationNet, OctNet, PointCNN , FusionNet, PointNet, PointNet++, SSCNet, MarrNet, VoxelNet, PAConv, VGGNet, ResNet, DGCNN, KPConv, FCGF, ModelNet40, ShapeNet, SemanticKITTI, SunRGB-D, VoteNet, LinkNet, Lambda Network, PREDATOR, 3D Medical Point Transformer, and PCT and the like are used to program the estimation model 122, but other programs such as forward propagation neural networks, recurrent neural networks, graph neural networks, attention mechanisms, and transformers are used to program the estimation model 122. may

The scanner interface 13 is an interface for connecting the three-dimensional scanner 2 and realizes input/output of data between the data processing device 1 and the three-dimensional scanner 2 . The data processing device 1 and the three-dimensional scanner 2 are connected via a cable using a cable or wirelessly (WiFi, BlueTooth (registered trademark), etc.).

The communication unit 14 transmits and receives data to and from the above-described dental laboratory or automatic manufacturing apparatus via wired communication or wireless communication. For example, the data processing apparatus 1 transmits data for prosthesis production generated based on three-dimensional data to a dental laboratory or an automatic manufacturing apparatus via the communication unit 14 .

The display interface 15 is an interface for connecting the display 3 and implements data input/output between the data processing device 1 and the display 3 .

The peripheral device interface 16 is an interface for connecting peripheral devices such as the keyboard 4 and the mouse 5, and realizes input/output of data between the data processing apparatus 1 and the peripheral devices.

The reading unit 17 reads various data stored in the removable disk 20, which is a storage medium. For example, reading unit 17 may acquire data processing program 121 from removable disk 20 .

[Configuration of three-dimensional scanner]
The configuration of the three-dimensional scanner 2 according to Embodiment 1 will be described with reference to FIGS. 3 and 4. FIG. FIG. 3 is a diagram showing the configuration of the three-dimensional scanner 2 according to Embodiment 1. As shown in FIG. FIG. 4 is a diagram for explaining an example in which the three-dimensional scanner 2 according to Embodiment 1 acquires three-dimensional data based on the confocal method.

As shown in FIG. 3, the three-dimensional scanner 2 is a hand-held handpiece and includes a housing 21, a probe 22 detachably connected to the housing 21, and a controller 40. As shown in FIG.

The probe 22 is inserted into the oral cavity and projects patterned light (hereinafter also simply referred to as “pattern”) onto objects in the oral cavity. The probe 22 guides the reflected light from the object on which the pattern is projected into the housing 21 .

The three-dimensional scanner 2 includes a light source 23 , a lens 24 , an optical sensor 25 , a prism 26 , a counterweight 27 and an opening 29 inside a housing 21 . 3 and 4, for convenience of explanation, the plane direction parallel to the opening 29 is defined by the X-axis and the Y-axis. Also, the direction perpendicular to the X-axis and the Y-axis is defined by the Z-axis.

The light source 23 includes a laser element, an LED (Light Emitting Diode), or the like. Light (optical axis L) from the light source 23 passes through the prism 26 and the lens 24 , is reflected by the reflector 28 provided in the probe 22 , and is output from the aperture 29 . The light output from the aperture 29 is applied to an object along the Z-axis direction and reflected by the object. That is, the optical axis direction of the light output from the three-dimensional scanner 2 coincides with the Z-axis direction and is orthogonal to the plane direction composed of the X-axis and the Y-axis.

The light reflected by the object enters the interior of the housing 21 again through the opening 29 and the reflecting portion 28, passes through the lens 24, and enters the prism 26. FIG. The prism 26 changes the traveling direction of light from the object to the direction in which the optical sensor 25 is located. The light whose traveling direction is changed by the prism 26 is detected by the optical sensor 25 .

When acquiring three-dimensional data of an object using the confocal technique, light having a pattern such as a checkerboard pattern that passes through a pattern generating element (not shown) provided between the lens 24 and the object is scanned. within the range R is projected onto the object. When the lens 24 reciprocates linearly on the same straight line, the focal position of the pattern projected onto the object changes on the Z-axis. The optical sensor 25 detects light from the object each time the focal position changes on the Z axis.

The control device 40 is composed of, for example, a CPU, a ROM, a RAM, and the like, and controls processing performed by the three-dimensional scanner 2 . Note that the control device 40 may be configured with an FPGA or a GPU. Further, the control device 40 may be composed of at least one of a CPU, an FPGA, and a GPU, or may be composed of a CPU and an FPGA, an FPGA and a GPU, a CPU and a GPU, or all of a CPU, an FPGA, and a GPU. good too. Controller 40 may also be comprised of processing circuitry. Based on the position of the lens 24 and the detection result of the optical sensor 25 at that time, the control device 40 calculates the position information of each point group indicating the surface of the object.

Thereby, the three-dimensional scanner 2 acquires position information (X coordinate and Y coordinate) of each point group indicating the surface of the object on the XY plane within the scanning range R. As shown in FIG. 4, when an object is viewed in the Z-axis direction from a virtual viewpoint between the three-dimensional scanner 2 and the object, a two-dimensional image showing the surface of the object can be represented on the XY plane. The three-dimensional scanner 2 sequentially acquires a bundle of two-dimensional data including such X and Y coordinates in the Z-axis direction, thereby obtaining three-dimensional data (X coordinate, Y coordinate, and Z coordinates) can be obtained. One scan corresponds to obtaining three-dimensional data (X coordinate, Y coordinate, and Z coordinate) of an object once with the position of the probe 22 of the three-dimensional scanner 2 fixed.

More specifically, when one scan is performed to acquire three-dimensional data with the optical axis fixed without moving the three-dimensional scanner 2, the control device 40 moves the surface of the object to be scanned. Three-dimensional position information is given to each of the indicated point groups, with the optical axis direction as the Z coordinate and the planar direction perpendicular to the optical axis direction (Z-axis direction) as the X coordinate and the Y coordinate. When the three-dimensional scanner 2 performs multiple scans, the control device 40 performs matching of overlapping portions when combining the three-dimensional data of the point cloud acquired by each scan between multiple scans. Combine the 3D data based on the shape of the object. When the connection is completed or at a certain timing, the control device 40 newly assigns X, Y, and Z coordinates based on an arbitrary origin to the three-dimensional data of the connected point group. , to obtain 3D point cloud data containing the position information of objects with unity as a whole.

Three-dimensional data of an object acquired by the three-dimensional scanner 2 is input to the data processing device 1 via the scanner interface 13 . Note that the data processing device 1 may have all or part of the functions of the control device 40 . For example, the calculation unit 11 of the data processing device 1 may have the functions of the control device 40 .

[An example of scanning by a three-dimensional scanner]
An example of scanning by the three-dimensional scanner 2 will be described with reference to FIGS. 5 to 8. FIG.

FIG. 5 is a diagram for explaining a scanning method by the three-dimensional scanner 2. FIG. The scanning range of the three-dimensional scanner 2 is limited by the size of the probe 22 that can be inserted into the oral cavity. Therefore, the user inserts the probe 22 into the oral cavity and scans the oral cavity multiple times by moving the probe 22 in the oral cavity along the dentition.

For example, as shown in FIG. 5, by moving the probe 22 in the oral cavity, the user sequentially switches the scanning range to R1, R2, R3, . Acquire 3D data. More specifically, the user scans a portion of the tooth by moving the probe 22 from the lingual side of the tooth through the occlusal side to the labial side of the tooth and performs such a scan from the left posterior side to the anterior side. A plurality of teeth are sequentially scanned by moving the probe 22 to the right molar side via . Since the method of moving the probe 22 in the oral cavity differs for each user or for each dental practice, the part of the oral cavity from which the three-dimensional data is acquired and the acquisition order may change.

FIG. 6 is a diagram showing an object in each scan range acquired by the three-dimensional scanner 2 according to Embodiment 1. FIG. As shown in FIG. 6, the user scans the object while moving the probe 22, so that the three-dimensional scanner 2 can acquire three-dimensional data of the object included in each scanning range. For example, the three-dimensional scanner 2 can acquire three-dimensional data in one scan range for each scan, and in the example of FIG. 6, acquire three-dimensional data in each of the scan ranges R11 to R15. can. The three-dimensional scanner 2 combines a plurality of three-dimensional data corresponding to each of the plurality of scan ranges R11 to R15 obtained by scanning a plurality of times, thereby obtaining an image of the entire object included in the plurality of scan ranges R11 to R15. 3D data can be obtained.

7 and 8 are diagrams showing how the three-dimensional scanner 2 is used to scan an object. As shown in FIGS. 7 and 8, in the oral cavity, the tongue, the frenulum between the mandibular dentition and the tongue, the mandibular dentition, the frenulum between the mandibular dentition and the lower lip (illustrated) are omitted), the lower lip, the hard palate, the frenulum between the maxillary dentition and the hard palate, the maxillary dentition, the frenulum between the maxillary dentition and the upper lip (not shown)), Multiple objects such as upper lip, gums, mucosa, and prostheses (metal, ceramic, resin teeth) are included. During scanning by the three-dimensional scanner 2, an operator's finger or a medical instrument, or a patient's tongue, lips, mucous membranes (back of the cheek), etc. are placed between an object such as a tooth to be scanned and the three-dimensional scanner 2. There is a case where an unnecessary object enters and the 3D scanner 2 cannot properly acquire the 3D data of the object. For example, in the example of FIG. 6, a finger, which is an unnecessary object, is included in the range R12 to R14. Note that the finger is not limited to the operator's real finger, but also includes the operator's gloved finger. Medical instruments also include dental instruments such as vacuums, mouth openings, and tongue depressors.

For example, as shown in FIGS. 7 and 8, when the probe 22 of the three-dimensional scanner 2 is inserted into the oral cavity, a finger is inserted between the tooth and the lip to press the soft tissue in the oral cavity. There is something.

Specifically, as shown in FIG. 7, the user inserts the probe 22 into the gap between the tooth and the lip when acquiring three-dimensional data of the side surface of the tooth on the lip side. Prevent soft tissue from interfering with the scan by pressing the soft tissue by inserting a finger into the gap between the lips. Also, as shown in FIG. 8, when the user acquires three-dimensional data of the side surface of the tooth on the lingual side, the user inserts the probe 22 into the gap between the tooth row and the tongue. Prevent soft tissue from interfering with the scan by pressing the soft tissue by inserting a finger into the gap between the It should be noted that not only the finger but also a medical instrument for pressing soft tissue may be inserted into the gap between the tooth and the lip or the gap between the tooth and the tongue. In this way, when the inside of the oral cavity is scanned with the finger pressing the soft tissue in contact with the dentition, as shown in FIG. is obtained.

In this way, in dental practice, the inside of the oral cavity is usually scanned with a finger and an insertion object such as a medical instrument inserted into the oral cavity. As indicated by R12 to R14, there are cases where an inserted object enters the scanning range and the 3D scanner 2 cannot properly acquire 3D data of the object.

Therefore, the data processing apparatus 1 according to Embodiment 1 utilizes AI (Artificial Intelligence) to detect teeth, tongue, lips, frenulum, gums, mucous membranes, prosthetics (metal teeth, ceramics) in the oral cavity. teeth, resin teeth), and objects to be inserted into the oral cavity. is configured as Specific functions of the data processing device 1 will be described below.

[Functional Configuration of Data Processing Device]
A functional configuration of the data processing device 1 according to the first embodiment will be described with reference to FIG. FIG. 9 is a block diagram showing the functional configuration of the data processing device 1 according to Embodiment 1. As shown in FIG.

As shown in FIG. 9, the data processing apparatus 1 includes, as main functional units, an input unit 1101, a two-dimensional data generation unit 1106, an identification unit 1102, a removal unit 1103, a combination unit 1104, and an image generation unit. 1105 and a storage unit 12 .

An input unit 1101 is a functional unit of the scanner interface 13 and acquires three-dimensional data for each scan acquired by the three-dimensional scanner 2 . Note that the input unit 1101 may be a functional unit of the communication unit 14 , the peripheral device interface 16 , or the reading unit 17 . For example, if the input unit 1101 is a functional unit of the communication unit 14, the communication unit 14 acquires three-dimensional data from an external device via wired communication or wireless communication. Note that the external device may be a server device installed at the dental clinic, or may be a cloud-type server device installed at a location other than the dental clinic. If the input unit 1101 is a functional unit of the peripheral device interface 16 , the peripheral device interface 16 acquires three-dimensional data input by the user using the keyboard 4 and mouse 5 . If the input unit 1101 is a functional unit of the reading unit 17 , the reading unit 17 acquires three-dimensional data stored on the removable disk 20 .

Three-dimensional data input to the data processing apparatus 1 will now be described with reference to FIG. FIG. 10 is a diagram for explaining three-dimensional data input to the data processing device 1 according to the first embodiment. As shown in FIG. 10, the three-dimensional data for each scan input to the input unit 1101 includes X coordinates, Y coordinates, and Z It contains the position information of the coordinates and the normal information of the X, Y and Z components. Although illustration is omitted, the three-dimensional data also includes color information associated with each point group indicating the surface of the object within the scan range.

The positional information is, as described with reference to FIG. 4, the X coordinate, Y coordinate, and Z coordinate of each point group indicating the surface of the object included in the scan range. The normal line information is the X component, Y component, and Z component of the normal line perpendicular to the tangent line at the point of interest, focusing on one point included in the point group. Note that a known technique such as principal component analysis may be used to generate a normal to one point included in the point group.

Returning to FIG. 9 , the two-dimensional data generation unit 1106 is a functional unit of the calculation unit 11 . A two-dimensional data generation unit 1106 generates two-dimensional data from three-dimensional data for one scan input from the input unit 1101 .

Specifically, as described with reference to FIG. 4, the three-dimensional scanner 2 using the confocal technique detects points that indicate the surface of the object by detecting light from the object included in the scanning range. Three-dimensional data including position information for each of the groups can be obtained. As shown in FIG. 10, the three-dimensional data acquired by the three-dimensional scanner 2 and input from the input unit 1101 includes, as position information, the X, Y, and Z coordinates of each point group indicating the surface of the object. , containing the color information at that position. The two-dimensional data generation unit 1106 generates two-dimensional data of each point cloud indicating the surface of the object using only the X and Y coordinates among the position information included in the three-dimensional data input from the input unit 1101. do. The X and Y coordinates are pixel positions of two-dimensional data, and the color information is the pixel value at that pixel position. That is, the two-dimensional data generated based on the three-dimensional data by the two-dimensional data generation unit 1106 is obtained by viewing at least one object included in the scan range from a position (for example, the virtual viewpoint position in FIG. 4) at a certain distance. 2D image data showing the appearance of the at least one object in the case. Two-dimensional data generation section 1106 outputs the generated two-dimensional data to identification section 1102 .

Note that the two-dimensional data generation unit 1106 uses the X, Y, and Z coordinates of the position information included in the three-dimensional data input from the input unit 1101 to generate each point group indicating the surface of the object. , two-dimensional data may be generated as a distance image. That is, the two-dimensional data generation unit 1106 may use the X and Y coordinates as the pixel positions of the two-dimensional data, and convert the Z coordinates into pixel values at the pixel positions. A distance image is two-dimensional data in which the Z coordinate is represented by color information including image gradation. Furthermore, the two-dimensional data generation unit 1106 uses two-dimensional data representing the surface of an object generated using only the X and Y coordinates, and a distance image generated using the X, Y, and Z coordinates. Both two-dimensional data may be generated. Using the Z coordinate as color information in this way is useful when humans visually view a two-dimensional image (distance image). For example, in a distance image (an image in which the Z coordinate is converted to a pixel value) when the back teeth are scanned from above, the color near the occlusal surface of the tooth is close to white, and the back side of the gingiva is close to black. . That is, the height difference in the back teeth can be represented by black and white. On the other hand, in the case of a normal two-dimensional image such as a color photograph, the shape of the back tooth is represented by the color or the contour on the XY plane, and the height difference cannot be represented. In particular, in machine learning, when it is difficult to determine whether the scanned object is gum, mucous membrane (back side of cheek), or lip only from a two-dimensional image such as a color photograph, Differential range images can be used to identify each object. Also, in a computer (AI) such as the computing unit 11, when pixel values are used for the Z coordinates such as in a distance image, the distance between a plurality of adjacent objects is greater than when simply using the height of a shape for the Z coordinates. Since the relationship between is easier to understand, it becomes easier to calculate using a convolution operation.

The identification unit 1102 is a functional unit of the calculation unit 11 . The identification unit 1102 identifies at least one object among a plurality of objects based on the two-dimensional data input from the two-dimensional data generation unit 1106 and the estimation model 122 . The identifying section 1102 outputs the identification result to the removing section 1103 .

Here, machine learning of the estimation model 122 will be described with reference to FIG. 11 . FIG. 11 is a diagram for explaining learning data used when performing machine learning on the estimation model 122 according to the first embodiment. As shown in FIG. 11, the estimation model 122 according to Embodiment 1 shows two-dimensional data including X coordinates and Y coordinates and each of the plurality of objects as position information of each of the plurality of objects in the oral cavity. Machine learning (supervised learning) is performed to estimate each of a plurality of objects based on one scan of two-dimensional data using one scan of learning data including correct labels. That is, the estimation model 122 is machine-learned so as to estimate an object based on two-dimensional data including X and Y coordinates without using Z coordinates.

Here, the relative positional relationship between multiple objects will be described. In the oral cavity, anatomically, the positions of each of a plurality of objects such as teeth and tongue are predetermined in relation to arbitrary landmarks, that is, in relative relation. For example, as shown in FIGS. 7 and 8 described above, when the face is viewed from the front with the mouth open, in the lower jaw, from near the center of the upper or lower jaw (deep side of the oral cavity) to the outside (opening direction) In order, the tongue, the frenulum between the mandibular dentition and the tongue, the mandibular dentition, the frenulum between the mandibular dentition and the lower lip (not shown), and the lower lip are located. . In other words, if the dentition of the mandible is the starting point (arbitrary landmark), the tongue is positioned further back in the mouth than the dentition of the mandible, and the lower lip is positioned closer to the mouth than the dentition of the mandible. do. In addition, when the face is viewed from the front with the mouth open, in the upper jaw, from near the center of the upper or lower jaw (back side of the oral cavity) to the outside (opening direction), the hard palate, the dentition of the upper jaw and the hard palate The frenulum between the teeth, the maxillary dentition, the frenulum between the maxillary dentition and the upper lip (not shown), and the upper lip are located. In other words, when the maxillary dentition is the starting point (arbitrary landmark), the hard palate is located further back in the oral cavity than the maxillary dentition, and the upper lip is closer to the opening of the oral cavity than the maxillary dentition. To position. That is, in the oral cavity, relative positional relationships among a plurality of objects such as teeth, tongue, and lips are fixed.

In this way, since the relative positional relationship between a plurality of objects in the oral cavity is determined, the three-dimensional data including the position information of the objects, which is the input data of the estimation model 122, and the output data of the estimation model 122 It can be said that there is a correlation with the identification result of the type of the object. That is, since there is a correlation between the input data and the output data such that the position information of the object included in the three-dimensional data is associated with the type of the object, the estimation model 122 Based on the three-dimensional data including the position information, the type of the object can be identified by specifying which region in the oral cavity contains the position corresponding to the three-dimensional data.

FIG. 12 is a diagram for explaining the correspondence relationship between each of a plurality of objects and correct labels. As shown in FIG. 12, data indicating "01" as a correct label is associated with the lower jaw and the tongue. Data indicating "02" as a correct label is associated with the mandibular first gap. Data representing "31" to "48" as correct labels are associated with each of the plurality of teeth included in the row of teeth of the lower jaw. Data indicating "04" as a correct label is associated with the second mandibular gap. Data representing "05" as a correct label is associated with the lower lip.

As for the upper jaw, data indicating "06" as a correct label is associated with the hard palate. Data indicating "07" as a correct label is associated with the maxillary first gap. Data representing "11" to "28" as correct labels are associated with each of the plurality of teeth included in the dentition of the upper jaw. Data indicating “09” as a correct label is associated with the second maxillary gap. Data indicating "10" is associated with the upper lip as a correct label.

As mentioned above, in dental practice, insert objects such as fingers and medical instruments may be inserted into the oral cavity. Among the inserted objects, the finger is associated with data indicating "51" as a correct label. Among the inserted objects, data indicating "52" as the correct label is associated with the medical instrument.

Returning to FIG. 9, in the learning data, each two-dimensional data (position information) of the point cloud indicating the surface of each of a plurality of objects obtained in one scan corresponds to the correct label indicating the type of object. attached.

For example, as shown in FIG. 11, data indicating "01" as a correct label is associated with two-dimensional data generated from three-dimensional data obtained by scanning the tongue. Two-dimensional data generated from three-dimensional data obtained by scanning the mandibular left third molar is associated with data indicating "38" as a correct label. Two-dimensional data generated from three-dimensional data obtained by scanning the mandibular left second molar is associated with data indicating "37" as a correct label. Furthermore, data indicating "51" as a correct label is associated with the two-dimensional data generated from the three-dimensional data obtained by scanning the finger inserted into the mandibular second gap.

Thus, in the first embodiment, two-dimensional data (X coordinate , Y coordinates) are associated with correct labels indicating the types of objects.

The estimation model 122 identifies each type of a plurality of scanned objects based on two-dimensional data for one scan, and adjusts the parameter 1222 based on the degree of matching between the identification result and the correct label.

As a result, the estimation model 122 performs machine learning to identify the type of object corresponding to the two-dimensional data based on the correct label associated with the two-dimensional data for one scan. will be able to identify the type of object corresponding to

Furthermore, since the estimation model 122 is machine-learned based on two-dimensional data whose dimension is reduced by the Z-coordinate, it is possible to perform machine-learning while reducing the computational load compared to using three-dimensional data including the Z-coordinate. can be done.

Returning to FIG. 9 , the removal unit 1103 is a functional unit of the calculation unit 11 . The removal unit 1103 acquires the identification result of the object type from the identification unit 1102 . As shown in FIG. 12, the identification results are the tongue, the first mandibular gap, each tooth included in the mandibular dentition, the second mandibular gap, the lower lip, the hard palate, the first maxillary gap, and the dentition of the upper jaw. Identification results for each of the teeth, maxillary second gaps, and upper lip, as well as identification results for inserted objects such as fingers and medical instruments are also included. If the identification result of the identification unit 1102 includes the identification result of an unnecessary object such as an operator's finger, a medical instrument, or a patient's tongue, the removal unit 1103 removes the three-dimensional data from the three-dimensional data input from the input unit 1101. , and the unnecessary three-dimensional data including the position information of each point group indicating the surface of the unnecessary object identified by the identification unit 1102 is removed to obtain the three-dimensional data after removing the unnecessary object (hereinafter referred to as “unnecessary three-dimensional data”). ) is generated.

Specifically, the removing unit 1103 extracts the position information (X coordinate, Y coordinate) in the XY plane direction of each point group representing the surface of the unnecessary object, and extracts the positional information (X coordinate, Y coordinate) corresponding to each point group representing the surface of the unnecessary object. Unnecessary three-dimensional data is extracted by extracting position information (Z coordinate) in the direction of the optical axis. For example, based on the two-dimensional data generated by the two-dimensional data generation unit 1106, the removal unit 1103 extracts the X and Y coordinates of each point group indicating the surface of the unnecessary object identified by the identification unit 1102. Further, the removal unit 1103 uses the extracted X and Y coordinates of the unnecessary object as search keys, and based on the three-dimensional data acquired by the input unit 1101, removes the Z coordinate corresponding to the X and Y coordinates of the unnecessary object. to extract The removing unit 1103 can use the X, Y, and Z coordinates of the extracted unnecessary object as unnecessary three-dimensional data. Here, extracting an unnecessary object includes, for example, storing identification data that can identify the unnecessary object in association with each data of the X coordinate, Y coordinate, and Z coordinate. The removing unit 1103 removes unnecessary 3D data from the 3D data input from the input unit 1101, thereby generating 3D data after removing unnecessary objects. The removing unit 1103 outputs the three-dimensional data after removing the unnecessary object to the combining unit 1104 .

A coupling unit 1104 is a functional unit of the calculation unit 11 . The combining unit 1104 acquires the three-dimensional data for one scan from the removal unit 1103 each time three-dimensional data for one scan is input to the input unit 1101, and combines the accumulated three-dimensional data for a plurality of scans. 3D data for a plurality of combined scans (hereinafter also referred to as "combined data") is generated.

FIG. 13 is a diagram for explaining combined data generated by the data processing device 1 according to the first embodiment after unnecessary object removal. If the identification result of the identification unit 1102 includes the identification result of the operator's finger or medical instrument, or the patient's tongue, lips, mucous membrane, or other unnecessary object, the removal unit 1103 performs the following operations as shown in FIG. A removal flag is set for three-dimensional data (unnecessary three-dimensional data) corresponding to the unnecessary object identified by the identifying unit 1102 . In the example of FIG. 13, "01" is set as the removal flag for the unnecessary three-dimensional data corresponding to the tongue, lips, mucous membranes, and fingers to be removed. The data processing device 1 is configured not to use the unnecessary three-dimensional data whose removal flag is set to “01” for the two-dimensional image displayed on the display 3 . Based on the three-dimensional data input from the removing unit 1103, the combining unit 1104 generates combined data after unnecessary object removal by combining three-dimensional data for a plurality of scans as shown in FIG.

Returning to FIG. 9 , combining section 1104 outputs combined data to storage section 12 and image generating section 1105 . Storage unit 12 stores the combined data input from combining unit 1104 . An image generation unit 1105 is a functional unit of the calculation unit 11 . The image generating unit 1105 generates two-dimensional image data corresponding to a two-dimensional image viewed from an arbitrary viewpoint based on the combined data input from the combining unit 1104, and outputs the generated two-dimensional image data to the display 3. . At this time, the image generation unit 1105 generates two-dimensional image data without using the unnecessary three-dimensional data whose removal flag is set to “01”. As a result, the data processing device 1 can display a two-dimensional image of the oral cavity from which the unnecessary objects have been removed on the display 3 for the user to see.

[Processing flow of data processor]
FIG. 14 is a diagram showing an example of processing executed by the data processing device 1, with reference to FIG. 14; FIG. 14 is a flowchart for explaining an example of processing executed by the data processing device 1 according to the first embodiment. Each step shown in FIG. 14 (hereinafter denoted by “S”) is implemented by the arithmetic unit 11 of the data processing device 1 executing the data processing program 121 . Further, after the scanning by the three-dimensional scanner 2 is started, the data processing device 1 executes the processing of the flowchart shown in FIG.

As shown in FIG. 14, the data processing device 1 acquires three-dimensional data of an arbitrary point scanned by the three-dimensional scanner 2 (S11). The data processing device 1 generates two-dimensional data from the acquired three-dimensional data using only the X and Y coordinates (S12). The data processing device 1 identifies each of the plurality of scanned objects based on the two-dimensional data and the estimation model 122 (S13).

The data processing device 1 determines whether or not an unnecessary object is detected based on the identification result (S14). That is, the data processing apparatus 1 outputs, as the identification result, data indicating "01" corresponding to the tongue, data indicating "51" corresponding to the finger, data indicating "52" corresponding to the medical instrument, data indicating "52" corresponding to the lower lip, It is determined whether or not the corresponding data indicating "05" and the data indicating "10" corresponding to the upper lip have been output. When an unnecessary object is detected (YES in S14), the data processing device 1 extracts unnecessary three-dimensional data corresponding to the detected unnecessary object, and removes the extracted unnecessary three-dimensional data (S15). That is, the data processing device 1 sets a removal flag for unnecessary three-dimensional data.

If no unnecessary object is detected (NO in S14), or after removing unnecessary three-dimensional data in S45, the data processing device 1 generates combined data by combining three-dimensional data for multiple scans. (S16).

The data processing device 1 stores the combined data in the storage unit 12 (S17). Furthermore, the data processing device 1 generates two-dimensional image data corresponding to a two-dimensional image viewed from an arbitrary viewpoint based on the combined data, and outputs the generated two-dimensional image data to the display 3, so that the intraoral is displayed on the display 3 (S18).

The data processing device 1 determines whether scanning by the three-dimensional scanner 2 has stopped (S19). If scanning by the three-dimensional scanner 2 has not stopped (NO in S19), the data processing device 1 returns the process to S11. On the other hand, when the scanning by the three-dimensional scanner 2 is stopped (YES in S19), the data processing device 1 ends this process.

As described above, the data processing device 1 identifies unnecessary objects unnecessary for dental treatment among the plurality of objects scanned by the three-dimensional scanner 2 using the estimation model 122, and displays the surfaces of the identified unnecessary objects. Unnecessary 3D data including position information of each point cloud can be extracted. As a result, the user can easily and appropriately extract the three-dimensional data of the unnecessary object without having to extract the three-dimensional data of the unnecessary object by himself/herself.

Furthermore, the data processing device 1 can identify each of a plurality of objects in the oral cavity using the estimation model 122 based on the two-dimensional data whose dimension is reduced by the Z coordinate. Compared to identifying each of a plurality of objects in the oral cavity using original data, it is possible to extract three-dimensional data of unnecessary objects while reducing the load of arithmetic processing.

Since the data processing device 1 can remove unnecessary 3D data from the 3D data input by the 3D scanner 2, the user can remove the 3D data of the unnecessary object by himself/herself and remove the unnecessary object after removing the unnecessary object. There is no need to create three-dimensional data, and three-dimensional data after unnecessary objects have been removed can be easily obtained.

Since the data processing device 1 outputs to the display 3 the image data generated using the three-dimensional data after removing the unnecessary three-dimensional data, the user can see the two-dimensional image of the oral cavity after removing the unnecessary objects. can be easily obtained after the unnecessary objects are removed.

<Embodiment 2>
Embodiment 2 of the present disclosure will be described in detail with reference to the drawings. In the second embodiment, only parts different from the first embodiment will be described, and the same parts as in the first embodiment will be denoted by the same reference numerals, and description thereof will not be repeated.

FIG. 15 is a diagram for explaining an example in which the three-dimensional scanner 102 according to Embodiment 2 acquires three-dimensional data based on the triangulation method. FIG. 16 is a diagram showing a two-dimensional image viewed from an arbitrary viewpoint based on three-dimensional data acquired by the three-dimensional scanner 102 according to the second embodiment. Unlike the three-dimensional scanner 2 according to the first embodiment, the three-dimensional scanner 102 according to the second embodiment acquires three-dimensional data based on triangulation.

As shown in FIG. 15, in the triangulation method, a pattern is projected onto an object by a projector 8, and the pattern projected onto the object is imaged by a camera 9 located at a position different from that of the projector 8. FIG. As shown in FIG. 15A, when the projector 8 projects a linear pattern onto the object, the angle formed by the line connecting the projector 8 and the object and the line connecting the camera 9 and the object , and as shown in FIG. 15B, the pattern image obtained by imaging with the camera 9 shows a pattern along the shape of the object. As shown in FIG. 15C, the three-dimensional scanner 102 measures the length of the line connecting the projector 8 and the object, the length of the line connecting the camera 9 and the object, and the length of the line connecting the projector 8 and the camera 9. Based on the length and angle of each vertex of the triangles generated by these lines, known triangulation techniques are used to locate the projected pattern on the object.

As shown in FIG. 16, the three-dimensional scanner 102 includes a projector 8 that projects a pattern onto an intraoral object, and a camera 9 that captures the pattern projected onto the object. As shown in FIG. 16A, the three-dimensional scanner 102 captures an image of the object with the camera 9 while the pattern is projected onto the object by the projector 8 . The three-dimensional scanner 102 acquires position information (X coordinate, Y coordinate, Z coordinate) of each point group indicating the surface of the object using a known triangulation method based on the captured image of FIG. 16(A). .

Furthermore, the three-dimensional scanner 102 may image an object with the camera 9 while the pattern is not projected on the object by the projector 8, as shown in FIG. 16(B). As described above, the data processing device 300 according to Embodiment 3 is configured to identify each of a plurality of intraoral objects based on two-dimensional data. Each of a plurality of intraoral objects may be identified based on the captured image of FIG. 16(B) acquired by . Furthermore, the data processing device 300 may extract the Z coordinate of the unnecessary object acquired based on the captured image of FIG. 16A using the X coordinate and Y coordinate of the extracted unnecessary object as search keys.

The three-dimensional scanner 102 performs a process of capturing an image of the object with the camera 9 while the pattern is projected onto the object by the projector 8, and capturing an image of the object with the camera 9 while the pattern is not projected onto the object by the projector 8. 16A and 16B may be obtained by switching the processing. Alternatively, the three-dimensional scanner 102 includes a plurality of cameras 9 , a first camera captures an image of the object with a pattern projected onto the object by the projector 8 , and a second image is captured while the pattern is not projected onto the object by the projector 8 . 16(A) and 16(B) may be obtained by capturing an image of an object with two cameras.

<Modification>
The present disclosure is not limited to the above embodiments, and various modifications and applications are possible. Modifications applicable to the present disclosure will be described below.

In the above-described embodiment, the "predetermined object" is an unnecessary object unnecessary for dental treatment, but the "predetermined object" is not limited to the unnecessary object, and may be a specific object arbitrarily designated in advance by the user. may For example, when caries is found in a specific tooth (for example, the lower left third molar) among a plurality of teeth in the oral cavity, the user picks up only the specific tooth and uses it as a "predetermined object". May be specified. In this case, the data processing device 1 generates two-dimensional data from the three-dimensional data of the oral cavity acquired by the three-dimensional scanner 2, and identifies a specific tooth designated by the user based on the generated two-dimensional data. , is configured to extract predetermined three-dimensional data including position information for each of the point clouds indicative of the identified particular tooth surface.

The identification unit 1102 is not limited to identifying each of a plurality of objects in the oral cavity using the estimation model 122. By pattern matching based on a feature amount designed in advance, that is, a predetermined shape included in a two-dimensional image, Each of a plurality of objects within the oral cavity may be identified. As described above, the two-dimensional data generated by the two-dimensional data generation unit 1106 is at least one object in the oral cavity viewed from a position (for example, the virtual viewpoint position in FIG. 4) at a certain distance. It is two-dimensional image data showing the appearance of one object. The identifying unit 1102 identifies the object (such as a tooth or an unnecessary object) from a two-dimensional image representing the appearance of an intraoral object (such as a tooth or an unnecessary object) to be identified, which is generated by the two-dimensional data generating unit 1106. Recognizing an image, determining the degree of matching between the image of the recognized object (for example, a tooth or an unnecessary object) and a pre-designed feature amount, that is, a predetermined shape included in the two-dimensional image, and determining Based on the results, the object of interest (eg, tooth or unwanted object) to be identified may be identified.

The three-dimensional data input to the input unit 1101 includes color information (RGB values) representing the actual color of each point group, in addition to the position information and normal line information of each point group representing the surface of the object. may be included. Furthermore, the estimation model 122 may be machine-learned so as to identify the type of object based on the color information (RGB values) associated with the three-dimensional data input to the input unit 1101 . Note that the three-dimensional data input to the input unit 1101 may contain only position information for each point group, and may not contain normal line information and color information.

The removal unit 1103 is not limited to removing unnecessary three-dimensional data, and may add color information indicating unnecessary objects to unnecessary three-dimensional data. The image generating unit 1105 is not limited to generating a two-dimensional image of the intraoral cavity after the unnecessary object has been removed. may be generated and output to the display 3.

As a three-dimensional measurement method, in addition to the above techniques, triangulation methods such as random pattern projection or SfM (Structure From Motion) and SLAM (Simultaneous Localization and Mapping) without pattern projection may be used. Of Flight) or laser technology such as LIDAR (Light Detection and Ranging) may be used.

It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present disclosure is indicated by the scope of claims rather than the above description, and is intended to include all modifications within the meaning and scope of equivalents of the scope of claims. Note that the configurations exemplified in this embodiment and the configurations exemplified in the modifications can be combined as appropriate.

1,200,300,400 data processing device, 2,102 three-dimensional scanner, 3 display, 4 keyboard, 5 mouse, 7 intraoral camera, 8 projector, 9 camera, 10 data processing system, 11 calculation unit, 12 storage unit , 13 scanner interface, 14 communication unit, 15 display interface, 16 peripheral device interface, 17 reading unit, 20 removable disk, 21 housing, 22 probe, 23 light source, 24 lens, 25 optical sensor, 26 prism, 27 counterweight, 28 Reflector 29 Aperture 40 Control Device 121 Data Processing Program 122 Estimation Model 1101 Input Unit 1102 Identification Unit 1103 Removal Unit 1104 Combiner 1105 Image Generator 1106 Two-Dimensional Data Generator 1221 Neural network, 1222 parameters.

Claims (9)

A data processing device for processing data containing position information of at least one object in the oral cavity,
an input unit into which three-dimensional data including position information of each point cloud indicating the surface of the at least one object is input;
A computing unit that processes the three-dimensional data input from the input unit,
The calculation unit is
generating two-dimensional data from the three-dimensional data;
identifying a predetermined object including at least one of an insertion object to be inserted into the oral cavity, a tongue, lips, and a mucous membrane among the at least one object based on the two-dimensional data;
extracting predetermined three-dimensional data including position information of each point group indicating the surface of the identified predetermined object;
A data processing device that removes the predetermined three-dimensional data from the three-dimensional data input from the input unit. The insertion object includes at least one of a finger and a medical instrument inserted to hold soft tissue in the oral cavity when an imaging device for imaging the intraoral cavity is inserted into the oral cavity. 2. A data processing apparatus according to claim 1. 3. The data processing apparatus according to claim 1, wherein said calculation unit outputs image data generated using three-dimensional data from which said predetermined three-dimensional data has been removed. The three-dimensional data input from the input unit is acquired by an imaging device that is inserted into the oral cavity and images the oral cavity based on a confocal method or a triangulation method,
The two-dimensional data includes position information of each point group indicating the surface of the at least one object in a plane direction perpendicular to the optical axis direction of the imaging device,
The computing unit extracts position information in the planar direction of each point group indicating the surface of the predetermined object, and extracts position information in the optical axis direction corresponding to each point group indicating the surface of the predetermined object. 4. The data processing device according to any one of claims 1 to 3, wherein said predetermined three-dimensional data is extracted by extracting . The two-dimensional data generated based on the three-dimensional data is data of a two-dimensional image showing the appearance of the at least one object when the at least one object is viewed from a position spaced apart by a certain distance,
2. The data processing apparatus according to claim 1, wherein said calculation unit identifies said predetermined object by recognizing an image of said predetermined object included in said two-dimensional image. The computing unit extracts position information in a direction perpendicular to a planar direction indicated by the two-dimensional image, corresponding to the position information of the predetermined object included in the two-dimensional image, thereby extracting the predetermined three-dimensional object. 6. A data processing apparatus as claimed in claim 5, for extracting data. The computing unit calculates the predetermined object based on the two-dimensional data and an estimation model subjected to machine learning for estimating the predetermined object from the at least one object based on the two-dimensional data. 2. A data processing apparatus as claimed in claim 1, which identifies . A data processing method for processing data containing position information of at least one object in the oral cavity by a computer, comprising:
a step of inputting three-dimensional data including position information of each point cloud indicating the surface of the at least one object;
and processing the input three-dimensional data;
The processing step includes:
generating two-dimensional data from the three-dimensional data;
identifying a predetermined object among the at least one object, including at least one of an insertion object to be inserted into the oral cavity, tongue, lips, and mucosa, based on the two-dimensional data;
a step of extracting predetermined three-dimensional data including position information of each point group indicating the surface of the identified predetermined object;
and removing the predetermined three-dimensional data from the input three-dimensional data. A data processing program for processing data containing position information of at least one object in the oral cavity,
to the computer,
a step of inputting three-dimensional data including position information of each point cloud indicating the surface of the at least one object;
and executing a step of processing the input three-dimensional data;
The processing step includes:
generating two-dimensional data from the three-dimensional data;
identifying a predetermined object among the at least one object, including at least one of an insertion object to be inserted into the oral cavity, tongue, lips, and mucosa, based on the two-dimensional data;
a step of extracting predetermined three-dimensional data including position information of each point group indicating the surface of the identified predetermined object;
and removing the predetermined three-dimensional data from the input three-dimensional data.

Images