JP2008262555A - Shape information processing method, device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a calculation time required for positioning while maintaining precision and efficiency of the positioning between shapes shown by predetermined information. <P>SOLUTION: In matching between a first three-dimensional shape shown by first information and a second three-dimensional shape shown by second information, input of the first information and the second information is received. Then, a plurality of reference points are set on the first three-dimensional shape. As to each of a plurality of set reference points, a corresponding point at the shortest distance from the reference point is found from a limited area on the second three-dimensional shape. When the first information is converted based on positional relationships between a plurality of set reference points and the found corresponding points, matching between the first and second shapes is carried out. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a shape information processing method, a shape information processing apparatus, and a shape information processing program for aligning shapes indicated by predetermined information.

  Conventionally, a technique for aligning a plurality of shapes is known. For example, a certain device measures a part of an object from a plurality of viewpoints, acquires a plurality of pieces of information indicating the three-dimensional shape of each part of the object, and aligns the three-dimensional shapes to represent a three-dimensional representation of the entire object. There are techniques for generating shapes. Such a method is used in various research and development fields and industrial fields such as computer graphics, CAD (Computer-Aided Design), and computer vision due to the recent spread of three-dimensional measuring devices. .

  As a typical method for performing such alignment, there is an iterative nearest point algorithm (hereinafter referred to as “ICP algorithm”). In this ICP algorithm, first, an initial state in which two shapes overlap to some extent is assumed. Subsequently, for each of a plurality of reference points of one shape, a corresponding point at the shortest distance from the reference point is extracted from the other shape. Subsequently, provisional association between a plurality of reference points and a plurality of corresponding points is performed, and coordinate transformation parameters are estimated based on the provisional association. By repetitively performing such provisional association and estimation of coordinate transformation parameters, the two shapes are aligned.

As described above, the ICP algorithm is highly versatile because it calculates and associates the corresponding point at the shortest distance from the reference point. Further, according to this ICP algorithm, when the shape on the side for calculating the rigid body motion is a part of the other shape, the alignment can be performed with high accuracy. Techniques using this ICP algorithm are disclosed in, for example, the following Patent Documents 1 to 4.
Japanese Patent Application Laid-Open No. 06-044361 JP 2000-251092 A JP 2002-175521 A JP 2003-162549 A

  However, in the ICP algorithm, in order to extract one corresponding point, it is necessary to calculate the Euclidean distance for all points that are candidates for corresponding points (hereinafter referred to as “candidate points”). Moreover, such a calculation needs to be performed for all corresponding points to be extracted, and estimation of coordinate transformation parameters is performed iteratively. Therefore, as the number of point pairs increases, the amount of calculation in the ICP algorithm becomes enormous and the calculation time for shape conversion becomes longer.

  As a method for shortening the calculation time, there are considered a method for reducing the number of reference points, and a method for performing automatic processing after a shape to be aligned is placed in an artificially close position in advance. However, if the number of reference points is reduced, the alignment accuracy is lowered. Further, artificially assisting automatic processing is complicated and inefficient.

  The present invention has been made to solve the above-described problem, and reduces the calculation time required for the alignment while maintaining the accuracy and efficiency in aligning the shapes indicated by the predetermined information. An object is to provide a shape information processing method, a shape information processing apparatus, and a shape information processing program.

  The shape information processing method according to the present invention includes a receiving step for receiving input of first information indicating a first shape and second information indicating a second shape, and first information received in the receiving step. A setting step in which a plurality of reference points are provided on one of the first shape indicated by 2 and the second shape indicated by the second information received in the receiving step, and a plurality of values set in the setting step. For each of the reference points, the calculation step for calculating the corresponding point at the shortest distance from the reference point from the limited range of the other shape, the plurality of reference points set in the setting step, and the calculation step Position alignment for aligning the first shape and the second shape by converting the first information or the second information based on the positional relationship with the plurality of corresponding points. Including the step to, the.

  The shape information processing apparatus according to the present invention includes a receiving unit that receives input of first information indicating a first shape and second information indicating a second shape, and a first information received by the receiving unit. Setting means for providing a plurality of reference points on one of the first shape indicated by the information and the second shape indicated by the second information received by the accepting means, and the setting means For each of the plurality of reference points, the calculation means for calculating the corresponding point at the shortest distance from the reference point from the limited range of the other shape, the plurality of reference points set by the setting means, and the calculation means Alignment means for aligning the first shape and the second shape by converting the first information or the second information based on the calculated positional relationship with the plurality of corresponding points; Prepare.

  The shape information processing program according to the present invention includes a reception function for receiving input of first information indicating the first shape and second information indicating the second shape, and the first information received by the reception function. A setting function for providing a plurality of reference points on one of the first shape indicated by the information and the second shape indicated by the second information received by the acceptance function, and the setting function. For each of the plurality of reference points, a calculation function for calculating a corresponding point at the shortest distance from the reference point from a limited range of the other shape, a plurality of reference points set by the setting function, and a calculation function An alignment function for aligning the first shape and the second shape by converting the first information or the second information based on the calculated positional relationship with the plurality of corresponding points; Real to computer Make.

  According to such a shape information processing method, shape information processing apparatus, and shape information processing program, first, one of the first shape indicated by the first information and the second shape indicated by the second information A plurality of reference points are provided on the shape. Subsequently, for each of the set reference points, the corresponding point at the shortest distance from the reference point is calculated from the limited range of the other shape. Then, the first shape and the second shape are aligned by converting the first information or the second information based on the positional relationship between the plurality of reference points and the plurality of corresponding points. . Thereby, when aligning the first shape and the second shape, the range for calculating the corresponding point is limited, so that the corresponding point is compared with the case where the corresponding point is calculated from the entire other shape. The time required to calculate is reduced. At this time, it is not necessary to reduce the number of reference points or add an artificial operation. Therefore, it is possible to reduce the calculation time required for the alignment while maintaining the accuracy and efficiency in aligning the shapes indicated by the predetermined information.

  In the shape information processing method of the present invention, the calculation step is based on the position of the calculated corresponding point corresponding to the first reference point and the position of the second reference point different from the first reference point. Preferably, a range around the calculated corresponding point is calculated, and the range is set as a limited range for calculating the corresponding point at the shortest distance from the second reference point. According to such a configuration, based on the known information (the position of the known reference point and the position of the known corresponding point), the unknown corresponding point (the corresponding point at the shortest distance from the second reference point) is determined. Since a limited range for calculation is set, an unknown corresponding point can be calculated earlier.

  In the shape information processing method of the present invention, the limited range is based on the calculated corresponding point and based on the position of the second reference point and the shortest distance field indicating the predetermined neighboring space of the other three-dimensional shape. The radius is preferably a value obtained by adding the distance defined in the above and the distance between the second reference point and the calculated corresponding point. According to such a configuration, the distance defined based on the position of the calculated corresponding point, the position of the second reference point, and the shortest distance field indicating the predetermined neighboring space of the other three-dimensional shape, A limited range for calculating an unknown corresponding point is set based on the distance between the second reference point and the calculated corresponding point. Therefore, an unknown corresponding point can be calculated earlier.

  In the shape information processing method of the present invention, the limited range is centered on the calculated corresponding point, the distance between the first reference point and the calculated corresponding point, the position of the second reference point, and the other The radius is a value obtained by adding a distance defined based on the shortest distance field indicating a predetermined neighborhood space of the three-dimensional shape and the distance between the first reference point and the second reference point. Is preferred. According to such a configuration, the position of the calculated corresponding point, the distance between the first reference point and the calculated corresponding point, the position of the second reference point, and the predetermined neighborhood of the other three-dimensional shape A limited range for calculating an unknown corresponding point is set based on the distance defined based on the shortest distance field indicating the space and the distance between the first reference point and the second reference point. Is done. Therefore, an unknown corresponding point can be calculated earlier.

  In the present invention, the corresponding point at the shortest distance from the reference point is calculated from the limited range of the other shape. At this time, it is not necessary to reduce the number of reference points or add an artificial operation. Therefore, according to the present invention, it is possible to reduce the calculation time required for the alignment while maintaining the accuracy and efficiency in aligning the shapes indicated by the predetermined information.

  Hereinafter, preferred embodiments of a shape information processing method, a shape information processing apparatus, and a shape information processing program according to the present invention will be described in detail with reference to the drawings. In the present embodiment, the shape information processing method, the shape information processing apparatus, and the shape information processing program are applied to a surgery support information display device, a surgery support information display method, and a surgery support information display program. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted. Further, the dimensional ratios in the drawings do not necessarily match those described.

  FIG. 1 is a configuration diagram schematically showing an embodiment of a surgery support information display device 1 according to the present invention. The surgery support information display device 1 is a device that provides information about an image captured by an endoscope to an operator or the like during surgery on a patient 60. Surgery in which the surgery support information display device 1 according to the present embodiment is used is intended for an operation in which imaging is performed by an endoscope, such as endoscopic surgery of a sinus in an otolaryngology department.

  As shown in FIG. 1, the surgery support information display device 1 includes an endoscope 11, a marker sphere 12, an imaging device 20, a CT device 30, a PC (Personal Computer) 40, and a monitor 50. It is configured.

  The endoscope 11 is a device that is operated by an operator and is inserted into a patient 60 to image the inside. The endoscope 11 has an elongated shape so that it can be inserted into a patient's living body, and a mechanism for imaging the inside of the patient 60 is provided at the distal end portion thereof. The mechanism is, for example, an imaging element such as a lens that is positioned so as to face the imaged portion and a CCD image sensor (Charge Coupled Device Image Sensor) that is provided at the determined position of the lens. The imaging direction A of the endoscope 11 is determined depending on how the mechanism is positioned. Usually, the optical axis direction of the lens is the imaging direction A of the endoscope 11. Information on the image captured by the endoscope 11 is output to the PC 40 connected to the endoscope 11 by a cable. The endoscope 11 does not need to have a special configuration, and a conventionally used endoscope can be used.

  The marker sphere 12 is a marker that is fixedly provided at a predetermined relative positional position with respect to the imaging direction of the endoscope 11. The marker sphere 12 is imaged by the imaging device 20, and the three-dimensional coordinates are obtained from the captured image. Specifically, the marker sphere 12 is a spherical member having a different size, which is fixed to the plurality of endoscopes 11 via a bar-shaped member 13. The reason why the sizes are different is to distinguish and detect each of the images captured by the imaging device 20.

  The position at which the marker ball 12 is provided in the endoscope 11 is a position that is not further inserted into the patient 60, further rearward from the portion inserted into the patient 60. Further, from the portion inserted into the patient 60 to the portion where the marker ball 12 is provided in the endoscope 11 so that the positional relationship between the marker ball 12 and the imaging direction A of the endoscope 11 is constant. It is made of a hard material and cannot be bent. However, since it is only necessary to know the positional relationship between the imaging direction of the endoscope 11 and the marker sphere 12, for example, only the distal end portion of the endoscope 11 may move in a predetermined direction. .

  Note that the marker provided in the endoscope 11 is in a relative positional relationship determined with respect to the imaging direction of the endoscope 11, and three-dimensional coordinates can be obtained from the image captured by the imaging device 20. Since it may be a thing, it does not necessarily need to be a spherical thing like the marker sphere 12 of this embodiment. Further, if the shape of the endoscope 11 itself can easily obtain a three-dimensional coordinate, the shape of the endoscope 11 itself becomes a marker, and therefore the marker sphere 12 is not necessarily provided.

  The imaging device 20 images the surface of the patient 60 and the marker sphere 12 when the endoscope 11 is inserted into the patient 60. As shown in FIG. 1, when the endoscope 11 is inserted from the nostril of the patient 60 and the head of the patient 60 is imaged by the endoscope 11, the face of the patient 60 and the marker ball 12 are The imaging device 20 is provided at a position where imaging can be performed. Specifically, for example, a CCD camera is used as the imaging device 20. The imaging device 20 is connected to the PC 40 and transmits information of the captured image to the PC 40.

  The image picked up by the image pickup device 20 is used to calculate the three-dimensional coordinates (three-dimensional position information) of the picked-up image. Therefore, it is necessary to provide the imaging apparatus 20 with a configuration necessary for this purpose. As a method for calculating a three-dimensional coordinate of an image captured from an image, there is a method using an optical method, and for example, a method described in Japanese Patent Laid-Open No. 2003-254732 can be used. When this method is used, a device for projecting a lattice pattern similar to white light similar to natural sunlight emitted from the xenon light is further provided in a range imaged by the imaging device 20 necessary for that purpose.

  According to the method described in Japanese Patent Laid-Open No. 2003-254732, imaging can be performed from a distance of 90 ± 10 cm in a measurement time of 1 second. The resolution is 0.1 to 0.6 mm. That is, a high-resolution color image having three-dimensional position information can be acquired in one second. Further, it is white light with an illuminance of about 28% during a cloudy daytime (outdoors), and three-dimensional position information can be acquired safely without using a laser or the like. FIG. 2 shows an example in which the face of the patient 60 is imaged by this method to obtain a high-resolution color image that has been processed and has three-dimensional position information.

  The CT apparatus 30 acquires first information indicating a surface constituting the inside of the patient 60 into which the endoscope 11 is inserted and a first three-dimensional shape of the surface of the patient 60. The surface constituting the patient 60 into which the endoscope 11 is inserted is, for example, the surface constituting the sinus when the endoscope 11 is inserted into the sinus cavity of the patient 60. In addition, the surface of the patient 60 is the face of the patient 60 in the above case. Here, the first information indicating the three-dimensional shape acquired by the CT apparatus 30 is configured by, for example, holding information indicating the three-dimensional shape for each coordinate with respect to a predetermined coordinate axis set in the CT apparatus 30 in advance. Has been. This coordinate axis is the first coordinate axis. That is, the first information is information on the first coordinate axis. In addition, the 1st information which shows a three-dimensional shape is comprised by the three-dimensional arrangement | sequence, for example. In this case, each element of the three-dimensional array corresponds to the coordinates of each point constituting the three-dimensional shape indicated by the first information.

  The CT apparatus 30 scans an object using radiation or the like, and creates an image (CT image) of the internal structure processed by using a computer by cutting it at equal intervals (for example, 1 mm). It is configured as information indicating a three-dimensional shape, and an existing CT apparatus can be used. An example of a CT image is shown in FIG. The CT apparatus 30 is connected to the PC 40 and transmits information indicating the acquired three-dimensional shape of the patient 60 to the PC 40. Note that the CT apparatus 30 does not need to be installed at the same location as the imaging apparatus 20, and usually, imaging by the imaging apparatus 20 and acquisition of information indicating a three-dimensional shape by the CT apparatus 30 are performed separately. For example, a method described in Japanese Patent Laid-Open No. 2005-278992 can be used to construct information indicating a three-dimensional shape from a CT image.

  Note that the surgery support information display device 1 only needs to acquire information indicating a three-dimensional shape including the inside of the patient 60. Accordingly, an MRI apparatus or the like may be used instead of the CT apparatus 30.

  The PC 40 is an apparatus that receives information on an image captured by the imaging apparatus 20 and first information indicating the three-dimensional shape of the patient 60 acquired by the CT apparatus 30 and performs information processing on the information. is there. Specifically, the PC 40 is configured by hardware such as a CPU (Central Processing Unit) and a memory, and the following functions of the PC 40 are realized by operating these information processing apparatuses. As shown in FIG. 1, the PC 40 includes, as functional components, a patient shape acquisition unit 41, a captured image acquisition unit 42, a surface shape calculation unit 43, a coordinate axis matching unit 44, and an endoscope optical axis calculation. Unit 45, intersection calculation unit 46, and output unit 47.

  The patient shape acquisition unit 41 is means for receiving first information indicating the three-dimensional shape of the patient 60 transmitted from the CT apparatus 30. The patient shape acquisition unit 41 outputs the received first information to the coordinate axis matching unit 44, the intersection calculation unit 46, and the like as necessary. Note that the surgery support information display device 1 does not necessarily include the CT device 30 itself, and the patient 60 (taken by a CT device not included in the surgery support information display device 1) by the patient shape acquisition unit 41. It is only necessary to receive the first information indicating the three-dimensional shape.

  The captured image acquisition unit 42 is means for receiving information of an image captured and transmitted by the imaging device 20. That is, the captured image acquisition unit 42 is a unit that acquires an image of the surface of the patient captured by the imaging device 20 when the endoscope 11 is inserted into the patient 60. The captured image acquisition unit 42 outputs the received image to the surface shape calculation unit 43, the endoscope optical axis calculation unit 45, and the like.

  The surface shape calculation unit 43 is means for calculating second information indicating the second three-dimensional shape of the surface of the patient 60 from the image of the surface of the patient 60 captured by the imaging device 20. The surface of the patient 60 is the face of the patient 60 in this embodiment. As a method for calculating a three-dimensional shape from an image, for example, the method described in Japanese Patent Laid-Open No. 2003-254732 described above can be used. The second information is calculated, for example, by configuring information indicating a three-dimensional shape for each coordinate with respect to a predetermined coordinate axis set in the surface shape calculation unit 43 in advance. This coordinate axis is different from the first coordinate axis described above and is a second coordinate axis. That is, the second information is information on the second coordinate axis. In addition, the 2nd information which shows a three-dimensional shape is comprised by the three-dimensional arrangement | sequence similarly to 1st information, for example. The surface shape calculation unit 43 outputs the calculated second information to the coordinate axis matching unit 44.

  The coordinate axis matching unit 44 matches (positions) the first information acquired by the patient shape acquisition unit 41 and the second information calculated by the surface shape calculation unit 43, so that the first coordinate axis and Means for matching the second coordinate axis. That is, the coordinate axis matching unit 44 can process the first information acquired by the CT apparatus 30 and the second information calculated from the image captured by the imaging apparatus 20 on the same coordinate axis. Means. For this purpose, the coordinate axis matching unit 44 includes an information receiving unit 44a, a reference point setting unit 44b, a corresponding point calculating unit 44c, and an alignment unit 44d, as shown in FIG.

  It should be noted that the expressions and rules necessary for the functions of the information receiving unit 44a, the reference point setting unit 44b, the corresponding point calculating unit 44c, and the positioning unit 44d shown below are the information receiving unit 44a and the reference point setting unit 44b. Are stored in each of the corresponding point calculation unit 44c and the alignment unit 44d, and are read out as necessary when the function is executed.

  The information reception unit 44 a is a reception unit that receives input of the first information acquired by the patient shape acquisition unit 41 and the second information calculated by the surface shape calculation unit 43. The information receiving unit 44a outputs the received first information to the reference point setting unit 44b and outputs the received second information to the corresponding point calculating unit 44c.

  The reference point setting unit 44b is a setting unit that provides a plurality of reference points on the first three-dimensional shape indicated by the first information input from the information receiving unit 44a. Specifically, the reference point setting unit 44b adds reference point information indicating the set reference point to the first information in order to be able to determine which point is set as the reference point. For example, when the first information is configured in a three-dimensional array, the reference point setting unit 44b sets each of the elements set as the reference points among a plurality of elements (point coordinates) configuring the three-dimensional array. Is associated with reference point information (for example, a flag). Note that the number (density) of reference points to be set is not limited. For example, the reference point setting unit 44b may provide a reference point for each predetermined distance, or may set all points constituting the first three-dimensional shape as reference points. The reference point setting unit 44b outputs the first information to which the reference point information is added to the corresponding point calculation unit 44c.

  The corresponding point calculation unit 44c indicates, for each of the plurality of reference points set by the reference point setting unit 44b, the corresponding point at the shortest distance from the reference point by the second information input from the information receiving unit 44a. Calculating means for calculating from a limited range of the second three-dimensional shape. Note that formulas, rules, data, and the like necessary for calculating the limited range are stored in the corresponding point calculation unit 44c, and are read out when the limited range is calculated.

  When the corresponding point is calculated, the corresponding point calculating unit 44c adds corresponding point information indicating the calculated corresponding point to the second information in order to be able to determine which point is calculated as the corresponding point. For example, when the second information is configured in a three-dimensional array, the corresponding point calculation unit 44c selects the element calculated as the corresponding point from among a plurality of elements (point coordinates) configuring the three-dimensional array. For each, the element is associated with corresponding point information (for example, a flag). Then, the corresponding point calculation unit 44c outputs the first information input from the reference point setting unit 44b and the second information added with the corresponding point information to the alignment unit 44d.

  The processing of the corresponding point calculation unit 44c will be described with reference to FIG. In FIG. 5, for the sake of simplicity, the matching figures are assumed to be two-dimensional curves p and m. In the example shown in FIG. 5, the curve m is fixed and the curve p is matched with the curve m. The matching in the two-dimensional space shown in FIG. 5 can be easily extended to the matching in the three-dimensional space as in this embodiment. For example, by replacing the curve p with the first three-dimensional shape indicated by the first information and replacing the curve m with the second three-dimensional shape indicated by the second information, Can be matched with the second information.

  The corresponding point calculation unit 44c first calculates the shortest distance field around the curve m. Here, the shortest distance field is a neighborhood space composed of points existing at a shortest distance (the length of the shortest path) from a certain figure. Next, the corresponding point calculation unit 44c selects an arbitrary reference point (first reference point) Pa on the curve p, and searches for the corresponding point Ma on the curve m that is the shortest distance from the reference point Pa. (calculate. The search for the corresponding point Ma is performed by calculating the distance from the reference point Pa for all points on the curve m and comparing the calculated distances. The corresponding point calculation unit 44c can calculate in advance the position of the point on the curve m for calculating the distance from the reference point Pa by using a predetermined relational expression, the arrangement of the points constituting the curve m, or the like. For example, the corresponding point calculation unit 44c can acquire the position of a point existing on the first three-dimensional shape or the second three-dimensional shape based on the first information or the second information.

  The positional relationship between the reference point and the corresponding point is defined on the assumption that the two three-dimensional shapes exist on a common coordinate axis. For example, the calculation of the distance between the reference point Pa and a point on the curve m assumes that the curve p and the curve m exist on a common coordinate axis. That is, the shortest distance is the distance between the reference point and the corresponding point when these two curves exist on a common coordinate axis.

  Next, the corresponding point calculation unit 44c selects another reference point (second reference point) Pb on the curve p, and searches for the corresponding point Mb on the curve m that is the shortest distance from the reference point Pb. (calculate. At this time, the corresponding point calculation unit 44c calculates the distance between a plurality of points existing in a limited range on the curve m and the reference point Pa.

  For this purpose, the corresponding point calculation unit 44c includes the coordinate values of the reference points Pa and Pb, the coordinate value of the corresponding point Ma, the distance l between the reference point Pa and the corresponding point Ma, the reference point Pb, and the corresponding point Ma. , The coordinate value of the reference point Pb, and the distance l ′ determined by the shortest distance field around the curve m. These values are known or can be calculated from known values. Then, the corresponding point calculation unit 44c determines a range for searching for the corresponding point Mb based on the acquired values.

In this case, the following triangular inequality is established from the positional relationship among the reference point Pb, the known corresponding point Ma, and the unknown corresponding point Mb. Incidentally, d _m is the distance between the known corresponding points Ma and unknown correspondence point Mb.
d _m ≦ l ′ + d

Therefore, the range of the curve m to be searched for the unknown corresponding point Mb is limited to the circle C whose center is the known corresponding point Ma and whose radius is expressed by the following equation.
r = l ′ + d (1)
The radius r may be calculated using the following formula. Here, d _p is a distance between known reference points Pa and Pb.
r = l + l ′ + d _p (2)

  In addition, since the figure shown by the 1st and 2nd information is a three-dimensional shape, in this embodiment, the corresponding point calculation part 44c respond | corresponds within the sphere which has the predetermined radius calculated according to the said theory. Search for a point.

  That is, the corresponding point calculation unit 44c only determines the distance between the corresponding point Ma and the reference point Pb in the curve m only for the points within the radius r defined by the above formula (1) or (2) from the corresponding point Ma. Is calculated. Then, the corresponding point calculation unit 44c determines the point having the shortest distance from the reference point Pb as the corresponding point Mb. The corresponding point calculation unit 44c can calculate and acquire the position of the point on the curve m in advance. Therefore, the corresponding point calculation unit 44c determines the reference point based on the position of the point on the curve m acquired in advance, the position of the corresponding point Ma, and the radius r defined by the above formula (1) or (2). A point on the curve m whose distance from Pb should be calculated can be acquired.

  Thereafter, the corresponding point calculation unit 44c searches (calculates) a corresponding point for one or more other reference points on the curve p. The method for calculating the corresponding point is the same as the method for calculating the corresponding point Mb. That is, when calculating the second and subsequent corresponding points, the corresponding point calculation unit 44c calculates the corresponding points from a limited range on the curve m.

  The alignment unit 44d converts the first information based on the positional relationship between the plurality of reference points set by the reference point setting unit 44b and the plurality of corresponding points calculated by the corresponding point calculation unit 44c. Alignment means for aligning the first three-dimensional shape indicated by the first information and the second three-dimensional shape indicated by the second information.

  Specifically, the alignment unit 44d calculates an average value of the distance between the reference point and the corresponding point corresponding to the reference point. The reference point can be extracted based on the reference point information added to the data constituting the first information, and the corresponding point is the corresponding point information added to the data constituting the second information. Extraction based on

Subsequently, the alignment unit 44d calculates the rotation matrix R and the translation matrix T based on the positional relationship between the plurality of calculated reference points and corresponding points, and performs rigid body transformation using these two matrices. In the case of the example of FIG. 5, this rigid body transformation is represented by the following equation, where p _i-1 is the position of the curve p before the rigid transformation and p _i is the position of the curve p after the rigid transformation. By this rigid transformation, the curve p rotates or translates, and the curve p and the curve m are aligned.
p _i = R * p _i-1 + T (3)

  The alignment unit 44d converts the first information based on the above equation (3). As a result, the first three-dimensional shape rotates or translates, and the first three-dimensional shape and the second three-dimensional shape are aligned. Subsequently, the alignment unit 44d determines whether or not the difference between the average value calculated this time and the average value calculated last time is less than a predetermined threshold value. Note that the fact that the difference is less than the predetermined threshold means that the first information and the second information are matched (positioned), and the first coordinate axis and the second coordinate axis match. . This means that a function for converting one of the first coordinate axis and the second coordinate axis to the other coordinate axis is calculated. In this case, the alignment unit 44d sends a function for converting the coordinate axis calculated as a result of matching the coordinate axes to the endoscope optical axis calculation unit 45, the intersection calculation unit 46, the output unit 47, and the like as necessary. Output.

  On the other hand, when the difference between the average value calculated this time and the average value calculated last time is equal to or greater than a predetermined threshold, the alignment unit 44d corresponds to the first information converted using the above equation (3). It outputs to the point calculation part 44c. As a result, the corresponding point calculation unit 44c calculates the corresponding points based on the converted first information and the acquired second information. That is, the processing from the calculation of corresponding points to the rigid body transformation is repeatedly executed.

  After the process of matching the coordinate axes by the process of the coordinate axis matching unit 44 as described above, the above functions and the like in the endoscope optical axis calculation unit 45, the intersection calculation unit 46, the output unit 47, and the like change the three-dimensional shape. By applying to the information to be shown, the information indicating the three-dimensional shape by the CT device 30 and the information indicating the three-dimensional shape calculated from the image captured by the imaging device 20 are processed on the same coordinate axis.

  The endoscope optical axis calculation unit 45 calculates the three-dimensional coordinates of the marker sphere 12 on the coordinate axis matched by the coordinate axis matching unit 44 from the image of the marker sphere 12 imaged by the imaging device 20, and the marker sphere 12 12 is a means for calculating a half line indicating the optical axis that is the imaging direction A of the endoscope 11 from the positional relationship between the imaging direction 12 of the endoscope 11 and the imaging direction A of the endoscope 11. In addition, the position which becomes the starting point of a half line is included in the half line which shows the optical axis which is the imaging direction A said here. That is, the half line indicating the optical axis that is the imaging direction A of the endoscope 11 indicates from which point the imaging is performed. The endoscope optical axis calculation unit 45 holds information indicating the positional relationship between the marker sphere 12 and the imaging direction A of the endoscope 11 in advance. As the information indicating the positional relationship, specifically, for example, a predetermined position of the base of the endoscope 11 so as to pass through the optical axis of the endoscope 11 (a half line indicating the optical axis that is the imaging direction A). Information indicating the position of the tip and the positional relationship between the two points and the marker sphere 12 is used.

  The calculation of the half line indicating the optical axis that is the imaging direction A of the endoscope 11 is specifically performed as follows. First, the endoscope optical axis calculation unit 45 calculates the three-dimensional shape and the three-dimensional coordinates from the marker sphere 12 imaged by the imaging device 20 in the same manner as the method of calculating the three-dimensional shape in the surface shape calculation unit 43. calculate. When a plurality of marker spheres 12 are provided, the endoscope optical axis calculation unit 45 identifies each marker sphere 12 based on the size of the marker sphere 12, for example.

  Subsequently, the endoscope optical axis calculation unit 45 includes the calculated three-dimensional coordinates of the marker sphere 12 on the coordinate axis and information indicating the positional relationship between the marker sphere 12 and the imaging direction A of the endoscope 11. Then, a half line indicating the optical axis that is the imaging direction A on the coordinate axis is calculated. The endoscope optical axis calculation unit 45 outputs half-line information indicating the optical axis that is the calculated imaging direction to the intersection calculation unit 46.

  The intersection calculation unit 46 relates to the half line indicating the optical axis that is the imaging direction A of the endoscope 11 calculated by the endoscope optical axis calculation unit 45 and the first information acquired by the patient shape acquisition unit 41. It is a means for calculating an intersection with a surface constituting the inside of the patient 60. This intersection point indicates a point (center point) where the endoscope 11 is imaging in the first information indicating the three-dimensional shape by the CT apparatus 30. Specifically, the intersection calculation unit 46 converts the surfaces constituting the inside of the patient 60 into polygon data, and shows each surface constituting the polygon data and the optical axis that is the imaging direction A of the endoscope 11. The intersection point with is calculated. The calculation of the intersection will be described in detail later. The intersection calculation unit 46 outputs the calculated intersection information to the output unit 47.

  The output unit 47 superimposes the information indicating the intersection calculated by the intersection calculation unit 46 on the CT image which is information indicating the surface constituting the inside of the patient 60 acquired by the patient shape acquisition unit 41, and monitors the monitor 50. It is a means to output to. The output unit 47 may output the endoscopic image captured by the endoscope 11 and input to the PC 40 to the monitor 50 together.

  The monitor 50 displays information input from the PC 40. By referring to the monitor 50, the surgeon can know which part of the patient 60 is imaging the endoscope 11. The above is the configuration of the surgery support information display device 1.

  Subsequently, the operation (surgery support information display method) of the surgery support information display device 1 will be described with reference to the flowchart of FIG. This operation is, for example, an operation when performing treatment or the like by inserting the endoscope 11 during surgery on the patient 60. In this description, the processing before the operation and the processing at the time of the operation will be described separately.

  First, before the operation, CT scan imaging is performed on the patient 60 using the CT apparatus 30 (step S01). This CT scan imaging is performed on the part of the patient 60 into which the endoscope 11 is inserted. Thereby, the 1st information which shows the 1st three-dimensional shape of the face which is the surface of patient 60, and the field which constitutes the inside of patient 60 into which endoscope 11 is inserted is acquired. The first information acquired by CT scan imaging performed by the CT apparatus 30 is transmitted to the PC 40. In the PC 40, the information is acquired by the patient shape acquisition unit 41 and stored in the PC 40 (step S02). The above is processing before surgery, for example, performed on the day before surgery.

  Subsequently, processing during surgery will be described. First, the patient 60 enters the operating room, and as shown in FIG. 1, the endoscope 11 is placed on the back on the operating table 70 so that the endoscope 11 can be inserted through the nostril. After the patient 60 is placed and before the endoscope 11 is inserted, the placed patient 60 is imaged by the imaging device 20 (step S03). The captured image is transmitted from the imaging device 20 to the PC 40 and received by the captured image acquisition unit 42 in the PC 40. The received image is output from the captured image acquisition unit 42 to the surface shape calculation unit 43.

  The surface shape calculation unit 43 calculates second information indicating the second three-dimensional shape of the face that is the surface of the patient 60 from the image (step S04). The calculated second information is output from the surface shape calculation unit 43 to the coordinate axis matching unit 44. Information indicating the three-dimensional shape of the patient 60 by the CT apparatus 30 stored in the PC 40 is output from the patient shape obtaining unit 41 to the coordinate axis matching unit 44 at the same timing.

  The coordinate axes related to the first information by the CT apparatus 30 and the second information calculated from the image by the imaging apparatus 20 do not match. When these pieces of information are displayed together, as shown in FIG. 7A, the information 81 by the CT apparatus 30 and the information 82 by the image of the imaging apparatus 20 are not aligned.

  Here, as shown in FIG. 7B, the coordinate shape matching unit 44 matches the shape of the face in the two pieces of information 81 and 82 to match the coordinate axes related to the two pieces of information (step S05). Note that the matching parts are set in advance, such as the whole face or a characteristic part such as the nose or cheek of the face.

  The process of matching the coordinate axes will be described in detail with reference to the flowchart of FIG. First, the input of the 1st information acquired by the patient shape acquisition part 41 and the 2nd information calculated by the surface shape calculation part 43 is received by the information reception part 44a (step S51, reception step). Subsequently, the reference point setting unit 44b sets a plurality of reference points on the first three-dimensional shape indicated by the first information (step S52, setting step). Subsequently, for each of the set reference points, the corresponding point corresponding to the reference point is extracted from the second three-dimensional shape indicated by the second information by the corresponding point calculation unit 44c (step S53). , Calculation step). This extraction is repeatedly executed until all corresponding points corresponding to a predetermined number are extracted (step S54; NO, calculation step). At this time, in the extraction of the second and subsequent corresponding points, the corresponding points are extracted from a limited range of the second three-dimensional shape.

  When all corresponding points are extracted (step S54; YES, calculation step), the alignment unit 44d performs the following series of processes. Specifically, first, the average value of the distance between the reference point and the corresponding point is calculated (step S55). Subsequently, a rotation matrix and a translation matrix are calculated based on the positional relationship between the plurality of calculated reference points and corresponding points (step S56, alignment step). Subsequently, the first information is rigidly transformed based on the rotation matrix and the translation matrix (step S57, alignment step). By this conversion, the first three-dimensional shape and the second three-dimensional shape are aligned. Subsequently, it is determined whether or not the difference between the average value calculated this time and the average calculated last time is less than a threshold value (step S57).

  Here, if the difference between the average values is equal to or larger than the threshold value (step S58; NO), the processing after step S52 is repeatedly executed. On the other hand, when the difference is less than the threshold value (step S58; YES), the process ends. As a result, the first coordinate axis and the second coordinate axis are matched.

  Information on the coordinate axes that have been matched is output from the coordinate axis matching unit 44 (positioning unit 44d) to the endoscope optical axis calculation unit 45, the intersection calculation unit 46, and the output unit 47, respectively, and conversion of the coordinate axes is performed. Thereafter, information processing on the three-dimensional shape is performed with reference to the coincident coordinate axes. The above is the process until the start of surgery.

  Subsequently, an operation is started, and the endoscope 11 is inserted into the patient 60 by the operator. At this time, the head of the patient 60 is prevented from moving after the processing of S03 to S05 is performed. This is to prevent the coordinate axes from shifting. When the endoscope 11 is inserted into the patient 60, the patient 60 and the marker sphere 12 are imaged by the imaging device 20 (step S06). As shown in FIG. 9A, the picked-up image includes a marker sphere 12 (image thereof). The captured image is transmitted from the imaging device 20 to the PC 40 and received by the captured image acquisition unit 42 in the PC 40. The received image is output from the captured image acquisition unit 42 to the endoscope optical axis calculation unit 45.

  Subsequently, as shown in FIG. 9B, the three-dimensional coordinates of the marker sphere 12 are calculated from the image by the endoscope optical axis calculation unit 45 (step S07). Subsequently, information indicating the positional relationship between the marker sphere 12 and the imaging direction A of the endoscope 11 held in advance is calculated from the three-dimensional coordinates of the marker sphere 12 calculated by the endoscope optical axis calculator 45. Based on this, a half line indicating the optical axis that is the imaging direction A of the endoscope 11 on the matched coordinate axis is calculated (step S08).

  The calculated half-line information is output from the endoscope optical axis calculation unit 45 to the intersection calculation unit 46. At the same timing, the first information by the CT apparatus 30 stored in the PC 40 is output from the patient shape acquisition unit 41 to the intersection calculation unit 46. Subsequently, the intersection calculation unit 46 calculates the intersection between the half line indicating the optical axis that is the imaging direction A of the endoscope 11 and the plane that forms the inside of the patient 60 (step S09).

  The intersection is calculated as follows. First, the intersection calculation unit 46 converts the first information from the CT apparatus 30 into polygon data. By this conversion, the surface constituting the patient 60 (inside) is constituted by, for example, a large number of triangles. Next, the intersection of each triangle and the half line indicating the optical axis that is the imaging direction A of the endoscope 11 is calculated.

The calculation of this intersection will be described with reference to FIG. A triangle can be expressed by the following expression, where PT is a reference point of a triangle constituting a polygon, two vectors of the triangle are vecA and vecB, and two parameters are α and β.
P1 = PT + αvecA + βvecB
Further, assuming that the reference point of the imaging direction A of the endoscope 11 (for example, the point at the tip of the endoscope 11) is PL, the vector indicating the direction of the half line is vecC, and the parameter is γ, The imaging direction A of the mirror 11 can be expressed by the following equation.
P2 = PL + γvecC

Here, when the two intersect, P1 = P2. The condition that the point where P1 = P2 exists inside the triangle is when the parameter satisfies the following condition.
Condition 1: 0 <α, 0 <β
Condition 2: 0 <α + β <1
Condition 3: γ> 0
Points that satisfy these conditions are derived for the triangles that make up all the polygon data, and the distance between all these points and the point at the tip of the endoscope 11 is calculated. The intersection at which this distance is the smallest is the intersection of the half line indicating the optical axis that is the imaging direction A of the endoscope 11 and the surface that forms the inside of the patient 60.

  Information on the coordinates of the calculated intersection is output from the intersection calculation unit 46 to the output unit 47. At this timing, a CT image that is information indicating a surface constituting the inside of the patient 60 is output from the patient shape acquisition unit 41 to the output unit 47. The information on the intersection is input to the monitor 50 by the output unit 47 by superimposing the CT image, which is information indicating the surface constituting the inside of the patient 60, at a location corresponding to the coordinates of the intersection. The input image is displayed on the monitor 50 (step S10). Specifically, for example, as shown in FIG. 11, the intersection point is displayed as a cross display 90 so that the point where the part imaged by the endoscope 11 is located can be seen. By referring to the displayed image, the surgeon can know which part of the patient 60 is imaging the endoscope 11.

  In addition, the image captured by the endoscope 11 is also received by the PC 40 and output from the output unit 47 to the monitor 50, so that the position where the image is captured by the endoscope 11 is located at any point. It is desirable to be displayed together with the display of whether or not For example, as in a screen example 91 shown in FIG. 12, an image 92 captured by the endoscope 11 and a CT image 93a that indicates where the position captured by the endoscope 11 is located. ~ 93c should be shown on one screen. In addition, it is desirable to superimpose information (for example, a cross (+) mark) 94 indicating a location corresponding to the intersection on the image of the endoscope 11 at that time. The position of the information indicating the location corresponding to the intersection in the image of the endoscope 11 is stored in advance in the output unit 47 according to the imaging direction of the endoscope 11 such as the center of the image, for example. In order to make it easier for the surgeon to understand the corresponding location, the CT images 93a to 93c preferably display a plurality of different cross sections (preferably cross sections that are orthogonal to each other). In FIG. 12, the CT image 93a has a cross section, the CT image 93b has a coronal section, and the CT image 93c has a sagittal section. Further, in the CT images 93 a to 93 c, in addition to the intersection information (indicated by a square 95 of a different color from the image), the direction in which the endoscope 11 is inserted (light that is the imaging direction of the endoscope 11). It is desirable to show axis 96.

  The processes from S06 to S10 are repeatedly performed at equal intervals such as 1 second. In addition, although the process of S03-S05 and the process of S06-S10 in PC40 are different, for example, if the process of matching the coordinate axes of S05 is performed, the process may be automatically shifted to the process after S06. Good. Further, the process may be switched by an operation of an operator or the like.

  Further, even when the head of the patient 60 is moved after the initial alignment by the processing of S03 to S05, the processing of S03 to S05 may be performed again to perform alignment again. The re-alignment may be performed, for example, by an operation of an operator or the like, or the image at the time of alignment is compared with the subsequent image to detect that the head has moved, and It may be performed as a trigger.

  As described above, in the surgery support information display device 1 (surgery support information display method) according to the present embodiment, the corresponding points are calculated when the first three-dimensional shape and the second three-dimensional shape are aligned. Since the range to be used is limited, the time required for calculating the corresponding point is reduced as compared with the case where the corresponding point is calculated from the entire second three-dimensional shape. At this time, the reference point can be arbitrarily set. In addition, no artificial operation is involved in calculating the corresponding points. As a result, it is possible to shorten the calculation time required for the alignment while maintaining the accuracy and efficiency in aligning the first three-dimensional shape and the second three-dimensional shape.

Such an effect becomes more remarkable as the number of elements (points) constituting the shape increases. For example, the first three-dimensional shape and the second three-dimensional shape each composed of hundreds of thousands of points are aligned by setting all the points on the first three-dimensional shape as reference points. In this case, the time can be reduced by about 10 ⁻⁴ to 10 ^{−5 as} compared with the conventional method (a method for calculating corresponding points from the entire range of the second three-dimensional shape).

For example, as shown in FIG. 5, the corresponding point calculation unit 44c uses the calculated corresponding point Ma as the center, the position of the corresponding point Ma, the position of the reference point Pb, and the shortest distance field of the curve m. A circle C having a radius r of a value (l ′ + d) calculated based on the value is defined as a limited range. In addition, as shown in FIG. 5, for example, the coordinate axis matching unit 44 is centered on the calculated corresponding point Ma, the position of the corresponding point Ma, the position of the reference point Pa, the position of the reference point Pb, and a curve A circle C having a radius r of a value (l + l ′ + d _p ) calculated based on the shortest distance field of m may be a limited range. Thereby, since the limited range (circle C) for calculating the unknown corresponding point Mb is set, the corresponding point Mb can be calculated earlier. The same applies to the calculation of other corresponding points.

  Subsequently, a description will be given of a surgery support information display program for causing a computer to execute the processing for displaying the series of surgery support information described above. As shown in FIG. 13, the surgery support information display program 82 is stored in a program storage area 80a formed in a recording medium 80 provided in the computer.

  The surgery support information display program 81 includes a main module 810 that comprehensively controls a display process of surgery support information, a patient shape acquisition module 811, a captured image acquisition module 812, a surface shape calculation module 813, and a coordinate axis matching module 814. An endoscope optical axis calculation module 815, an intersection calculation module 816, and an output module 817. The coordinate axis matching module 814 includes an information receiving module 814a, a reference point setting module 814b, a corresponding point calculating module 814c, and an alignment module 814d. Patient shape acquisition module 811, captured image acquisition module 812, surface shape calculation module 813, coordinate axis matching module 814, reference point setting module 814b, corresponding point calculation module 814c, alignment module 814d, endoscope optical axis calculation module 815, intersection Functions realized by executing the calculation module 816 and the output module 817 include the above-described patient shape acquisition unit 41, captured image acquisition unit 42, surface shape calculation unit 43, coordinate axis matching unit 44, information reception unit 44a of the PC 40, Functions of the reference point setting unit 44b, the corresponding point calculation unit 44c, the alignment unit 44d, the endoscope optical axis calculation unit 45, the intersection calculation unit 46, and the output unit 47 are the same.

  The surgery support information display program 81 may be partly or wholly transmitted via a transmission medium such as a communication line, received by another device, and recorded (including installation).

  The present invention has been described in detail based on the embodiments. However, the present invention is not limited to the above embodiment. The present invention can be modified in various ways as described below without departing from the scope of the invention.

  In the above embodiment, the shape information processing method, the shape information processing apparatus, and the shape information processing program according to the present invention are applied to the surgery support information display device, the surgery support information display method, and the surgery support information display program. The processing method, shape information processing apparatus, and shape information processing program are applicable to various research and development fields and industrial fields. Application examples include, for example, CAD, component surface quality inspection, production of a prototype model, and autonomous movement control of the robot, but of course not limited thereto.

It is a figure which shows the structure of the surgery assistance information display apparatus which concerns on embodiment of this invention. It is a figure which shows the image of a patient's face which was imaged with the imaging device, was information-processed, and had 3D position information. It is a figure which shows CT image acquired by CT apparatus. It is a figure which shows the structure of the coordinate-axis matching part shown in FIG. It is a figure which shows the concept of a process of the coordinate axis matching part shown in FIG. It is a flowchart which shows the process in the surgery assistance information display apparatus which concerns on embodiment of this invention. It is a figure which shows the matching process of the information which shows the patient's three-dimensional shape by CT apparatus, and the information which shows the patient's three-dimensional shape calculated from the image by an imaging device. It is a flowchart which shows the coordinate axis matching process shown in FIG. It is the figure which showed the process which calculates the three-dimensional coordinate of a marker ball from the image by an imaging device. It is the figure which showed the intersection of the triangle which comprises a patient's surface, and the half line which shows the optical axis which is the imaging direction of an endoscope. It is a figure which shows the CT image in which the point imaged with the endoscope was shown. It is a figure which shows the CT image in which the point imaged with the endoscope was shown, and the image imaged with the endoscope. It is a figure which shows the structure of the surgery assistance information display program which concerns on this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Surgery support information display apparatus, 11 ... Endoscope, 12 ... Marking ball, 20 ... Imaging apparatus, 30 ... CT apparatus, 40 ... PC, 41 ... Patient shape acquisition part, 42 ... Captured image acquisition part, 43 ... Surface Shape calculation unit 44 ... Coordinate axis matching unit 44a ... Information receiving unit 44b ... Reference point setting unit 44c ... Corresponding point calculation unit 44d ... Positioning unit 45 ... Endoscope optical axis calculation unit 46 ... Intersection calculation , 47 ... output unit, 50 ... monitor, 60 ... patient, 70 ... operating table, 80 ... recording medium, 80a ... program storage area, 81 ... surgery support information display program, 810 ... main module, 811 ... patient shape acquisition module 812, a captured image acquisition module, 813, a surface shape calculation module, 814, a coordinate axis matching module, 814a, an information reception module, 814b, a reference point setting module, 814c. Corresponding point calculation module, 814d ... positioning module, 815 ... endoscope optical axis calculating module, 816 ... intersection calculation module, 817 ... output module.

Claims (6)

An accepting step for accepting input of first information indicating a first shape and second information indicating a second shape;
A plurality of reference points are placed on one of the first shape indicated by the first information received in the reception step and the second shape indicated by the second information received in the reception step. A setting step to be provided;
For each of the plurality of reference points set in the setting step, a calculation step for calculating a corresponding point at the shortest distance from the reference point from a limited range of the other shape;
By converting the first information or the second information based on the positional relationship between the plurality of reference points set in the setting step and the plurality of corresponding points calculated in the calculation step, An alignment step of aligning the first shape and the second shape;
Shape information processing method including The calculation step is based on the calculated position of the corresponding point corresponding to the first reference point and the position of the second reference point different from the first reference point. Calculating a range centered on the corresponding point, and setting the range as the limited range for calculating the corresponding point at the shortest distance from the second reference point;
The shape information processing method according to claim 1. The limited range is a distance defined on the basis of the position of the second reference point and the shortest distance field indicating a predetermined neighboring space of the other three-dimensional shape with the calculated corresponding point as the center. And a range having a radius of a value obtained by adding the distance between the second reference point and the calculated corresponding point.
The shape information processing method according to claim 2. The limited range is centered on the calculated corresponding point, the distance between the first reference point and the calculated corresponding point, the position of the second reference point, and the other three-dimensional shape. A distance defined based on the shortest distance field indicating a predetermined neighborhood space and a value obtained by adding the distance between the first reference point and the second reference point as a radius.
The shape information processing method according to claim 2. Receiving means for receiving input of first information indicating a first shape and second information indicating a second shape;
A plurality of reference points are placed on one of the first shape indicated by the first information received by the receiving means and the second shape indicated by the second information received by the receiving means. Setting means to be provided;
For each of a plurality of reference points set by the setting means, a calculation means for calculating a corresponding point at the shortest distance from the reference point from a limited range of the other shape;
By converting the first information or the second information based on the positional relationship between a plurality of reference points set by the setting means and a plurality of corresponding points calculated by the calculation means, Alignment means for aligning the first shape and the second shape;
A shape information processing apparatus comprising: A reception function for receiving input of first information indicating a first shape and second information indicating a second shape;
A plurality of reference points are placed on one of the first shape indicated by the first information received by the receiving function and the second shape indicated by the second information received by the receiving function. A setting function to be provided;
For each of a plurality of reference points set by the setting function, a calculation function for calculating a corresponding point at the shortest distance from the reference point from a limited range of the other shape;
By converting the first information or the second information based on the positional relationship between a plurality of reference points set by the setting function and a plurality of corresponding points calculated by the calculation function, An alignment function for aligning the first shape and the second shape;
Shape information processing program for causing a computer to execute.