JP5987998B2 - Surgery support system, projector system, and medical image projection method - Google Patents

Description

  The present invention is a surgery support technique using a projector system.

  In recent years, technology related to project mapping has attracted attention. By projecting unreal images on the walls of Tokyo Station Station and Osaka Castle, you can create a fantastic space. In addition, it is possible to change the cityscape by projecting artistic images on the wall of the building. Events that use project mapping can delight the eyes of the audience and can be expected to attract a large amount of customers. As a result, a great economic effect can be expected.

  Moreover, since it can project an unreal image and entertain the eyes of the audience, it is also applied to the field of entertainment (Patent Document 1).

JP, 2014-010362, A

  On the other hand, surgery support technology using a projector is also being researched in the medical field. Project medical images taken with CT and MRI onto the body surface and organs. Projection makes it easy to grasp the internal structure intuitively and can reduce surgical errors.

  At this time, alignment (registration) of the projection image and the projection target is important. Various registration methods have been studied, and there is no established technology.

  For example, an image of a body surface is acquired together with a medical image in advance, the body surface image is projected onto the body surface, alignment is performed, and position matching is confirmed, and then a target image (projection target image based on the medical image) Project.

  By the way, in the above-mentioned project mapping technology, a video engineer with specialized knowledge uses a special machine. On the other hand, surgeons often lack expertise in video technology. If there is a defect in the registration, not only will it not be possible to provide appropriate surgical support, but conversely, the surgical site may be confused.

  There is no tolerance for surgery. The surgeon is forced more than expected. Therefore, it is not preferable to impose any additional burden on the surgeon.

  The present invention solves the above-described problems, and an object thereof is to provide a surgical operation support system that performs projector registration in a fully automatic manner.

  The surgical operation support system of the present invention that solves the above problems includes a camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device, and the image processing device is based on the camera. Registration is performed based on a captured image of the registration tracker and registration tracker information acquired at the same time as acquiring a medical image in advance, and target information for the projector to project onto a projection target is acquired. .

  The surgical operation support system of the present invention that solves the above problems includes a camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device, and the image processing device includes: A camera coordinate system setting function unit for setting a viewed three-dimensional coordinate system, a projector coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the projector, a positional relationship between the camera and the projector, and the camera coordinates A registration tracker information conversion function unit that acquires registration tracker information of the projector coordinate system based on the registration tracker information of the system, and registration tracker information and medical image coordinates of the medical image coordinate system acquired in advance A medical image input function unit for inputting system target information; Target imaging for acquiring target information of the projector coordinate system based on the positional relationship between the registration tracker information of the medical image coordinate system and the target information of the medical image coordinate system and the registration tracker information of the projector coordinate system An image acquisition function unit; and a dimension conversion function unit that converts target information of the projector coordinate system from three-dimensional coordinates to two-dimensional coordinates.

  With the above configuration, projector registration can be performed fully automatically.

  In the above invention, preferably, the camera captures the registration tracker, and the image processing apparatus acquires registration tracker information of the camera coordinate system based on a captured image of the registration tracker by the camera. It has a registration tracker shooting function section.

  In the above invention, preferably, the registration tracker is provided with a registration marker, the camera photographs the registration marker, and the image processing device is configured to capture the camera based on a captured image of the registration marker. The registration marker information of the coordinate system is acquired, and the registration tracker information of the camera coordinate system is obtained based on the positional relationship between the registration tracker and the registration marker and the registration marker information of the camera coordinate system. A registration marker photographing function unit to be acquired is included.

  In the above invention, preferably, the projector projects a target image, which is an image relating to a target inside the projection target, onto the projection target based on the target information.

  In the above invention, preferably, the registration tracker information of the medical image coordinate system is acquired in a state where the registration tracker is installed at a predetermined position on the surface of the projection target.

  The surgical operation support system of the present invention further includes a unit moving mechanism for moving a unit including the camera and the projector.

  In the above invention, the registration tracker is a polyhedron.

  In the above invention, the image processing apparatus includes a movement detection function unit that repeatedly outputs a control command when the movement of the camera is detected.

  Thereby, a target image can be projected from multiple directions. Thereby, it can be avoided that the surgeon becomes an obstacle to projection. Furthermore, a pseudo three-dimensional image is obtained.

  A projector system of the present invention that solves the above problems includes a camera, a projector, a registration tracker installed on a surface of a projection target, and an image processing device, and the image processing device is viewed from the camera. A camera coordinate system setting function unit for setting a three-dimensional coordinate system, a projector coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the projector, a positional relationship between the camera and the projector, and the camera coordinate system A registration tracker information conversion function unit for acquiring registration tracker information of the projector coordinate system based on the registration tracker information of Pre-image input function unit for inputting target information The projector coordinate system target information is acquired based on the positional relationship between the registration tracker information of the prior image coordinate system and the target information of the prior image coordinate system and the registration tracker information of the projector coordinate system. A target captured image acquisition function unit; and a dimension conversion function unit that converts target information of the projector coordinate system from three-dimensional coordinates to two-dimensional coordinates.

  The projector system of the present invention can be applied to uses other than surgical support.

  In the medical image projection method of the present invention that solves the above-described problem, the medical image creation device creates registration tracker information simultaneously with the creation of the medical image related to the target, and is installed on the surface of the camera, the projector, and the projection target. A unit including the camera and the projector of the surgery support system including the registration tracker and the image processing apparatus is installed so as to correspond to the projection target, and the camera is a surface of the projection target. The registration tracker installed in the camera is photographed, and the registration is performed based on the registration tracker information acquired at the same time when the image processing apparatus acquires the medical track and the captured image of the registration tracker. Related to the target inside the projection target. That acquires target information is image information, the projector, based on said target information, to project the target image on a projection target.

  In the medical image projection method of the present invention that solves the above problems, the medical image creation device creates registration tracker information in the medical image coordinate system and target information in the medical image coordinate system in advance, and includes a camera, a projector, Of the surgical operation support system including a registration tracker installed on the surface of the projection target and an image processing apparatus, a unit including the camera and the projector is installed to correspond to the projection target, and the camera Image a registration tracker installed on the surface of the projection target, and the image processing device acquires registration tracker information of the camera coordinate system based on a captured image of the registration tracker by the camera, The positional relationship between the camera and the projector, and the register of the camera coordinate system Registration tracker information of the projector coordinate system based on the registration tracker information, a positional relationship between the registration tracker information of the medical image coordinate system and target information of the medical image coordinate system, and the projector coordinate system Based on the registration tracker information of the projector, the target information of the projector coordinate system is acquired, the target information of the projector coordinate system is converted from the three-dimensional coordinates to the two-dimensional coordinates, and the projector is based on the two-dimensional target information. The target image is projected onto the projection target.

  In the above invention, a unit including the camera and the projector is moved, and each time the unit is moved, photographing by the camera, image processing by the image processing device, and projection by the projector are repeated.

  The surgical operation support system of the present invention can perform projector registration fully automatically. As a result, the burden on the operator is greatly reduced.

Functional block diagram of system (first embodiment) Registration tracker Registration tracker (variation) Operation concept diagram of this system System processing flow diagram Illustration of operations when creating medical images Operation explanatory diagram at the time of target image projection Functional block diagram of system (second embodiment) System processing flow diagram Pseudo three-dimensional image principle explanatory diagram Pseudo three-dimensional image operation explanatory diagram Functional Block Diagram of System (Second Embodiment) (Modification) Registration tracker (variation) Operation explanatory diagram at the time of target image projection (third embodiment)

<First Embodiment>
~Constitution~
FIG. 1 is a functional block diagram of the system. The structure which concerns on the surgery assistance system of this embodiment is demonstrated. The surgery support system includes a camera 1, a projector 2, a registration tracker 3, and an image processing device 4.

  The camera 1 and the projector 2 are a unit 5. Therefore, the positional relationship between the camera 1 and the projector 2 is known and stored in the image processing device 4.

  The optical characteristics (lens characteristics) of the camera 1 and the projector 2 are known.

  The camera 1 photographs the registration tracker 3. The projector 2 projects the target image onto the projection target. The target image is an image related to a target inside the projection target, and is acquired based on a medical image acquired separately.

  FIG. 2 is an example of a registration tracker. The registration tracker 3 is preferably a polyhedron (see the second embodiment). In the figure, it is a hexahedron.

  At the time of creating a medical image, the registration tracker 3 is installed at an arbitrary position on the body surface (see FIG. 6). However, it is installed at the same position when the target image is projected (see FIG. 7).

  In addition, when imaging the registration tracker 3 as a CT image, it is preferable that the registration tracker 3 is a material (for example, wood) harder than the living skin tissue.

  The three-dimensional shape information of the registration tracker 3 is known and stored in the image processing device 4.

  A reflection type registration marker 6 is provided on the surface of the registration tracker 3. The registration marker 6 is provided on at least one surface so as to face the camera 1, but it is preferable that the registration marker 6 be provided on a plurality of surfaces as shown (see the second embodiment).

  The positional relationship of the registration marker 6 in the registration tracker 3 is known and stored in the image processing device 4.

  FIG. 3 shows a modification of the registration tracker. The registration tracker 3 in FIG. 2 is a hexahedron, whereas the modified example has a plate shape. That is, it need not be a polyhedron. In FIG. 2, the registration marker 6 is provided on the surface of the registration tracker 3, whereas in the modified example, the four corners of the registration tracker 3 function as the registration marker 6.

  That is, the registration tracker is not limited to this as long as it can acquire three-dimensional shape information and can be identified by a camera.

  Returning to FIG. 1, the functional units 41 to 47 of the image processing apparatus 4 will be described.

  The camera coordinate system setting function unit 41 sets a three-dimensional coordinate system viewed from the camera 1 based on the optical characteristics of the camera 1.

  The projector coordinate system setting function unit 42 sets a three-dimensional coordinate system viewed from the projector 2 based on the optical characteristics of the projector 2.

The registration marker photographing function unit 43 obtains registration marker information Mm (for example, three-dimensional coordinates of four points at the four corners of the marker) in the camera coordinate system based on the photographed image information m _m (two-dimensional) of the registration marker 6. get. Furthermore, based on the positional relationship between the registration tracker 3 and the registration marker 6 and the registration marker information Mm in the camera coordinate system, the registration tracker information Mt in the camera coordinate system (for example, 4 at the four corners of the opposing surface). 3D coordinates of the point) are acquired.

  For convenience of explanation, only one surface of the registration tracker is used. However, when there are registration markers on a plurality of surfaces (see FIG. 2), the camera 1 identifies the registration markers on the plurality of surfaces. This improves the accuracy of registration.

  The registration tracker information conversion function unit 44 acquires registration tracker information Pt in the projector coordinate system based on the positional relationship between the camera 1 and the projector 2 and the registration tracker information Mt in the camera coordinate system.

  The medical image input function unit 45 inputs registration tracker information CTt of the medical image coordinate system and target information CTg of the medical image coordinate system acquired in advance.

  The target photographed image acquisition function unit 46 calculates the projector based on the positional relationship between the registration tracker information CTt in the medical image coordinate system and the target information CTg in the medical image coordinate system, and the registration tracker information Pt in the projector coordinate system. Get target information Pg of coordinate system.

Dimensional transform function unit 47 converts the target information of the projector coordinate system from the three-dimensional coordinates Pg in two-dimensional coordinates p _g.

~ Operation ~
FIG. 4 is a conceptual diagram of the operation of this system, and FIG. 5 is a processing flow diagram of the system. The operation | movement which concerns on the surgery assistance system of this embodiment is demonstrated.

  The medical image coordinate system registration tracker information CTt and the medical image coordinate system target information CTg are created in advance by a medical image creation device (see FIG. 6) (step S1). Medical images include CT images and MRI images. Three-dimensional information can be obtained by overlapping two-dimensional images in layers.

  FIG. 6 is an explanatory diagram of an operation when creating a medical image. The illustration shows a CT scanning device. At the time of CT scan, the registration tracker 3 is installed at an arbitrary position on the body surface. However, it is necessary to set the same position as the installation position of the registration tracker 3 at the time of projecting the target image (see FIG. 7). Therefore, consideration is given so as not to hinder target image projection.

  Usually, medical images are created the day before surgery. After creating the medical image, the registration tracker 3 is marked on the body surface, and then the registration tracker 3 is removed.

  Registration tracker information CTt in the medical image coordinate system and target information CTg in the medical image coordinate system are input to the image processing apparatus 4.

  Next, the operation during the operation will be described. FIG. 7 is an explanatory diagram of an operation at the time of projecting a target image.

  The installation position of the registration tracker 3 is marked on the patient's body surface. The registration tracker 3 is installed at a predetermined position.

  The unit 5 including the camera 1 and the projector 2 is installed so as to correspond to the projection target (predetermined range on the patient body surface) (step S2). More specifically, the projector 2 can project the target image onto the projection target so that the camera 1 can photograph the registration tracker 3.

  The camera 1 images the registration tracker 3 installed on the surface of the projection target (step S3).

  The image processing apparatus 4 performs registration based on the image taken by the registration tracker by the camera and the registration tracker information acquired at the same time as acquiring the medical image, and the inside of the projection target (under the body surface) Target information, which is image information related to the target at, is acquired (step S4). Details of the image processing will be described below.

  First, registration tracker information Mt in the camera coordinate system is acquired based on a photographed image of the registration tracker 3 by the camera 1 (step S41).

The camera 1 identifies the registration marker 6. The actual registration DOO marker 6 to form a dimension known square (see FIG. 2), registration preparative marker 6 in the captured image is a two-dimensional form a quadrangle m _m was crushed. Based on the degree of deformation, registration marker information Mm (three-dimensional coordinates) in the camera coordinate system is acquired.

  In addition, the positional relationship between the registration tracker 3 and the registration marker 6 is known, whereby registration tracker information Mt in the camera coordinate system can be acquired.

  Furthermore, the positional relationship between the camera 1 and the projector 2 is known, and registration tracker information Pt in the projector coordinate system is acquired (step S42).

  This completes registration. For confirmation, a test image may be projected onto the registration tracker 3.

  The positional relationship between the registration tracker information CTt in the medical image coordinate system and the target information CTg in the medical image coordinate system is known, and the target information Pg in the projector coordinate system is acquired (step S43).

The projection image of the projector 2 is two-dimensional. Therefore, to convert the target information of the projector coordinate system from the three-dimensional coordinates Pg in two-dimensional coordinates p _g (step S44).

The projector 2 based on the 2-dimensional target information p _g, projecting a target image on a projection target (step S5).

  The surgeon can easily grasp the internal structure intuitively from the target image. Thereby, an operation is started.

~effect~
As described above, in this system, projector registration can be performed fully automatically. There is no risk of disruption of the surgical site due to a registration failure. As a result, the burden on the operator for introducing the system is greatly reduced, and the operator can concentrate on the operation.

  Furthermore, with the support of the target image, the operator can easily grasp the internal structure intuitively. As a result, the burden on the surgeon for the surgery is greatly reduced, and a reduction in surgical mistakes can be expected.

Second Embodiment
~ Configuration / Operation ~
FIG. 8 is a functional block diagram of the system of the second embodiment. The structure which concerns on the surgery assistance system of this embodiment is demonstrated. The operation support system includes the camera 1, the projector 2, the registration tracker 3, and the image processing device 4 in common with the first embodiment. The second embodiment further includes a unit moving mechanism 7 that moves the unit 5.

  For example, in FIG. 7, the unit moving mechanism 7 is a rail that moves the unit 5 and a driving device. The rail is provided in an annular shape on the upper part of the operating table. For understanding, a partial rail cross section is displayed. As the unit 5 makes a round on the rail, the projector 2 can project from all directions.

  The image processing apparatus 4 further includes a movement detection function unit 48. The movement detection function unit 48 receives movement information from the unit moving mechanism 7 and outputs a control command repeatedly when detecting movement of the camera 1.

  FIG. 9 is a processing flowchart of the system according to the second embodiment. The operation | movement which concerns on the surgery assistance system of this embodiment is demonstrated. The operations from step S1 to S5 are common to the first embodiment. In the second embodiment, the process related to step S6 is further performed.

  When the unit moving mechanism 7 moves the unit 5, the position of the projector 2 changes and it is necessary to register again. The image processing device 4 inputs the movement information output by the unit moving mechanism 7 and repeatedly outputs a control command (step S6).

  Again, the camera 1 images a registration tracker installed on the surface of the projection target (step S3), and the image processing apparatus 4 performs registration and acquires target information for projection (step S4). The projector 2 projects the target image onto the projection target (step S5), and a series of operations are repeated.

  The registration tracker 3 is a polyhedron, and there are registration markers on a plurality of sides (see FIG. 2). When the unit 5 moves greatly, the camera 1 recognizes the registration marker on the other surface, and the image processing apparatus 4 performs registration again.

~effect~
During the operation, the position of the operator may change, and the operator may become an obstacle and the target image may not be projected onto the projection target.

  In the present embodiment, the unit 5 is moved with the change of the operator position. Since registration is performed in real time, the target image can be projected again from another direction. Thereby, it can be avoided that the surgeon becomes an obstacle to projection.

  Furthermore, in this embodiment, a pseudo three-dimensional image is obtained. This will be described in detail below.

  FIG. 10 is an explanatory diagram of the principle of a pseudo three-dimensional image. When there are two objects on the same line of sight and the line of sight is moved to the left and right, a near object appears to move greatly and a distant object appears to move slightly. This phenomenon is called visual motion parallax. Thereby, the depth can be recognized.

  FIG. 11 is an explanatory diagram of an operation for obtaining a pseudo three-dimensional image. As the unit 5 moves, the projection direction from the projector 2 changes. As a result, a phenomenon similar to visual motion parallax occurs. That is, the object in the deep part from the body surface moves little, and the object near the body surface moves greatly. In the illustrated example, the movement of the sternum is larger than the movement of the spine. Thereby, the depth can be intuitively recognized.

  Although the projection image from the projector 2 is two-dimensional information, a pseudo three-dimensional image is obtained by the movement of the unit 5.

~ Modification ~
FIG. 12 is a functional block diagram according to a modification. In FIG. 8, the movement detection function unit 48 inputs movement information from the unit moving mechanism 7, whereas in FIG. 12, the movement detection function unit 48 inputs movement information from the registration marker photographing function unit 43.

  That is, when the unit 5 moves, the field of view of the camera 1 also moves, and the camera 1 cannot recognize a known registration marker. The image processing device 4 repeatedly outputs a control command based on the change in the captured image by the camera 1 (step S6).

  7 and 8, the unit is moved by the unit moving mechanism 7, but the unit 5 can be made portable by downsizing the camera and the projector. For example, the same effect as in the second embodiment can be obtained when the surgical assistant moves with the unit 5.

  FIG. 13 shows a modification of the registration tracker. In the second embodiment, it has been described that the registration tracker 3 is preferably a polyhedron. In FIG. 2, it is a hexahedron, whereas in FIG. 13, it is a dodecahedron. This improves the registration accuracy.

<Third Embodiment>
The present invention is a surgery support system, but is not limited to medical treatment or surgery, and can be applied to a case where the objective is to project an internal structure by a projector.

  FIG. 14 is an example of application to a nondestructive inspection of a projector system. The system configuration and system processing flow are the same as those of the surgery support system.

  For example, after an earthquake, even if there is no damage to the concrete structure by visual confirmation, the internal rebar may be damaged. On the other hand, ultrasonic inspection of the reinforcing bar is performed from the concrete surface to confirm that the reinforcing bar is healthy. The operation of the application example will be described below.

  First, three-dimensional CAD information of a concrete structure and bar arrangement is created in advance based on a design drawing. Further, the installation position of the registration tracker 3 is determined in advance and added to the three-dimensional CAD information (step S1). As a result, the positional relationship between the registration tracker information CDt in the three-dimensional CAD coordinate system and the target information CDg in the three-dimensional CAD coordinate system becomes known.

  A registration tracker 3 is installed at a predetermined position on the surface of the concrete structure. Further, the unit 5 including the camera 1 and the projector 2 is installed so as to correspond to the projection target (inspection range of the concrete structure) (step S2). More specifically, the projector 2 can project the target image onto the projection target so that the camera 1 can photograph the registration tracker 3. Whether the camera 1 is properly capturing the registration tracker 3 may be confirmed by a monitor.

  As soon as the above preparation is completed, the bar arrangement image is projected onto the concrete structure. The camera 1 images the registration tracker 3 installed on the surface of the projection target (step S3), and the image processing apparatus 4 performs registration and acquires target information for projection (step S4). 2 projects a target image (reinforcement image) onto a projection target (step S5).

  Further, a pseudo three-dimensional image may be obtained by moving the unit 5 (step S6).

  The target rebar position is estimated while referring to the projection image, and ultrasonic exploration is performed. By visualizing the bar arrangement, it is possible to identify the survey target and improve the search accuracy.

  At the survey site, projector registration can be performed fully automatically, so that the investigator can concentrate on the survey without taking the trouble of projection work.

DESCRIPTION OF SYMBOLS 1 Camera 2 Projector 3 Registration tracker 4 Image processing apparatus 5 Unit 6 Registration marker 7 Unit moving mechanism 41 Camera coordinate system setting function part
42 Projector coordinate system setting function unit 43 Registration marker imaging function unit 44 Registration tracker information conversion function unit 45 Medical image input function unit 46 Target captured image acquisition function unit 47 Dimension conversion function unit 48 Movement detection function unit

Claims (13)

A unit including a camera and a projector, which is movable during projection, a registration tracker installed on the surface of the projection target, and an image processing device,
The positional relationship between the camera and the projector is known,
The registration tracker has a three-dimensional shape that can be acquired simultaneously with acquisition of a three-dimensional medical image,
The three-dimensional shape includes a plurality of planes,
Two or more planes of the plurality of planes are provided with a plurality of marker points that can be photographed by the camera,
The positional relationship between the marker points is known,
The positional relationship of the marker points in the registration tracker is known;
The image processing apparatus includes:
First registration tracker information acquired from a captured image of the registration tracker by the camera;
Second registration tracker information acquired at the same time as acquiring a three-dimensional medical image in advance;
Based on the registration,
Obtaining target information for the projector to project onto the projection target;
A surgical operation support system that converts target information in a projector coordinate system from three-dimensional coordinates to two-dimensional coordinates. A camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device,
The image processing apparatus includes:
A camera coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the camera;
A projector coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the projector;
A registration tracker information conversion function unit for acquiring registration tracker information of the projector coordinate system based on a positional relationship between the camera and the projector and registration tracker information of the camera coordinate system;
A medical image input function unit for inputting registration tracker information of a medical image coordinate system acquired in advance and target information of a medical image coordinate system;
Target imaging that acquires target information of the projector coordinate system based on the positional relationship between the registration tracker information of the medical image coordinate system and the target information of the medical image coordinate system and the registration tracker information of the projector coordinate system An image acquisition function unit;
A dimension conversion function unit for converting target information of the projector coordinate system from three-dimensional coordinates to two-dimensional coordinates;
A surgical operation support system characterized by comprising: The camera shoots the registration tracker,
The image processing apparatus includes:
The surgery support system according to claim 1, further comprising: a registration tracker imaging function unit that acquires registration tracker information of the camera coordinate system based on a captured image of the registration tracker by the camera. The registration tracker is provided with a registration marker,
The camera photographs the registration marker,
The image processing apparatus includes:
The registration marker information of the camera coordinate system is acquired based on the photographed image of the registration marker, and based on the positional relationship between the registration tracker and the registration marker, and the registration marker information of the camera coordinate system The surgery support system according to claim 1, further comprising a registration marker imaging function unit that acquires registration tracker information of the camera coordinate system. The surgery support system according to claim 1, wherein the projector projects a target image, which is an image related to a target inside the projection target, onto the projection target based on the target information. The surgery support system according to claim 1 or 2, wherein the registration tracker information of the medical image coordinate system is acquired in a state where the registration tracker is installed at a predetermined position on the surface of the projection target. The surgery support system according to claim 1, further comprising a unit moving mechanism that moves a unit including the camera and the projector. The surgery support system according to claim 1, wherein the registration tracker is a polyhedron. The image processing apparatus includes:
The surgery support system according to any one of claims 1 to 8, further comprising a movement detection function unit that repeatedly outputs a control command when movement of the camera is detected. A camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device,
The image processing apparatus includes:
A camera coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the camera;
A projector coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the projector;
A registration tracker information conversion function unit for acquiring registration tracker information of the projector coordinate system based on a positional relationship between the camera and the projector and registration tracker information of the camera coordinate system;
A pre-image input function unit for inputting registration tracker information of the pre-image coordinate system acquired in advance and target information of the pre-image coordinate system;
Target that acquires target information of the projector coordinate system based on the positional relationship between the registration tracker information of the prior image coordinate system and the target information of the prior image coordinate system and the registration tracker information of the projector coordinate system A captured image acquisition function unit;
A dimension conversion function unit for converting target information of the projector coordinate system from three-dimensional coordinates to two-dimensional coordinates;
A projector system comprising: Using a registration tracker with a three-dimensional shape that can be acquired simultaneously with the acquisition of a three-dimensional medical image,
The three-dimensional shape includes a plurality of planes,
Two or more planes of the plurality of planes are provided with a plurality of marker points that can be photographed by the camera,
The positional relationship between the marker points is known,
The positional relationship of the marker points in the registration tracker is known;
The medical image creation device creates the second registration tracker information simultaneously with the creation of the 3D medical image related to the target,
Using a surgery support system comprising a unit including a camera and a projector, which is movable during projection, a registration tracker installed on the surface of the projection target, and an image processing device,
The positional relationship between the camera and the projector is known,
The unit is installed to correspond to the projection target,
The camera shoots a registration tracker installed on the surface of the projection target,
The image processing apparatus is
First registration tracker information acquired from a captured image of the registration tracker by the camera;
Second registration tracker information acquired at the same time as acquiring a three-dimensional medical image in advance;
Based on the registration,
Get target information that is image information about the target inside the projection target,
Convert the target information of the projector coordinate system from 3D coordinates to 2D coordinates,
A medical image projecting method, wherein the projector projects a target image onto a projection target based on target information of the two-dimensional coordinates. The medical image creation device creates registration tracker information in the medical image coordinate system and target information in the medical image coordinate system in advance,
Of the surgical support system including a camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device, a unit including the camera and the projector corresponds to the projection target. Installed as
The camera shoots a registration tracker installed on the surface of the projection target,
The image processing apparatus is
Based on the captured image of the registration tracker by the camera, obtain registration tracker information of the camera coordinate system,
Based on the positional relationship between the camera and the projector and the registration tracker information of the camera coordinate system, obtain registration tracker information of the projector coordinate system,
Based on the positional relationship between the registration tracker information of the medical image coordinate system and the target information of the medical image coordinate system, and the registration tracker information of the projector coordinate system, obtain target information of the projector coordinate system,
Convert the target information of the projector coordinate system from 3D coordinates to 2D coordinates,
A medical image projecting method, wherein the projector projects a target image onto a projection target based on the two-dimensional target information. A unit including the camera and the projector is moved;
Each time the unit moves,
The medical image projection method according to claim 11 or 12, wherein photographing by the camera, image processing by the image processing device, and projection by the projector are repeated.

Images