JP2007151686A - Inside-human body information projection system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To project in vivo information on the body surface of a patient following the movement of the patient and to protect the privacy of the patient. <P>SOLUTION: The inside-human body information projection system includes a computer. The computer acquires a far infrared ray image photographed by a thermal camera 14, detects the position and posture of a thermal marker 18 from the acquired image, and specifies the patient equipped with the thermal marker 18 for instance. The computer estimates the position and posture of the patient from the position and posture of the thermal marker, calculates the projection position of the 3DCG model of the in vivo information of the patient for instance matched with the position of the patient, and deforms the posture of the 3DCG model matched with the posture of the patient. Then, the computer projects the 3DCG model of the in vivo information on the body surface of the patient by using a projector 16. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a human body internal information projection system, and more particularly to a human body internal information projection system that projects internal information of a human body such as organs, muscles, bones, joints, and blood vessels.

  For example, Non-Patent Document 1 describes a technique for projecting a chest skeleton image onto a device to which a retroreflective material is applied or pasted. In this technique, since retroreflected light is observed, a projector is arranged at a position optically conjugate with the observer's viewpoint, and projection is performed from the position onto the screen via a half mirror.

In addition, a software library called ARTToolKit developed by Kato et al. As a means to measure the position and orientation of a marker used as a reference by image processing for the alignment of the real world and the virtual world in an augmented reality system Is known (see Non-Patent Document 2). In this technique, the position and orientation of the camera with respect to the marker are measured in real time based on an image photographed by a color camera attached to an HMD (Head Mounted Display). For example, an image in which a three-dimensional virtual character appears on a card (marker) is displayed on the HMD.
Inami, Kawakami, Yanagida, Maeda, Kaoru, "Research of reality fusion (1st report)-Proposal and experimental implementation of seamless reality fusion method", Proceedings of the 3rd Annual Conference of the Virtual Reality Society of Japan, 281- 284, 1998 ARTToolKit, http://www.hitl.washington.edu/artoolkit/

  In the technique of Non-Patent Document 1, a retroreflective material is used as a screen, and for example, an image cannot be projected onto a patient's hand or shirt. Moreover, the observer had to exist in the position which can observe the retroreflection light from a screen. However, since the patient holds the retroreflective screen in front of the body, it is difficult for the patient to see the projected image. Thus, since it was difficult for the medical staff and the patient to share the projected information, smooth communication could not be established between the two. In addition, since the patient's movement could not be detected, the patient who had the screen had to be stationary on the spot. Thus, it was not possible to cope with the movement and movement of the patient. For example, in scenes involving exercise such as rehabilitation, it is difficult not only to hold the screen to the patient but also to project an image to the patient.

  In the technique of Non-Patent Document 2, a marker can be tracked and its position and posture can be detected, but a color video camera is used. Therefore, when it is assumed that a virtual object is projected onto the real world using a projector, the marker and the projection image overlap each other, which makes it difficult to detect the marker. In order to reduce the calibration error, it is desirable to place the marker in the projection area. As described above, since the position and orientation of the marker cannot be tracked, it cannot be applied to an augmented reality system based on a projector, that is, a virtual object corresponding to the position and orientation of the marker is implemented. Projecting to the world was difficult. Even if an image in which a virtual object is superimposed on the real world is displayed on an HMD or a portable display, smooth communication between the medical staff and the patient is not desired, and in a situation involving rehabilitation, the patient Will be forced to bear further burdens. Therefore, wearing an HMD, possessing a display, etc. are not realistic. Further, since the marker is detected by a color video camera, information relating to privacy such as the body and face of the patient is captured as a color image. Therefore, when the technique of Non-Patent Document 2 is used for medical treatment, there is a serious problem that the privacy of the patient cannot be protected.

  Therefore, a main object of the present invention is to provide a human body internal information projection system capable of realizing smooth communication between a patient and a medical staff.

  Another object of the present invention is to provide a human body internal information projection system capable of projecting in-vivo information onto the patient's body surface following the movement of the patient and protecting the privacy of the patient. is there.

  The invention according to claim 1 is a system for projecting internal human body information onto a person, a thermal marker mounted at a predetermined position of the human, a thermal camera for acquiring a far-infrared image, and internal human body information. Storage means, first detection means for detecting the position of the thermal marker based on the far-infrared image acquired by the thermal camera, and a position for projecting human body internal information based on the position of the thermal marker detected by the first detection means A human body internal information projection system comprising: a position calculation means for calculating; and a projection means for projecting human body internal information onto a human body surface based on the position calculated by the position calculation means.

  The invention according to claim 1 is a system for projecting internal information of a human body such as an organ, skeleton, muscle or blood vessel onto a person such as a patient. The internal body information is, for example, a 3DCG model of the organ, and is stored in advance in the storage unit. A thermal marker is attached to a person at a predetermined position. A pattern that gives the intensity distribution of the heat rays is formed on the thermal marker. When a far-infrared image of a person is photographed with a thermal camera through the thermal marker, the pattern appears in the far-infrared image. The first detection means detects the position of the thermal marker based on such a far infrared image. The position calculating means calculates a position at which the human body internal information is projected based on the detected position of the thermal marker. Then, the human body internal information is projected by the projecting means onto a position on the human body surface and corresponding to the position where the human body internal information exists.

  Thus, according to the invention of claim 1, since the position of the thermal marker is detected based on the far-infrared image captured by the thermal camera, the position of the person wearing the thermal marker is tracked. can do. Then, the human body internal information is projected at a position determined based on the detection position of the thermal marker. Even if the internal information of the human body is projected on the patient body surface, the far-infrared image is taken by the thermal camera, so that the thermal marker, that is, the patient can be detected. Therefore, it is possible to project human internal information on the body surface of the patient following the movement of the patient. Thus, since the internal information of the human body can be shared between the medical staff and the patient, smooth communication between the two can be realized. In addition, since the position of the patient is detected by the thermal marker and the thermal camera using invisible information such as the patient's heat, the patient's privacy is protected without burdening the patient.

  The invention according to claim 2 is dependent on the invention according to claim 1, wherein the first detection means further detects the attitude of the thermal marker, and internal information on the human body based on the position and attitude of the thermal marker detected by the first detection means. And a posture changing means for changing the posture. The projecting means projects the human body internal information in a posture deformed by the posture deforming means.

  In the invention of claim 2, the first detection means further detects the attitude of the thermal marker based on the far-infrared image. The posture deforming unit deforms the posture of the human body internal information based on the detected position and posture of the thermal marker. Since the thermal marker is mounted at a predetermined position of the patient, the position and posture of the patient can be grasped by detecting the position and posture of the thermal marker. Therefore, the in-vivo information of the posture transformed according to the patient's posture can be projected on the patient's body surface.

  The invention of claim 3 is dependent on the invention of claim 1 or 2, wherein the thermal marker is formed with a pattern associated with human identification information, and the first detection means further detects the pattern of the thermal marker, The projecting means projects human body internal information corresponding to the pattern detected by the first detecting means.

  In the invention of claim 3, the thermal marker is formed with a pattern associated with identification information of the person to be worn. The first detection means further detects the pattern of the thermal marker, so that the person wearing the thermal marker can be specified by the detection pattern. The projecting means projects human body internal information corresponding to the detection pattern. That is, it is possible to project human body internal information corresponding to the person wearing the thermal marker, for example, human internal information of the patient himself or human internal information corresponding to the affected part of the patient.

  According to this invention, since the position of the thermal marker is detected and the projection position of the human body internal information is calculated based on the detected position, the human body internal information is transferred to the patient following the movement of the patient wearing the thermal marker. Can be projected onto the body surface. Therefore, the inside of the human body can be presented to the patient in an easy-to-understand manner, and the internal information of the human body can be shared between the patient and the medical staff. As described above, the visual support enables smooth communication between the patient and the medical staff, and for example, effective rehabilitation can be performed. Moreover, since the patient wearing the thermal marker is photographed with a thermal camera, it is possible to avoid the situation of photographing information related to privacy such as the patient's face and body with a color camera, and to protect the privacy of the patient. Can do.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a human body internal information projection system (hereinafter also simply referred to as “system”) 10 of this embodiment is for projecting human body internal information onto a human body surface. For example, an inquiry or rehabilitation is performed. It can be applied in medical consultation such as. The system 10 includes a computer 12, a thermal camera 14 and a projector 16. A thermal marker 18 is attached to a person to be projected, that is, a patient.

  The computer 12 includes a CPU, RAM, HDD, input device, display device, and the like. The HDD stores a program for controlling the overall operation of the system 10 and necessary data. The CPU loads the program into the RAM and executes processing while generating or acquiring temporary data in the RAM according to the program. The computer 12 further includes a video capture card and a graphic board. A thermal camera 14 is connected to the video capture card, and a projector 16 is connected to the graphic board.

  The thermal camera 14 is an image sensor that detects or observes far infrared rays (heat rays). The thermal camera 14 is installed in a predetermined direction in a predetermined position of a room where medical care such as medical examination or rehabilitation is performed so that a person, that is, a patient can be photographed. Since the temperature of the human body is higher than the environmental temperature, far infrared rays emitted from the body temperature can be detected separately from the environment. Based on the far-infrared image acquired by the thermal camera 14, the movement of the body of the patient to which the thermal marker 18 is attached is acquired. For example, the thermal camera 14 outputs the captured infrared video signal to the computer 12, the video capture card of the computer 12 converts the video signal into digital data at a predetermined frame rate, and the CPU stores the infrared image data in the RAM. get.

  Since the thermal camera 14 is not sensitive to the visible light wavelength, it is impossible to recognize an individual's face from the captured image, and it is also impossible to photograph the patient's skin itself. Thus, patient privacy is protected. Note that thermography has already been widely used in the medical field, and it is considered that there is no resistance to installing the thermal camera 14 at a medical site.

  The projector 16 projects an image onto the patient's body surface. The projector 16 is installed in a predetermined direction at a predetermined position in a room where medical care is performed so that it can be projected onto the patient's body surface. For example, an LCD projector may be used. When projecting an image, the CPU uses, for example, graphic data to display image data for displaying a two-dimensional image in which the 3DCG object is arranged at a desired position in a desired posture based on the 3DCG object data. The image data is generated and given to the projector 16, and the image is projected from the projector 16.

  The thermal marker 18 is a mark for detecting a patient as a subject based on an infrared image photographed by the thermal camera 14, and is attached to the patient's body surface, for example. It functions as a mark by forming an intensity distribution of the heat ray at the place where the thermal marker 18 is mounted using a heat ray that is derived from the body temperature of the patient. The patient can wear clothes made of a material that transmits far infrared rays, such as a cotton T-shirt. That is, the patient's body surface onto which the in-vivo information is projected includes clothing as well as the skin. When a patient wears clothing, the clothing needs to have a color that does not adversely affect the visibility of projected in-vivo information, such as white. An adhesive substance is provided on the back surface of the thermal marker 18 so that it can be attached to human skin or clothes. The thermal marker 18 has a predetermined shape and size (for example, a 30 mm × 30 mm square), and is attached to a predetermined position (for example, the sternum) of the body. In consideration of visibility, a plurality of thermal markers 18 may be distributed on the body surface. In a rehabilitation scene, a plurality of thermal markers 18 are attached to a plurality of appropriate positions on the body so that the posture and body movement of the patient can be detected with high accuracy. The size, shape, mounting site, and the like of the thermal marker 18 are appropriately determined by experiments so that the motion of the patient can be detected with higher accuracy.

  In the thermal marker 18, a distribution of physical characteristics such as permeability to heat rays, shielding property, or heat ray radiation, that is, a pattern, is formed depending on the presence or absence of materials having the respective properties. For example, in the thermal marker 18, the pattern may be formed by combining any one of a material that radiates far infrared rays, a material that blocks far infrared rays, and a material that transmits far infrared rays. Alternatively, the thermal marker 18 may be formed by cutting out a predetermined pattern from a material that blocks far infrared rays. The thermal marker 18 may be embedded in clothing worn by the patient by combining the above-described materials.

  The pattern to be formed may have any shape such as an alphabetic character, a number, a symbol, or a graphic as long as it can be recognized. However, the pattern must be capable of detecting the posture (direction) of the thermal marker 18. As shown in FIG. 2, the pattern may be an inverted triangle (A), a shape combining two different rectangles (B), a shape resembling a postal symbol (C), or the like. In this embodiment, an aluminum foil covered with urethane foam is used, and a pattern is formed by cutting out. That is, the heat ray is transmitted through the cut-out portion, and the heat ray is blocked at the material portion.

  At the location where the thermal marker 18 is attached, a heat difference is formed by transmitting, blocking, or radiating the heat rays from the body temperature corresponding to the shape of the pattern. Therefore, the far infrared intensity distribution corresponding to the pattern of the thermal marker 18 is generated at the location where the thermal marker 18 is present in the infrared image captured by the thermal camera 14. For example, when the thermal marker 18 of FIG. 2 (A) is used, in the infrared image, as shown in FIG. 3, the central inverted triangle portion is displayed white by heat ray transmission, and the surrounding portion is blocked by the heat ray. Is displayed in black. Therefore, the pattern of the thermal marker 18 can be detected by observing the heat ray of the patient through the thermal marker 18.

  In this embodiment, the pattern of the thermal marker 18 is different for each patient to be worn. That is, the pattern of the thermal marker 18 is associated with patient identification information. Therefore, in this embodiment, the patient can be specified by detecting the identification information, that is, the pattern of the thermal marker 18.

  An example of a technique such as the thermal marker 18 and personal identification using the thermal marker 18 is described in Japanese Patent Application No. 2004-205809 filed on Jul. 13, 2004 by the applicant of the present application.

  Further, the position and direction (posture) of the thermal marker 18 can be detected by extracting a pattern from the infrared image captured by the thermal camera 14. The ArtTool Kit (http://www.hitl.washington.edu/artoolkit/) described in the Background Art section can be used for extracting the pattern of the thermal marker 18 and detecting the position and orientation.

  ARToolKit is a software library that measures the position and orientation of a marker by image processing for alignment between the real world and the virtual world. ARTToolKit provides a function for detecting the position (three-dimensional coordinates) and posture or direction (pan, tilt, roll) of a two-dimensional marker from an input image. Further, the identification information of the marker, that is, the pattern can be automatically extracted.

  A program including the function of ARTToolKit is stored in a storage device such as an HDD of the computer 12. The computer 12 detects the position and orientation of the thermal marker 18 together with the pattern of the thermal marker 18 based on the infrared image acquired by the thermal camera 14 using the ARTToolKit function.

  Since the thermal marker 18 is mounted at a predetermined position on the patient's body as described above, the position and posture of the patient can be estimated based on the detected position and posture of the thermal marker 18. For example, a storage device such as an HDD or a RAM of the computer 12 stores initial data indicating, for example, a natural body or a mounting position and direction of the thermal marker 18 in a predetermined posture. Therefore, the computer 12 can detect the body movement such as the position and posture of the patient in real time based on the detected position and direction of the thermal marker 18 and the stored initial position and direction. Note that standard initial data may be acquired by experiments or the like and stored in advance in the storage device of the computer 12. Alternatively, the initial data may be acquired in advance or before the examination and rehabilitation by attaching the thermal marker 18 to the patient himself and making it natural or in a predetermined posture.

  Thus, since invisible information such as the heat of the human body is used as an information source, no change occurs in the appearance of the patient, and management information is not unnecessarily known to a third party. Therefore, by using the thermal marker 18 and the thermal camera 14, it is possible to detect the position and posture of the patient's body without adding a new load to the patient and without violating the privacy information.

  Since the patient is present at the detection position of the thermal marker 18, the computer 12 can superimpose an image on the body surface of the patient using the projector 16. The image projected by the projector 16 is human body internal information, and specifically, is a body organ or component other than the skin, such as a skeleton, muscles, blood vessels, and internal organs.

  As shown in FIG. 1, the computer 12 includes an in-vivo information database (DB) 20 and a patient information DB 22 inside or outside. In the in-vivo information DB 20, human internal information (in-vivo information) projected on the patient is registered. The human body internal information is a three-dimensional CG model, which may be obtained by photographing the patient himself or may be a standard model.

  Specifically, as shown in FIG. 4, the in-vivo information DB 20 stores a plurality of three-dimensional CG object data in association with the in-vivo information ID. The file format of the three-dimensional object data may be VRML (Virtual Reality Modeling Language) 2.0, for example. For example, a 3DCG model of an individual patient such as AA's hip joint is stored together with standard models such as a lung sample, a heart sample, a digestive system sample, and a torso skeleton sample. When the patient's individual 3DCG model is not registered, the standard model is projected. The patient's individual 3D CG object data is 3D model data created from medical images (X-ray CT, MRI, ultrasound, etc.) of the patient. The standard model may be created by a human hand using CG software, or may be created from a medical image of an unspecified person. Moreover, as a standard model, you may prepare different data for every physique (appearance size, sex, etc.).

  The patient information DB 22 stores information about the patient as shown in FIG. For example, an electronic medical record may be stored. Specifically, information relating to the marker ID, the patient's own in-vivo information, the affected area information, the physique information, and the like is stored in association with the patient ID. The marker ID is identification information of the thermal marker 18 attached to the patient. For example, the ID of the pattern formed on the thermal marker 18 may be stored.

  When the patient's own 3DCG model is registered in the in-vivo information DB 20, the in-vivo information ID is stored in the patient's own in-vivo information, and the patient's own 3DCG model is not registered in the in-vivo information DB 20 For example, data indicating none is stored. In the example of FIGS. 4 and 5, “Mr. AA's hip joint” is stored in the in-body information DB 20, and therefore, the in-vivo information of Mr. AA's patient in the patient DB 22 is “100005” as the in-vivo information ID. It is remembered.

  In the affected area information, information indicating the affected area of the patient is stored. For example, when the patient's own in-vivo information is not registered, the in-vivo information of the standard model corresponding to the affected part information may be projected. In addition, a standard model that matches the physique of the patient may be selected.

  In the physique information, information indicating the physique of the patient is stored. For example, information distinguished by appearance size and gender such as an adult male, an adult female, a child male, and a female child is stored. Alternatively, the physique information may be information distinguished by height, weight, sex, and the like. In this embodiment, the projection position of each 3DCG model is adjusted based on this physique information. This is because each in-vivo information, that is, the arrangement of each organ differs depending on the physique of the person. By calculating the position where the in-vivo information is projected in consideration of the patient's physique, the in-vivo information can be projected at a more accurate position.

  The projection position of the in-vivo information is calculated based on the detected position of the thermal marker 18. For example, the storage device of the computer 12 stores in-vivo information arrangement data indicating the position of each in-vivo information with respect to the mounting position of the thermal marker 18, and the in-vivo information arrangement data and the detected thermal marker 18 are stored. Based on the position, a projection position of each in-vivo information is determined. Therefore, each in-vivo information is projected on the patient's body surface at a position corresponding to the position where the in-vivo information exists. In addition, when considering a physique like this Example, the in-vivo information arrangement | positioning data for every physique may be memorize | stored, and the in-vivo information arrangement | positioning data corresponding to a patient's physique information may be used.

  Further, the in-vivo information to be projected, that is, the posture of the 3DCG model is deformed according to the posture of the patient. As described above, the posture of the patient can be estimated based on the detected position and posture of the thermal marker 18. Therefore, the posture of the projected 3DCG model can be changed according to the posture of the patient.

  It should be noted that in normal use of ARTToolKit, the photographing process by the photographing device (for example, a color camera) and the display processing by the display device (for example, an LCD monitor) share the same coordinate system, so that calibration is not necessary. However, if the thermal camera 14 and the projector 16 are simply arranged in this system 10, the coordinate system of the two is not the same, so that the virtual world cannot be accurately projected at the designated location on the patient's body surface. . Therefore, calibration is required to unify the coordinate system of the thermal camera 14 and the projector 16 into a common coordinate system. The inventors conducted an alignment experiment between the thermal camera 14 and the projector 16 as pre-processing, prepared information for correcting the projection position in advance, and stored the projection position correction information in a storage device such as an HDD of the computer 12. I remember it. Before the in-vivo information is projected, the projection position of the in-vivo information is corrected based on the projection position correction information.

  In this way, using the position and posture of the patient acquired by detection of the thermal marker 18, for example, in order to maintain geometric and temporal consistency between the patient during rehabilitation and in-vivo information, for example. The in-vivo information can be transformed with time, and the deformed in-vivo information can be superimposed on the patient's body surface, which is the moving area.

  For example, as shown in FIG. 6, even when the patient P moves a relatively long distance within the projection range, by changing the projection position of the in-vivo information Q following the movement of the patient P, The in-vivo information Q can be projected onto a position corresponding to the position where the in-vivo information Q exists on the body surface of the patient P. Further, as shown in FIG. 7, even in a medical scene involving the movement of the patient P such as rehabilitation, the position of the in-vivo information Q (the skeleton of the lower limb including the hip joint in FIG. 7) in real time according to the movement of the patient P. And the posture can be changed and projected onto the body surface of the patient P. Therefore, the in-vivo information Q can be shared between the patient P and a medical worker M such as a doctor or a nurse.

  It should be noted that in-vivo information projected onto a part that cannot be seen directly from the patient's viewpoint, such as the face or back, can be forced to observe by the patient by using a mirror.

  Furthermore, even if the in-vivo information Q is projected on the body surface of the patient P by the projector 16, the thermal marker 18 can be detected by observing the far-infrared radiation with the thermal camera 14. Can always track, and can always grasp the position and posture of the patient P. Moreover, the acquired patient image is invisible, and the thermal marker 18 attached to the patient can be hidden or embedded in the clothing, and the pattern cannot identify personal information in appearance. Therefore, the privacy of the patient can be protected.

  Therefore, it is possible to visually support tasks such as patient care and communication between the medical staff and the patient, which have been left to the field experience. In-vivo information projected on the patient's body surface can be shared between the patient and medical staff, enabling smooth communication between the patient and medical staff, for example, effective rehabilitation become. Specifically, since the patient can receive treatment while confirming the effect of rehabilitation with his / her eyes based on the in-vivo information projected on the body surface, the patient's active approach can be derived. In addition, since the patient can proceed with rehabilitation while checking the state of his / her body, accidents due to unreasonable rehabilitation can be avoided. For example, in rehabilitation after hip joint replacement surgery, such as a hip joint, it is possible to prevent a situation in which a patient has caused a dislocation accident by forcibly moving his / her joint's movable range in an unexpected manner. .

  FIG. 8 shows an example of the projection processing operation performed by the CPU of the computer 12. This projection process is repeatedly executed at regular time intervals, for example, every one frame or every predetermined number of frames. When the projection process is started, first, in step S1, a far-infrared image captured by the thermal camera 14 is acquired. As described above, the computer 12 acquires image data from the video imaged by the thermal camera 14 at a predetermined sampling rate.

  Next, in step S3, image processing is executed. Specifically, preprocessing for recognizing the thermal marker 18 is executed. In the above-mentioned ARTToolKit, a visible light image is assumed as input information, and infrared light is much weaker than visible light. Therefore, in order to increase the contrast ratio between the part of the thermal marker 18 and the part of the body surface, preprocessing is performed on the acquired infrared image.

  Subsequently, in step S5, the thermal marker 18 is recognized using the function of ARToolKit. For example, the thermal marker 18 is extracted from the preprocessed image by pattern recognition. Data relating to the marker including information such as the pattern, size, and outer shape associated with the marker ID of the thermal marker 18 is stored in advance in the storage device. Then, the pattern of the thermal marker 18 is specified, and the position and direction (posture) of the thermal marker 18 are detected.

  In step S7, the in-vivo information reading process is executed. Thereby, the in-vivo information is read as necessary. An example of the operation of the in-vivo information reading process is shown in FIG.

  In the first step S31 of FIG. 9, the marker ID is identified from the recognized pattern of the thermal marker 18 and the patient information DB 22 is searched, and the patient ID corresponding to the thermal marker 18 is identified.

  Next, in step S33, it is determined whether or not the patient ID is recognized for the first time. That is, it is determined whether in-vivo information has not yet been projected on the patient. If “YES” in the step S33, an in-vivo information ID corresponding to the patient ID is specified in a step S35. For example, when the in-vivo information ID is registered as the in-vivo information of the patient in the patient information DB 22, the in-vivo information ID of the patient is read from the patient information DB 22 to the RAM. On the other hand, when the patient's own in-vivo information is not registered, the affected part information of the patient is read from the patient information DB 22 and the in-vivo information ID corresponding to the affected part is specified. In addition, as affected area information, in-vivo information ID corresponding to the affected area may be stored. In addition, when a plurality of in-vivo information or affected part information is registered in the patient, all of the in-vivo information IDs may be specified as projection information, or input from among the plurality of in-vivo information IDs One or several selected according to the input of the apparatus may be specified as projection information. When step S35 ends, the process proceeds to step S41.

  On the other hand, if “NO” in the step S33, that is, if some in-vivo information has already been projected on the patient, whether or not the in-vivo information to be displayed has been selected is input from the input device in the step S37. Judgment based on data. For example, the display device of the computer 12 displays an input screen on which the in-vivo information to be projected can be selected, and the medical staff can select the in-vivo information to be projected by operating the input device. It is determined whether or not the selection button has been clicked with a selection from the list.

  If “YES” in the step S37, that is, if the in-vivo information is selected, the in-vivo information ID corresponding to the selected in-vivo information is specified in the step S39. When step S39 ends, the process proceeds to step S41.

  If “NO” in the step S37, that is, if the in-vivo information has already been superimposed on the patient and the in-vivo information has not been selected, the in-vivo information reading process is ended as it is. Therefore, the projection of the current in-vivo information is continued.

  In step S41, the in-vivo information DB 20 is searched based on the specified in-vivo information ID, and the corresponding 3DCG object data is read into the RAM. When step S41 ends, the in-vivo information reading process ends, and the process returns to step S9 in FIG. In this way, the in-vivo information corresponding to the thermal marker 18 or the in-vivo information selected by the medical staff is read out at a necessary timing.

  In step S9 of FIG. 8, the in-vivo information, that is, the projection position of the 3DCG model is calculated. As described above, for example, the detected position of the thermal marker 18, the mounting position of the thermal marker 18 stored in the storage device, the physique information of the patient, and the in-vivo information based on the mounting position of the thermal marker 18 The projection position of the in-vivo information is determined based on the arrangement information. As a result, the 3DCG model is arranged at a corresponding position in the virtual three-dimensional world so as to exist at the projection position calculated on the projection plane.

  Subsequently, in step S11, the posture of the patient is estimated from the position and direction of the thermal marker 18. Since the mounting information (initial position and direction) of the thermal marker 18 is stored as described above, the current posture of the patient can be estimated based on the detected position and direction.

  In step S13, the in-vivo information, that is, the posture of the 3DCG model is converted according to the posture of the patient. For example, the posture that the 3DCG model should take in the virtual three-dimensional world is calculated according to the posture of the patient. Based on this attitude information, the 3DCG model is rotated in a virtual three-dimensional world.

  In step S15, the projection position is corrected. For example, the coordinates of the projection position of the 3DCG model are converted based on the projection position correction information stored in the storage device. Accordingly, the 3DCG model is moved in the virtual three-dimensional world so as to exist at the correction position on the projection plane.

  In step S17, the projector 16 projects the in-vivo information, that is, the 3DCG model, onto the patient body surface. For example, a drawing command is given to the graphic board together with the in-vivo information, that is, 3DCG object data, posture information, corrected position information, and the like, and image data for projecting the 3DCG model at a desired position and posture is generated. . Then, the image data or image signal is given to the projector 16 from the graphic board, and the projector 16 projects the image. Therefore, the organ transformed into a posture that matches the posture of the patient is superimposed on a position corresponding to the position where the organ actually exists on the patient's body surface. When step S17 ends, the process returns to step S1. In this way, the movement of the patient can be tracked, and in-vivo information matching the patient's posture can be projected onto the patient's body surface at a position corresponding to the position where the in-vivo information exists.

  In the above-described embodiment, not only the position and posture of the patient are detected by the thermal marker 18 but also the individual recognition of the patient can be performed. However, in other embodiments, the thermal marker 18 may not be associated with patient identification information. In this case, identification information of the patient to be projected may be input by a medical worker using the input device of the computer 12.

  In each of the above-described embodiments, an image in which the projection position of the in-vivo information is changed according to the position of the patient is generated and projected from the projector 16. However, in another embodiment, the in-vivo information is projected onto the patient surface by changing the projection position according to the position of the patient by the projector 16 having a displacement mechanism capable of panning and / or tilting the projection direction. May be.

  Further, in each of the above-described embodiments, one thermal camera 14 is provided. However, in another embodiment, each of the plurality of thermal cameras 14 may be provided at each predetermined position in the room, so that the detection accuracy of the position and posture of the patient as well as the visibility of the thermal marker 18 may be improved.

It is an illustration figure which shows an example of a structure of the human body internal information projection system of one Example of this invention. It is an illustration figure which shows an example of the pattern of a thermal marker. It is an illustration figure which shows the thermal image of the thermal marker of FIG. It is an illustration figure which shows an example of the data registered into in-vivo information DB. It is an illustration figure which shows an example of the data registered into patient information DB. It is an illustration figure which shows that in-vivo information is projected following a patient. It is an illustration figure which shows the outline | summary when the FIG. 1 Example is applied to rehabilitation. It is a flowchart which shows an example of the operation | movement in the projection process of CPU of the computer of FIG. 1 Example. It is a flowchart which shows an example of the operation | movement of the in-vivo information reading process of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Human body internal information projection system 12 ... Computer 14 ... Thermal camera 16 ... Projector 18 ... Thermal marker 20 ... In-body information DB
22 ... Patient information DB

Claims (3)

A system for projecting human internal information to a person,
A thermal marker to be worn at a predetermined position of the person,
Thermal camera for acquiring far-infrared images,
Storage means for storing the internal information of the human body;
First detection means for detecting a position of the thermal marker based on the far-infrared image acquired by the thermal camera;
Position calculating means for calculating a position for projecting the internal information of the human body based on the position of the thermal marker detected by the first detecting means; and the internal part of the human body based on the position calculated by the position calculating means. A human body internal information projection system comprising projecting means for projecting information onto the human body surface. The first detection means further detects the posture of the thermal marker;
Further comprising posture changing means for changing the posture of the internal information of the human body based on the position and posture of the thermal marker detected by the first detecting means;
The human body internal information projection system according to claim 1, wherein the projection unit projects the human body internal information in a posture deformed by the posture deformation unit. The thermal marker is formed with a pattern associated with the identification information of the person,
The first detecting means further detects the pattern of the thermal marker;
The human body internal information projection system according to claim 1, wherein the projection unit projects the human body internal information corresponding to the pattern detected by the first detection unit.

Images