JP2005348797A - Medical practice recording system and medical practice recording device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To compile moving images of medical practice after the medical practice with as little time and efforts of a user as possible, and to make the moving images of the medical practice easily handleable. <P>SOLUTION: The medical practice recording system for recording medical practice of an operator in the medical site comprises a video camera 11 for capturing images of the operation of the operator 1 performing the medical practice, a recording part 12 structured to record prescribed auxiliary information according to a command from outside, together with the moving images of the medical practice captured by the video camera 11, a detection part 13 including the space between the operator 1 and an assistant 2 for assisting the operator in a detection range for detecting the change of the state within the detection range, and a control part 14 for preparing the auxiliary information based on the result of detection of the detection part 13 and commanding the recording part 12 to record the auxiliary information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a medical practice recording system and a medical practice recording apparatus for recording a medical practice of an operator in a medical field.

  In recent years, opportunities for using images in the medical field are increasing. For example, in an operating room, microscopic images and endoscopic images are recorded and used for post-surgery studies and education. Alternatively, an attempt to record all moving images (videos) treating an affected part in an operating room with a video recorder has started.

  Specifically, in Non-Patent Document 1, a sensor is attached to a surgical instrument such as forceps to measure the position / posture of the surgical instrument during surgery, and the position / posture information is also recorded together with the image and sound of the surgical field. .

  In addition, as disclosed in Non-Patent Document 2, an attempt is made to integrate and record monitoring information, surgical field images, and the like on the same time axis.

The moving image recorded at the time of the operation is edited after the operation, and it is indispensable as a surgical record in a case study of a doctor. In addition, in relation to the problems related to medical errors that have been increasing in recent years, hospitals that are open to the public in hopes are increasing in order to prove that treatment by doctors was appropriate for patients undergoing surgery. Alternatively, it is very useful as a material for inexperienced doctors to learn surgical techniques and improve surgical techniques.
“Proposal and Prototype of Surgery Recording System”, Yuki Ikuta et al., J JSCAS, vol.4, no.2, 2002 "Development of surgical event recorder", Kiyoshi Naemura et al., J JSCAS, vol.3, no.3, 2001

  In order to use a moving image after shooting as described above, the moving image must be edited over a long period of time. However, at present, this operation is performed manually, that is, the staff (assistant) involved in the operation including the doctor (surgeon) who performs the operation is checking the moving image one by one with the editing device, and it takes a lot of time. There is a cost. For this reason, a method has been considered in which the operation staff tags the important part of the moving image during the operation and saves the trouble of using it. However, it is troublesome and troublesome during manual tagging.

  At the same time, from the viewpoint of recording medical practice, medical procedures, for example, recording of medical practice of a doctor in an operating room, are performed manually. There is a demand to automate these records.

  In addition, as described above, research on intelligent processing for video images is progressing.

  In the said nonpatent literature 2, the data disperse | distributed and time series of surgery relevant information are unified and recorded by collecting the monitor and image | video information which are disperse | distributed in the operating room in one place. However, the information is only integrated in chronological order, and it is not known what kind of treatment the surgeon performed at what point. For this reason, it takes time to discriminate important parts even when it is used after surgery.

  In Non-Patent Document 1, when recording an image, the position and orientation information of a surgical instrument such as forceps is recorded in time series, so that the skill of operating the instrument is converted into data, and the operator's vision is further visualized. Comments from information and audio information are added and recorded. However, according to this method, only the position / posture information of the treatment tool operation that is an action of the surgeon is a means for detecting the action of the surgeon, the amount of information is limited, and the complicated features are It cannot be extracted. In addition, because the position and orientation information at that time is recorded by the surgeon, processing such as predicting the risk at the next time in real time based on the movement record of the forceps up to a certain time Is impossible.

  The present invention has been made in view of the above points, and after a medical act, a medical act that can edit a moving image of the medical act as much as possible without making the user's effort as much as possible and make the moving image of the medical act easy to handle. An object is to provide a recording system and a medical practice recording apparatus.

One aspect of the medical practice recording system of the present invention is a medical practice recording system for recording a medical practice of an operator at a medical site,
A camera that captures the treatment of the surgeon performing the medical procedure;
A recording unit configured to record predetermined auxiliary information in accordance with an instruction from the outside, together with a time-series image group (moving image) of medical actions taken by the camera,
A detection means for detecting a change in a state in the detection area, including a surgeon and an assistant who assists the surgeon in the detection area;
Control means for creating the auxiliary information based on the detection result of the detection means and instructing the recording means to record the auxiliary information;
It is characterized by comprising.

Moreover, one aspect of the medical practice recording apparatus of the present invention is a medical practice recording apparatus that records a medical practice of an operator at a medical site,
A detection means for detecting a change in a state in the detection region, including a region between a surgeon performing a medical practice and an assistant assisting the operator in a detection region;
The auxiliary information is created based on the detection result of the detection means, and the auxiliary information is recorded in association with a time-series image group obtained by photographing the surgeon's treatment;
It is characterized by comprising.

  According to the present invention, after the medical practice, the auxiliary information is obtained based on the state change in the region including the surgeon and the assistant so that the time series image group of the medical practice can be edited without taking the user as much as possible. Medical action recording system and medical action record capable of making medical action time series images easy to handle by creating and semi-automatically tagging medical action time series images as an index An apparatus can be provided.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[First Embodiment]
FIG. 1A is a diagram showing a configuration of a medical practice recording system according to the first embodiment of the present invention.

  The medical practice recording system according to the present embodiment records a medical practice of an operator at a medical site, for example, a surgical operation in an operating room. This medical practice recording system includes a camera 11 that captures the treatment of a surgeon 1 (doctor) who performs a medical practice, a moving picture of the medical practice photographed by the camera 11, and predetermined assistance according to instructions from the outside. Between a recording unit 12 (video tape recorder, hard disk recorder, or DVD recorder) configured to record information, and the surgeon 1 and an assistant 2 (surgical staff such as a nurse) who assists the surgeon. Is detected in the detection area, and the auxiliary information is created based on the detection result of the detection section 13 and the detection section 13 detects the change, and the recording of the auxiliary information is recorded in the recording section 12. And a control unit 14 for instructing.

  Here, the moving image captured by the camera 11 may be a time-series image group in which intervals between images acquired in time series are more sparse or irregular.

  In such a medical practice recording system, processing is performed according to the processing procedure shown in FIG.

  That is, first, a medical action performed by the operator 1 is photographed by the camera 11 that photographs the treatment of the surgeon 1 performing the medical action, and the photographed moving image is transmitted to the recording unit 12 and recorded (step). S10). Although not specifically shown, an image obtained with a surgical microscope may also be recorded. In this case, the camera moving image and the surgical microscope image are recorded in the recording unit 12 by the same time chart. However, if a synchronization signal serving as a trigger is added to each information, the time record is not attached. I do not care.

  Next, the state change 3 between the surgeon 1 and the assistant 2 is detected by the detection unit 13 that includes the area between the surgeon 1 and the assistant 2 in the detection region and detects the change in the state in the region. Then, the detection result is transmitted to the control unit 14 as detection information (step S20).

  The control unit 14 receives this detection information and creates detectable auxiliary information based on the detection information (step S30). Then, the recording unit 12 is instructed to record the created auxiliary information (step S40).

  Hereinafter, a specific implementation method will be described.

  In consideration of general surgery, the surgeon 1 mainly performs treatment on the patient 4 during the operation, and the treatment tool 5 is used at that time. In view of the sterilization environment in the operating room, as shown in FIG. 2 (B), the operator 1 and the assistant 2 usually wear sterilized surgical clothes 6 and gloves 7 and the treatment tool 5 is sterilized. It is placed on a cloth drape (treatment device storage 8). Usually, the surgical gown 6, gloves 7 and treatment tool storage 8 are each of a predetermined color (the surgical gown 6 is blue, the glove 7 is skin color, the treatment tool storage 8 is green, etc.). Image extraction processing based on color information can be easily performed.

  The surgeon 1 selects the treatment tool 5 directly from the treatment tool place 8, or the surgeon 1 gives an instruction to the assistant 2 and is handed over from the assistant 2 to the surgeon 1. In this embodiment, paying attention to this point, paying attention to the movements of the treatment tool 5 and the treatment tool place 8 surrounding the treatment tool, the surgeon 1 and the assistant 2 who performs the treatment tool removal, the surgeon recorded as an image. The movement 3 of the hand and the state change 3 relating to the treatment instrument 5 are detected by image processing from the area between 1 and the assistant 2.

  In other words, when the treatment instrument 5 is aligned with the treatment instrument storage place 8, there are exceptions to the procedure performed by the surgeon 1 and the assistant 2 over the treatment instrument 5, but the next procedure has a large proportion. Occupy.

    Procedure 1. The surgeon 1 verbally indicates the necessary treatment tool 5.

    Procedure 2. The assistant 2 selects an appropriate treatment tool 5 from the treatment tool place 8.

    Procedure 3. The assistant 2 gives the treatment tool 5 to the surgeon 1.

    Procedure 4. The surgeon 1 performs treatment with the delivered treatment tool 5.

    Procedure 5. Return the treatment tool 5 to the assistant 2 and Return to.

  In order to estimate the treatment of the surgeon 1 from the images recorded between the surgeon 1 and the assistant 2, the procedure 1. ~ Procedure 5 In the above flow, the above procedure 3. ~ Procedure 5 It is only necessary to specify that the surgeon 1 has the treatment tool 5 during the period, that is, the treatment by the surgeon 1 is being performed. To determine whether the treatment is being performed, the above procedure 3. And procedure 5. What is necessary is just to determine that has been performed.

  In order to make these determinations, in the medical practice recording system, the detection unit 13 includes the camera 131 and the state change detection device 132. First, a detection area between the operator 1 and the assistant 2 is detected by the camera 131. Take a picture. The state change detection device 132 detects a change 3 in the state of the treatment tool 5, the hands of the surgeon 1 and the assistant 2, the treatment tool place 8, and the like from the moving image of the detection area captured by the camera 131. Information is transmitted to the control unit 14. Then, the control unit 14 determines that the treatment tool 5 has the procedure 1. To Step 5. When the treatment is performed, that is, in the above procedure 3. Or procedure 5. When this is done, auxiliary information that can be detected is created.

  In the present embodiment, the auxiliary information that can be detected is created by the procedure as described above. Details of these algorithms will be described below.

  In step S20, specifically, a change in state is detected based on an algorithm as shown in FIG.

  That is, first, an image of the operator 1 and the assistant 2 is captured by the camera 131 of the detection unit 13, and the captured moving image is input to the state change detection device 132 (step S201).

  The state change detection device 132 extracts the center of gravity position of an area (hereinafter referred to as a hand area) in which the hand is assumed to exist in steps S202 to S205, and performs a process in steps S206 to S209 for the input current frame image. Extract the storage area. Here, the state change detection device 132 includes a plurality of processors, and it is preferable to execute the processing in steps S202 to S205 and the processing in steps S206 to S209 in parallel for speeding up the processing. Of course, the processing in steps S202 to S205 and the processing in steps S206 to S209 may be performed sequentially instead of in parallel.

  The number of cameras 131 as a component of the detection unit 13 is not limited to one, and a plurality of cameras 131 may be provided. In the case where a plurality of cameras are provided, it is possible to suppress the decrease in the detection rate due to shielding and perform detection more accurately.

  In the detection processing of the center of gravity position of the hand region, first, the hand region is determined by performing processing (step S203) for determining based on color information based on segmentation (step S202) based on edge information for the input image. . Specifically, the following procedure is followed.

  That is, in the segmentation in step S203, for example, the input current frame image is subjected to an edge extraction filter having sensitivity to a vertical straight line or curve as shown in FIG. 2C, and FIG. ) Apply a sensitive edge extraction filter to the horizontal straight line or curve as shown in Fig. 2), combine the edges obtained from each to form the boundary line of the area, and divide it into the area surrounded by the boundary line The region may be divided by the method.

  After such segmentation, in step S203, a histogram for discrimination based on color information is created for the current frame image. First, the obtained current frame image is converted from RGB to rg space by the following conversion formula.

r = R / (R + G + B), g = G / (R + G + B)
By this conversion, it is possible to cope with a change in brightness due to a change in illumination in the operating room.

  Then, a two-dimensional rg histogram is created for the converted r and g, and the color of the hand (glove 7 worn) of the surgeon 1 who performs the treatment is designated, and the color (in this case, skin color) is possessed. Extract only objects. For example, as shown in FIG. 4 (A), if an rg plane is divided into several parts and an image having no pixel value is created except for the hand region 71 designated in advance, the hand (gloves) 7) An image extracted only as an object is created. Alternatively, the hand region 72 may be defined as shown in FIG. 4B by using a generally known k-NN (k nearest neighbor) method.

  Using the result of the color histogram processing performed on the image thus segmented, the area of the designated color is determined by determining the area in which the designated color occupies a certain pixel or more among the divided areas ( Step S204).

  Next, the center of gravity is obtained for the determined hand region, and the position of the center of gravity is recorded as current frame hand region information in a recording device (not shown) provided in the state change detecting device 132 (step S205). . In calculating the center of gravity of the hand region, a weighted average may be used for the coordinate value of each pixel.

  Similarly, also in the detection process of the center of gravity position of the treatment instrument storage place 8, first, by performing a process (step S207) for determining based on color information based on the segmentation (step S206) based on the edge information with respect to the input image. Determine the storage area. However, in the process of discriminating based on the color information, the color of the drape / cloth drape (treatment tool storage 8) occupying a large position as the background color is designated, and only the object having that color (green) is extracted. For example, as shown in FIG. 4A, the rg plane is divided into several parts, leaving only a color area (treatment tool place area 81) corresponding to the treatment tool place 8 designated in advance, and the others are pixels. If an image having no value is created, an image in which only the treatment tool storage 8 is extracted as an object is created. Or you may prescribe | regulate the treatment tool storage area | region 82 like FIG.4 (B) using k-NN method.

  Using the result of the color histogram processing performed on the image thus segmented, the area of the designated color is determined by determining the area in which the designated color occupies a certain pixel or more among the divided areas ( Step S208). Then, position information is calculated for the determined treatment tool storage area, and this is recorded as a current frame treatment tool storage area information in a recording device (not shown) (step S209).

  Next, using the current frame and previous frame hand area information and treatment tool storage area information calculated and recorded in a recording device (not shown), the current frame hand area information and treatment tool storage area information are obtained. Compared to the hand frame information and the treatment tool storage area information of the previous frame, `` There is a change in the treatment tool storage area from the previous frame, the hand tool center of gravity is in and out of the treatment tool storage area, and the hand area gravity center It is determined whether or not “has changed more than a certain amount” (step S210). Here, if it is determined that there is a change in the treatment tool storage area, that there is a change from the previous frame in the treatment tool storage area, and that the hands of the operator 1 and the assistant 2 are moving more than a certain amount, There is a high possibility that the assistant 2 has started the operation of picking up the treatment tool 5 from the treatment tool place and passing it to the operator 1. That is, it is possible to detect a trigger for estimating that the surgeon 1 performs a treatment after a certain time from this operation. In this case, it is estimated that treatment is performed, and the state change detection device 132 transmits detection information to the control unit 14.

  FIG. 4C shows a processing procedure performed by the control unit 14 that has received the detection information in step S30.

  That is, the control unit 14 receives detection information from the state change detection device 132 of the detection unit 13 (step S301), and creates detectable auxiliary information based on the received detection information (step S302). Thereafter, the recording unit 12 is instructed to add the detected auxiliary information to the recorded camera moving image and / or surgical microscope image (step S303).

  Alternatively, the control unit 14 may execute a processing procedure as shown in FIG.

  That is, detection information is received from the state change detection device 132 of the detection unit 13 (step S301), and detectable auxiliary information is created based on the received detection information (step S302). Thereafter, the recording unit 12 is instructed to create a detectable auxiliary information table corresponding to the camera moving image and / or the surgical microscope image (step S304).

  As described above, the control unit 14 instructs to record detectable auxiliary information related to the camera moving image and / or the surgical microscope image recorded in the recording unit 12. Thereby, as shown in FIG. 1 (B), auxiliary information is recorded during the operation. That is, it is tagged as an index.

  Finally, after the operation is completed, as shown in FIG. 1 (B), from the information recorded in the recording unit 12, a plurality of frames near the frame to which the auxiliary information is added are extracted, If it is presented to the user together with the auxiliary information, only the important part in the vicinity where the treatment is performed from the long-time operation record image can be presented to the user, and the user can save time and effort to search the important part from the huge operation record image. It becomes possible. In the figure, images A, B, and C represent images with different treatment contents. For example, A is an image performing a bone crushing treatment, B is an image performing a dissection treatment, and C is an image performing a tumor excision treatment.

  Furthermore, if a long-term surgical record image is compressed and recorded only at the important part near the treated area, the compression rate is low, and the other less important part is compressed at a high compression rate. For information, it is possible to compress recorded information in terms of semantic content.

  As described above, according to the present embodiment, the time when the treatment operation was performed during the operation can be automatically recorded by the image processing technique for the moving image captured between the operator 1 and the assistant 2, and the image is taken at the time of the operation. The camera moving image (and / or surgical microscope image) can be handled easily. In the present embodiment, the content of the treatment is not determined, but the user can easily know the content of the treatment from the presented image as shown in FIG.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the present embodiment, the content of the treatment is also determined if necessary, and information indicating the content of the treatment is also recorded as auxiliary information.

  Therefore, in the medical practice recording system according to the present embodiment, the control unit 14 includes a main control unit 141 and a determination unit 142 as shown in FIG. Here, the determination unit 142 determines the type of treatment performed by the operator 1 based on the result detected by the detection unit 13, and the main control unit 141 is based on the determination result of the determination unit 142. Thus, auxiliary information is created, and the recording unit 12 is instructed to record the auxiliary information.

  In the present embodiment, the state change detection device 132 of the detection unit 13 receives information necessary for executing the above determination, that is, the camera 131 in addition to the detection result of the state change 3 as detection information. It shall include the photographed image (or information obtained by processing it).

  FIG. 6 shows the entire processing procedure in this embodiment.

  That is, first, a medical action performed by the operator 1 is photographed by the camera 11 that photographs the treatment of the surgeon 1 performing the medical action, and the photographed moving image is transmitted to the recording unit 12 and recorded (step). S10). Although not specifically shown, an image obtained with a surgical microscope may also be recorded.

  Here, the moving image captured by the camera 11 may be a time-series image group in which intervals between images acquired in time series are more sparse or irregular.

  Next, the state change 3 between the surgeon 1 and the assistant 2 is detected by the detection unit 13 that includes the area between the surgeon 1 and the assistant 2 in the detection region and detects the change in the state in the region. Then, the detection result and the captured image of the camera 131 are transmitted as detection information to the control unit 14 (step S21).

  The main control unit 141 in the control unit 14 receives this detection information (step S31). Then, it is determined whether or not more detailed judgment regarding the treatment of the surgeon 1 is necessary (step S32). This is due to, for example, user mode selection.

  Here, when it is determined that more detailed determination is not necessary, the main control unit 141 detects detectable auxiliary information based on the detection information (the detection result of the state change 3), as in the first embodiment. Is created (step S33), and the recording unit 12 is instructed to record the created auxiliary information that can be detected (step S41).

  On the other hand, if it is determined that more detailed determination is necessary, first, the main control unit 141 transmits the received detection information to the determination unit 142 (step S34). Based on the detection information received from the main control unit 141, the determination unit 142 makes a more detailed determination regarding treatment (step S35), and transmits the determination result to the main control unit 141 (step S36). Then, the main control unit 141 creates detectable auxiliary information based on the received determination result (step S37), and instructs the recording unit 12 to record the created detectable auxiliary information (step S41). .

  The determination unit 142 makes a more complicated determination regarding the treatment than the state change detection device 132 of the detection unit 13. Specifically, for example, a procedure as shown in FIG.

  That is, the determination unit 142 receives the detection information from the main control unit 141 (step S351), and determines the type of treatment based on the hand movements of the operator 1 and the assistant 2 as the detection information and the characteristics of the treatment tool 5. (Step S352). Here, the movement of the hand when the treatment tool 5 is handled is different for each treatment tool 5, and if the movement of the hand unique to the treatment tool 5 is determined, the treatment tool 5 being handled can be estimated.

  FIGS. 8A and 8B show how the hand movement differs for each treatment instrument 5. FIG. 8A is a camera image in which the delivery of the treatment instrument 5 (surgical drill 51) used in surgery to destroy the bone is taken, and FIG. 8B is a diagram of coagulating the tissue. It is the camera image which image | photographed the mode that the treatment tool 5 (the tweezers-shaped instrument 52 for coagulation | solidification) in the shape of the tweezers for making it deliver is performed.

  According to this, when the surgical drill 51 is held, the assistant 2 is held by both the left and right hands, and when the assistant 2 hands the tweezers-like instrument 52 for coagulation to the operator 1, the assistant 2 takes the method of changing the instrument lifted with the left hand to the right hand and handing it to the surgeon 1. These movements differ depending on the treatment instrument 5 to be selected. In addition, the number of treatments that can be performed with one treatment tool 5 is often limited to one to several types, and therefore, by specifying the movement of the hand, the treatment tool 5 that the assistant 2 is trying to select is highly accurate. Can be estimated. Therefore, since the treatment tool 5 can be estimated, medical actions such as incision, bone removal, tumor extraction, and suturing that the operator 1 will perform next can be estimated with high probability.

  If the detailed information on the treatment determined by the determination unit 142 is added in addition to the detectable auxiliary information described in the first embodiment, the information is recorded for a long time as shown in FIG. 7B. From the surgical treatment image, it is possible to designate only a portion where a specific treatment has been performed, and it is possible to present information that satisfies the needs of the user.

  Further, if a change in the state between the operator 1 and the assistant 2 is detected in real time by the detection unit 13 during the operation, the type of the treatment instrument 5 is determined in more detail, and then between the operator 1 and the assistant 2 The determination unit 142 determines risk information about the treatment instrument 5 used by the surgeon 1 for medical practice, and sends the determination result to the control unit 14. Further, the recording unit 12 or a separately provided warning device ( By presenting it on (not shown), it is possible to inform the user of the danger associated with the treatment instrument 5 to be used or medical practice. By predicting such a risk, the surgeon 1 can perform a safer operation, and thus has a great advantage that the patient 4 can also receive a safer operation.

  As described above, according to the present embodiment, when and what kind of treatment operation is performed during the operation is automatically recorded by the image processing technique for the moving image captured between the operator 1 and the assistant 2. This makes it easier to handle camera moving images (and / or surgical microscope images) taken during surgery.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention.

  For example, in the above-described embodiment, the surgical treatment image and auxiliary information captured by the camera 11 are simultaneously recorded in the recording unit 12, but the present invention is not limited to this. That is, during the operation, the surgical treatment image captured by the camera 11 and the image of the detection area captured by the camera 131 in the detection unit 13 are recorded in the recording unit 12, and after the operation, the control unit 14 (main control unit) 141 and the determination unit 142) and the state change detection device 132 of the detection unit 13 create the auxiliary information as described above from the image of the detection area recorded in the recording unit 12 and record it in the recording unit 12. You may make it record in the corresponding position of a surgical treatment image, or may record the table which recorded the correlation.

(Appendix)
The invention having the following configuration can be extracted from the specific embodiment.

(1) In the medical practice recording system that records the medical practice of the surgeon in the medical field,
A camera that captures the treatment of the surgeon performing the medical procedure;
A recording unit configured to record predetermined auxiliary information in response to an instruction from the outside, together with a time series image group of medical actions taken by the camera,
A detection means for detecting a change in a state in the detection area, including a surgeon and an assistant who assists the surgeon in the detection area;
Control means for creating the auxiliary information based on the detection result of the detection means and instructing the recording means to record the auxiliary information;
A medical practice recording system comprising:

(Corresponding embodiment)
The embodiment relating to the medical practice recording system described in (1) corresponds to the first and second embodiments. Here, the camera corresponds to the camera 11 in the first and second embodiments, the recording means corresponds to the recording unit 12 in the first and second embodiments, and the detection means corresponds to the first and second embodiments. In the second embodiment, the detection unit 13 corresponds, and the control unit corresponds to the control unit 14 in the first and second embodiments.

(Function and effect)
According to the medical practice recording system described in (1), it is possible to automatically tag time-series images (moving images) at the time of medical practice over a long period of time. It becomes simple.

  (2) The medical practice recording system according to (1), wherein the medical practice in the medical field is a surgical operation in an operating room.

(Corresponding embodiment)
The first and second embodiments correspond to the embodiment relating to the medical practice recording system described in (2).

(Function and effect)
According to the medical practice recording system described in (2), the time series image group (moving images) after the operation is extremely managed especially when used in an operating room where a long-term recording is likely to be performed. It becomes simple and brings a big effect.

  (3) The medical practice recording system according to (1), wherein the detection means includes at least one camera different from a camera that photographs the surgeon's treatment.

(Corresponding embodiment)
The first and second embodiments correspond to the embodiment relating to the medical practice recording system described in (3). Here, the at least one camera corresponds to the camera 131 in the first and second embodiments.

(Function and effect)
According to the medical practice recording system described in (3), since a camera is used as the detecting means, it is possible to tag information that is optimal for the situation as an index by image processing.

  If a plurality of cameras are used, it becomes possible for another camera to cope with a portion where shielding has occurred with one camera. In addition, more information can be tagged by checking with a plurality of cameras, and more precise tagging can be performed.

  (4) The medical action recording system according to (1), wherein the detection means detects a change in a state related to a medical instrument used by the operator for the medical action.

(Corresponding embodiment)
The first and second embodiments correspond to the embodiment relating to the medical practice recording system described in (4). Here, the medical instrument corresponds to the treatment instrument 5 in the first and second embodiments.

(Function and effect)
According to the medical practice recording system described in (4), more information can be obtained by image processing by paying attention to changes in the situation regarding the treatment tool, and as a result, treatment can be estimated in more detail and reliably. It becomes possible.

  (5) The medical practice recording system according to (1), wherein the detection means detects a change in a state related to movement of the hand of the surgeon or the assistant between the surgeon and the assistant.

(Corresponding embodiment)
The embodiment relating to the medical practice recording system described in (5) corresponds to the first and second embodiments.

(Function and effect)
According to the medical practice recording system described in (5), it is possible to cope with the case where the treatment tool cannot be extracted by focusing on the movement of the hand, not the treatment tool. Further, more detailed and reliable estimation can be performed.

  (6) The detection means detects a change in a state related to a medical instrument used by the surgeon for medical treatment and a change in a state related to the movement of the surgeon or the assistant's hand between the surgeon and the assistant. The medical practice recording system according to (1), wherein the medical practice recording system is detected.

(Corresponding embodiment)
The first and second embodiments correspond to the embodiment related to the medical practice recording system described in (6).

(Function and effect)
According to the medical practice recording system described in (6), the amount of information increases by using both the treatment tool and the hand movement, and more accurate estimation is possible.

(7) The control means includes
Based on the detection result of the detection means, a determination means for determining the type of treatment performed by the operator;
Main control means for creating auxiliary information based on the determination result of the determination means and instructing the recording means to record the auxiliary information;
(1) The medical practice recording system according to (1).

(Corresponding embodiment)
The second embodiment corresponds to the embodiment relating to the medical practice recording system described in (7). Here, the determination unit corresponds to the determination unit 142 in the second embodiment, and the main control unit corresponds to the main control unit 141 in the second embodiment.

(Function and effect)
According to the medical practice recording system described in (7), the judgment of more advanced medical practice can be performed by having the judging means.

  (8) The medical practice recording system according to (7), wherein the judging means includes risk judging means for predicting a type of medical practice performed by an operator or an assistant and judging the risk. .

(Corresponding embodiment)
The second embodiment corresponds to the embodiment related to the medical practice recording system described in (8).

(Function and effect)
According to the medical practice recording system described in (8), it is possible to notify the user of the risk associated with the treatment instrument to be used and the medical practice. By predicting such a risk, the surgeon can perform a safer operation, and thus can perform a safer operation for the patient.

(9) In the medical practice recording device that records the medical practice of the surgeon in the medical field,
A detection means for detecting a change in a state in the detection region, including a region between a surgeon performing a medical practice and an assistant assisting the operator in a detection region;
The auxiliary information is created based on the detection result of the detection means, and the auxiliary information is recorded in association with a time-series image group obtained by photographing the surgeon's treatment;
A medical practice recording apparatus comprising:

(Corresponding embodiment)
The first and second embodiments correspond to the embodiment relating to the medical practice recording apparatus described in (9). Here, the detection means corresponds to the detection unit 13 in the first and second embodiments, and the recording means corresponds to the recording unit 12 and the control unit 14 in the first and second embodiments.

(Function and effect)
According to the medical practice recording apparatus described in (9), it is possible to automatically tag time-series images (moving images) at the time of medical practice over a long period of time, and the use of images after medical practice is very It becomes simple.

(A) is a figure which shows the structure of the medical practice recording system which concerns on 1st Embodiment of this invention, (B) is a figure which shows the recording position of the auxiliary information on a time-axis. (A) is a figure which shows the flowchart of the processing operation of the whole medical practice recording system which concerns on 1st Embodiment, (B) is a figure which shows the example of the image image | photographed with the camera of the detection part, (C) and (D ) Are diagrams illustrating edge extraction filters used for segmentation processing. It is a figure which shows the operation | movement flowchart of the state change detection apparatus of a detection part. (A) And (B) is a figure which shows the color of the hand area | region in the rg plane, and the color of a treatment tool storage place, respectively, (C) And (D) is a figure which respectively shows the operation | movement flowchart of a control part. . It is a figure which shows the structure of the medical practice recording system which concerns on 2nd Embodiment of this invention. It is a figure which shows the flowchart of the processing operation of the whole medical practice recording system which concerns on 2nd Embodiment. (A) is a figure which shows the operation | movement flowchart of a judgment part, (B) is a figure which shows the recording position of the auxiliary information on a time-axis, and the treatment content at that time. (A) is a figure which shows a series of camera images which image | photographed the state which is delivering the surgical drill, (B) is a series of camera images which imaged the state which is delivering the tweezer-shaped instrument for coagulation | solidification. FIG.

Explanation of symbols

    DESCRIPTION OF SYMBOLS 1 ... Surgeon, 2 ... Assistant, 3 ... Change of state, 4 ... Patient, 5 ... Treatment tool, 6 ... Surgical clothes, 7 ... Gloves, 8 ... Treatment tool place, 11, 131 ... Camera, 12 ... Recording part , 13 ... detection unit, 14 ... control unit, 51 ... surgical drill, 52 ... coagulant tweezers, 71, 72 ... hand region, 81, 82 ... treatment instrument storage region, 132 ... state change detection device, 141 ... Main control unit, 142... Judgment unit.

Claims (9)

In the medical practice recording system that records the medical practice of the surgeon in the medical field,
A camera that captures the treatment of the surgeon performing the medical procedure;
A recording unit configured to record predetermined auxiliary information in response to an instruction from the outside, together with a time series image group of medical actions taken by the camera,
A detection means for detecting a change in a state in the detection area, including a surgeon and an assistant who assists the surgeon in the detection area;
Control means for creating the auxiliary information based on the detection result of the detection means and instructing the recording means to record the auxiliary information;
A medical practice recording system comprising:   The medical practice recording system according to claim 1, wherein the medical practice in the medical field is a surgical operation in an operating room.   The medical action recording system according to claim 1, wherein the detection unit includes at least one camera different from a camera that photographs the treatment of the surgeon.   The medical action recording system according to claim 1, wherein the detecting means detects a change in a state related to a medical instrument used by the surgeon for the medical action.   The medical action recording system according to claim 1, wherein the detection unit detects a change in a state related to the movement of the hand of the surgeon or the assistant between the surgeon and the assistant.   The detection means detects a change in a state related to a medical instrument used by the surgeon for medical practice and a change in a state related to the movement of the surgeon or the assistant's hand between the surgeon and the assistant. The medical practice recording system according to claim 1. The control means includes
Based on the detection result of the detection means, a determination means for determining the type of treatment performed by the operator;
Main control means for creating auxiliary information based on the determination result of the determination means and instructing the recording means to record the auxiliary information;
The medical practice recording system according to claim 1, comprising:   8. The medical practice recording system according to claim 7, wherein the judging means includes risk judging means for predicting a type of medical practice performed by an operator or an assistant and judging the risk. In the medical practice recording device that records the medical practice of the surgeon in the medical field,
A detection means for detecting a change in a state in the detection region, including a region between a surgeon performing a medical practice and an assistant assisting the operator in a detection region;
The auxiliary information is created based on the detection result of the detection means, and the auxiliary information is recorded in association with a time-series image group obtained by photographing the surgeon's treatment;
A medical practice recording apparatus comprising:

Images