JP2000138927A - Image comparative display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simply compare and display moving pictures and data of a rehabilitational work training in an easily understandable way. SOLUTION: This image comparative display device is provided with a image pickup section 14 that stores moving pictures of a person 28 to be the object of photographing by a photographing means 12 and exercising rehabilitational walk training in a storage section 16, a picture selection means that selects two moving pictures to be compared from the storage section 16, and a processing section 20 that reads the two moving pictures selected by the picture selection means from the storage section 16 and synthesizes the two moving pictures to be displayed on a display section 22 simultaneously. Thus, a plurality of walk training moving pictures or the like after the rehabilitational walk training are simply compared and displayed in an easily understandable way and the degree of recovery by the rehabilitational walk training is compared and displayed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image comparison and display apparatus using image processing, and more particularly to a trajectory of a reference position in a subject's body calculated from a plurality of sets of moving images or a plurality of sets of moving image data. The present invention relates to an image comparison display device that displays a plurality of combinations.

[0002]

2. Description of the Related Art In rehabilitation at a medical site, a training plan is prepared before conducting training in order to provide training suitable for a patient's disease symptoms. During the training period, a motion analysis, a visual inspection, and the like are performed, the training plan is compared with the training plan, and the training plan is updated according to the degree of recovery.

[0003] For example, the analysis of the walking motion of a rehabilitation patient is as follows.
Generally, measurement is performed by two-dimensional or three-dimensional non-contact measurement by image processing, and analysis of displacement of a joint position, walking speed, step length, and the like is performed. Then, the results are stored in a personal database, graphed, and the degree of recovery of the patient is determined based on how the numerical values change. Inspections include facial expressions of the patient during training,
This is a method of evaluating the overall movement and evaluating it based on the experience and intuition of a physiotherapist. At this time, a moving image may be left with an 8 mm video or digital camera.

[0004]

However, in the case of a moving image photographed by an 8 mm video or the like used for visual inspection, only one moving image can be displayed at a time, so that the symptoms of the patient photographed in the past and the symptoms of the recent patient are displayed. It was hard to judge how much he had recovered by comparing with. Also, if a method of displaying on a personal computer using a digital camera, it is possible to simultaneously display a plurality of still images, but in this case, special operations are required for the operation for displaying, and media There is a problem that only a limited number of people can operate the device since setting is difficult from display to display.

A dedicated CC is used for analyzing a walking motion and the like.
Using different equipment depending on the purpose, such as using a D camera or personal computer and using an 8 mm video camera to acquire moving images, comparing the results of walking motion analysis with moving images Is very time-consuming and complicated.

The present invention has been made in view of such circumstances, and uses the same database as the result of analysis of walking motion and the like, in which moving image data taken by an imaging device for visual inspection of walking motion and the like after rehabilitation walking training is obtained. It is an object of the present invention to provide an image comparison and display device capable of performing comparison and display in a simple and easy-to-understand manner by managing and synthesizing an arbitrary moving image together with a graph of an analysis result.

[0007]

According to the present invention, in order to achieve the above-mentioned object, a comparison is made between an image capturing means for storing a moving image of a person to be photographed photographed by a photographing means in a memory means and the memory means. Image selecting means for selecting two power images to be output, and image processing means for reading out the two moving images selected by the image selecting means from the storage means and synthesizing the two moving images so as to be simultaneously displayed on the display means And, it is characterized by having.

According to the present invention, an image capturing means for storing a moving image of a person to be photographed photographed by a photographing means in a memory means, and an image selecting means for selecting two moving images to be compared from the memory means Reading out the two moving images selected by the image selecting means from the storage means;
Image processing means for synthesizing the two moving images so as to be simultaneously displayed on the display means, so that a plurality of walking moving images and the like after the rehabilitation walking training can be displayed in a simple and easy-to-understand manner.

In order to achieve the above object, the invention according to claim 2 is characterized in that display start times of the two moving images can be set independently and freely. According to the present invention, since the display start times of the two moving images can be freely set independently, it is possible to easily compare and display a plurality of walking moving images after rehabilitation walking training.

According to a third aspect of the present invention, in order to achieve the above object, the operation of the moving image is analyzed for each moving image, and the analysis result is linked to the moving image and stored in the storage means. Means for causing the image processing means to execute
An analysis result corresponding to one moving image is read out from the storage unit, and the analysis result is combined with the two moving images so as to be displayed on a display unit.

According to the present invention, there is provided means for analyzing the operation of the moving image for each moving image, linking the analysis result to the moving image and storing the result in the storage means, wherein the image processing means The analysis results corresponding to the two moving images are read out from the storage unit, and the analysis results are combined so as to be displayed on the display unit together with the two moving images. Comparison display can be performed easily and easily.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which an image comparison and display apparatus according to the present invention is applied to rehabilitation walking training for medical use will be described below with reference to the drawings. FIG. 1 is a configuration diagram showing a configuration of a rehabilitation image comparison and display device 10 according to the present invention, and FIG. 2 is a diagram showing a shooting state of the rehabilitation image comparison and display device 10 based on the configuration of FIG. .

The rehabilitation image comparison and display device 10 shown in FIG. 1 includes a photographing means 12 such as a video camera for photographing a state of rehabilitation walking training, and an image capturing section 14 for digitally encoding and capturing a captured moving image. A storage unit 16 for compressing the captured moving image, shooting conditions at the time of shooting, analysis result data of walking motion analysis, and the like as necessary, and storing the compressed moving image, and a compressed image stored in the storage unit 16. An image processing unit 18 that calls the converted moving image and converts it into still image data, converts it into joint position information and the like, displays the skeleton, analyzes the walking motion, and the like, and the storage unit 1
6, a processing unit 20 that calls up the compressed moving image stored in the storage unit 6 and expands the data into a displayable form, and an analysis result such as a walking motion analysis stored in the storage unit 16 and the processing unit 20 combines the processing results. A display unit 22 for displaying a processed moving image; a control unit 2 for controlling the image capturing unit 14, the image processing unit 18, the processing unit 20, and the display unit 22
4 and an input unit 26 for inputting control instructions such as processing contents and image display to the control unit 24. The input unit 26 is composed of a keyboard, a mouse, and the like, and the command is input to the control unit 24 to give an instruction such as selection of a processing function and simultaneous reproduction, pause, and frame advance of a moving image on two screens. A photographing signal obtained by the photographing unit 12 is input to the image capturing unit 14 to photograph the photographed person 28. Depending on the front and rear positions of the photographing unit 12, the enlargement ratio of the photographed person 28 in the photographed screen is determined. Since the walking speed changes, a scale 29 is installed near the subject 28. Since the image of the scale 29 is used to calibrate the scale of the data to be analyzed with reference to the image processing performed later, the scale 29 need not be permanently installed but only installed at the first photographing. May be removed later. Further, in the example of FIG. 2, lighting stands 30, 30 for illuminating the photographing subject 28 at the time of photographing are provided.

Next, a procedure from photographing to data storage will be described. The operation power is turned on, the lighting device 30 is turned on to prepare the photographing equipment, and the operator instructs the input unit 26 to capture a moving image. If it is necessary to attach a marker to the person to be photographed 28 in order to calculate the movement trajectory during the walking motion analysis, a marker (not shown) is attached to a predetermined portion as necessary. When an operator inputs an instruction to capture a moving image to the input unit 26, the moving image captured by the imaging unit 12 is transferred to the image capturing unit 14, where it is digitally encoded and stored in the storage unit 16 as a moving image. At the same time, information such as a file name and a shooting date of the captured moving image is written in a database provided in the storage unit 16 in advance.

FIG. 3 shows a flowchart of a moving image display method according to the present invention, and FIG. 4 shows a state in which a moving image is displayed on the display unit 22 according to the flowchart of FIG. In step 52, when the operator inputs the ID of the person to be imaged 28 and the instruction of the walking motion analysis to the input unit 26, the input unit 26 searches the database of the storage unit 16 by the control unit 24 and specifies the specified moving image. Is issued (step 54). The obtained moving image is transferred to the image processing unit 18 as needed, and the walking motion is analyzed. The analysis result is stored in the database of the storage unit 16 again. For example, when there is an instruction to display moving images on two screens at the same time, the operator also inputs two moving image file names to be compared and displayed to the input unit 26 as shown in step 52 of FIG. The procedure for displaying the stored data is as follows: an operator inputs a command to the input unit 26 to search the database of the storage unit 16 for two corresponding moving images,
After the two moving images are transferred to 0 and processed, the first frame of each moving image is displayed on the screen of the display unit 22 (step 58). After receiving an instruction such as reproduction, pause, or frame advance from the input unit 26 for one of the two moving images displayed in this manner, the control unit 2
No. 4 shifts and stops a frame of one moving image until the start frame is convenient for control and comparison (step 60). When the frame of one moving image is determined, the start frame is determined for the other moving image by the same operation. When the start frames of the two moving images are determined (step 56), the operator uses the input unit 26 to input a command for simultaneous reproduction, pause, frame advance, etc. while viewing the display unit 22. When the control unit 24 receives the instruction, in step 62, the compressed frame data after the currently displayed start frame of the two moving images is read one by one, transferred to the processing unit 20, and decompressed. In step 64, the two moving image frames are combined into one in the processing unit 20, and then output to the display unit 22 for display. In the case of continuous playback, in step 66, the process of taking out the next frames of the moving images of the two screens one by one, sending them to the processing unit 20, synthesizing them, and displaying them on the display unit 22 is continuously repeated. Simultaneous control of two moving images is sequentially performed.

Although the above-mentioned display content is a display of only a captured moving image, in the training of rehabilitation and the like, the degree of recovery, the degree of improvement, the degree of achievement with respect to the recovery target, and the like are displayed as numerical values or graphs. This will help increase the patient's willingness to train. Hereinafter, an embodiment according to the present invention will be described in which a captured moving image is displayed after analyzing a walking motion.

The moving image data stored in the storage unit 16 is transferred to the image processing unit 18. When a marker indicating a measurement point is attached to the person to be imaged 28, a marker portion is extracted from the data of the moving image to determine the area centroid of each marker. Measurement point. The measurement point of the marker is calculated for each captured frame and stored in the storage unit 16 as a measurement point (called a locus) at a certain time. When the marker is not used for the person 28 to be photographed, the position of the reference joint of the body of the person 28 to be photographed is cut out from the image of the whole body as a part from the photographed moving image data. By performing the normalized correlation and calculating the joint position coordinates, the joint position is stored in the storage unit 16 as a temporal joint position trajectory. When the trajectory is obtained as described above, the displacement and speed between the frames of each joint, and the stance period, which is the time and time when the foot is in contact with the floor, are calculated from the obtained trajectory and stored in the storage unit 16. .

FIG. 5 shows an example in which the trajectory obtained as described above is displayed as a skeleton image 72. FIG. 5 shows a comparison of the state of walking with two moving images 70 arbitrarily different in date from the same patient and a comparison of the state of walking with two skeleton images 72 processed based on the original image. . The skeleton image 72 has an advantage that the movement of the body skeleton is easy to see because there is no extra information as compared with the moving image 70 displayed at the same time. Also, there is an advantage that the amount of data is overwhelmingly smaller than that of the moving image 70. Focusing on the locus of the determined body part of the subject 28, the locus of the waist joint during walking is recovered as a graph as quantitative data as shown in a graph 76 in FIG. 6, for example. It is also possible to accurately examine the degree of the problem. Note that the graph 74 in FIG. 6 shows that the walking speed of the imaging subject 28 (patient) increases day by day as a result of the rehabilitation walking training. Note that, in order to make it easier to see more information, the moving image 70 and the skeleton image 72 of FIG. 5 and the graph shown in FIG. 6 may be combined and displayed on one screen. Also, the data to be displayed and compared may be data of the same patient at different imaging times, or one of them may be compared as data of a healthy subject.

In FIG. 5, two images to be compared are displayed side by side. However, the present invention is not limited to this, and may be arranged vertically, and processing such as changing display colors may be performed. May be superimposed and displayed. FIG. 7 is an example in which a result of comprehensive judgment based on data obtained by analyzing a captured moving image is displayed. According to FIG.
The patient's medical record information such as name, ID, symptoms, etc., analysis data or graphs such as walking speed, stride length, pace, left / right overlapping pace, displacement between joint positions, etc. On the other hand, a graph showing the degree of recovery indicating the percentage recovered, a display of the degree of achievement of the recovery target, evaluation and advice, and the like are displayed on one screen, so that it is easy to understand, which helps to increase the motivation of the patient for rehabilitation.

The medical rehabilitation image comparison and display device 10 described above uses not only a specialized doctor and a physiotherapist but also an operation method such as a graphic user interface on the input unit 26 and the display unit 22. By designing the system so that it can be operated by general patients and devising the processes from preparation and imaging to display quickly, it is possible to make use of a wider range of rehabilitation.

In the above embodiment, the embodiment in which the image comparison and display apparatus according to the present invention is applied to rehabilitation walking training for medical use has been described. However, the present invention is not limited to this. The present invention can also be applied to the purpose of observing and supporting progress in other sports and occupations, such as swinging and swinging a baseball hit, and the results of practice.

In the embodiment of the present invention described above, the image capturing section 14 and the processing section 20 are described as being independent from each other in FIG. 1. However, the present invention is not limited to this, and the processing is performed inside the personal computer. May be.

[0023]

As described above, according to the present invention, an image capturing means for storing a moving image of a person to be photographed photographed by a photographing means in a memory means, and two moving images to be compared from the memory means. Image selecting means for selecting an image, and image processing means for reading out the two moving images selected by the image selecting means from the storage means and synthesizing the two moving images so as to be simultaneously displayed on the display means. Therefore, a plurality of walking moving images and the like after rehabilitation walking training can be comparatively displayed in a simple and easy-to-understand manner.

According to another aspect of the present invention, the display start times of the two moving images can be freely set independently, so that a plurality of walking moving images after rehabilitation walking training can be displayed in a comparatively easy-to-understand manner. can do. According to still another aspect of the present invention, the image processing means further comprises means for analyzing an operation of the moving image for each moving image, linking the analysis result to the moving image and storing the result in the storage means, Is the 2
The analysis results corresponding to the two moving images are read out from the storage means, and the analysis results are combined so as to be displayed on the display means together with the two moving images, so that the walking analysis data after the rehabilitation walking training can be easily and easily understood. It can be easily compared and displayed.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an image comparison and display device for rehabilitation according to the present invention.

FIG. 2 is a diagram showing a shooting state by a rehabilitation image comparison device based on the configuration of FIG. 1;

FIG. 3 is a flowchart showing a moving image display method according to the present invention.

FIG. 4 is a diagram showing a display example according to the moving image display method of FIG. 3;

FIG. 5 is a diagram showing an example of a display method of two moving images and two skeleton images according to the present invention.

FIG. 6 is a diagram showing an example in which a locus of a hip joint during walking and recovery of walking speed are displayed.

FIG. 7 is a diagram showing a display example of a result of comprehensively determining a rehabilitation patient based on data obtained by analyzing a moving image according to the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Image comparison display apparatus 12 ... Photographing means 16 ... Storage part 18 ... Image processing part 22 ... Display part 28 ... Person to be photographed

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G09G 5/377 H04N 5/225 C H04N 5/76 A61B 5/10 310Z // H04N 5/225 G09G 5 / 36 520L F-term (Reference) 4C038 VA04 VB35 VC01 VC05 5C022 AA08 AC00 5C052 AA01 AB02 AC01 DD05 5C054 AA05 CA04 CC03 CD03 EA01 EA05 FA01 FC12 FE09 FF03 GA01 GA02 GB02 HA12 5C082 AA42 BA42 BA42 BA42BA42 DA42 DA87 DA89 MM09 MM10

Claims (3)

[Claims] An image capturing unit that stores a moving image of a person to be imaged captured by an imaging unit in a storage unit; an image selection unit that selects two moving images to be compared from the storage unit; An image processing means for reading out the two moving images selected by the selecting means from the storage means and synthesizing the two moving images so as to simultaneously display the two moving images on the display means. 2. The image comparison display device according to claim 1, wherein the display start times of the two moving images can be set freely independently. 3. A means for analyzing the operation of the moving image for each moving image, linking the analysis result to the moving image and storing the result in the storage means, wherein the image processing means
2. The image comparison and display device according to claim 1, wherein analysis results corresponding to the two moving images are read out from the storage unit, and the analysis results are combined so as to be displayed on the display unit together with the two moving images.