JP2013066696A - Image processing system and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system and image processing method suitable for determining, especially, propriety of exercises in a human being.SOLUTION: An image processing system includes: a treadmill 4; an imaging device 2 for imaging the exercise of a user 5, that is to be done on the treadmill 4; an image processor 1 for processing moving image data that are acquired through the imaging device 2 and indicates the exercise of the user 5; and a monitor 3 for displaying an image processing result R acquired by the image processor 1 thereby to enable the user 5 to view the result on the treadmill 4. There are disclosed the image processing system and an image processing method where the image processor 1 displays on the monitor 3, overlap moving image data where inversion moving image data obtained by inverting the right and left of moving image data with one pivot foot between pieces of moving image data from the starts to the ends of exercises with left foot as a pivot and with right foot as a pivot to be acquired from the original moving image data of the imaged past exercises of the user 5 is overlapped with present moving image data to be acquired from the start of exercise with the other pivot foot.

Description

  The present invention relates to an image processing system and an image processing method that are particularly suitable for determining whether or not a human motion is being performed correctly. More specifically, the present invention relates to the suitability of a human walking balance or the accuracy of motion. The present invention relates to an image processing system and an image processing method suitable for gait rehabilitation support or sports training by walking and exercising while directly checking and confirming sex and the like.

For example, when symptoms such as cerebral hemorrhage develop, symptoms of hemiplegia may appear. In that case, it is necessary to perform rehabilitation in order to return to a normal state, and a system capable of performing rehabilitation at home is desired.
In rehabilitation gait training, it is important to correct the gait after correctly understanding the gait while receiving instructions from the physical therapist.
In addition, it is necessary to correctly grasp the state of actual exercise during training so that exercise (sports) such as walking, racewalking, and running can be performed accurately.

  Therefore, an invention of an “image comparison display device” that displays a moving image showing the state of past exercise and a moving image showing the state of current exercise as they are has been proposed (Patent Document 1).

  Conventionally, as an “exercise support system and exercise support method” for correctly executing a preset exercise, a video showing a user's exercise obtained by an imaging means and an exercise to be referred to An invention has been proposed in which a moving image showing the image is displayed side by side on a display screen (Patent Document 2).

  In the invention described in Patent Document 1, although it is possible to compare the movements of the same user, it is not possible to directly compare the movement with a suitable normal operation, and how to correct where. There were issues such as not being able to show users specifically what to do, and not having their own past videos in the first place.

  In addition, in the invention described in Patent Document 2, since a reference pattern of another person is displayed, a difference in height and body shape, a period of exercise, and the like are different, and the actual movement of the user does not correspond accurately. For this reason, there is a fear that a sufficient rehabilitation effect cannot be expected.

Therefore, the present inventors extracted the normal motion part from the original video data that captured the situation of the user walking by “Proposal of Voluntary Rehabilitation Support System Focusing on Right-Left Balance in Gait”. The image processing system which shows a user the existence of bad motion by superimposing with bad motion is proposed (nonpatent literature 1).
However, in the above image processing system, since the user grasps the bad movement by looking at the image processed image after finishing the walking, the user grasps the bad movement directly while walking. There was a problem that it was not possible to correct the bad behavior in real time.

Furthermore, in the above image processing system, there is a limitation in walking space, so that it is impossible to walk for a long distance and for a long time, and the size and position of an image taken by walking change, so that image processing is performed. There is a problem that it takes time.
Moreover, in the conventional walking rehabilitation apparatus, there was a problem that the device for allowing the user to enjoy walking training while maintaining high motivation was not sufficient.

JP 2000-138927 A Japanese Patent No. 3565936

Masaya Umezu et al. “Proposal of voluntary rehabilitation support system focusing on left / right balance in gait”, Proceedings of the IEICE General Conference, IEICE, Information and Systems (2), p. 153, March 2, 2010

The problem to be solved by the present invention is that, depending on the conventional image processing system or image processing method, the user can directly grasp in real time whether the walking balance is appropriate or the accuracy of the exercise operation while walking or exercising. An object of the present invention is to provide an image processing system and an image processing method that enable a user to easily grasp where to correct while walking or exercising happily, and to enhance the rehabilitation effect or It is to be able to acquire appropriate motor movements.
That is, an object of the present invention is to provide an image processing system and an image processing method that are particularly suitable for determining whether or not a human movement is being performed correctly.

  The present invention relates to a "treadmill, a photographing device that photographs a user's movement performed on the treadmill, an image processing device that processes moving image data indicating the user's movement obtained by the photographing device, and An image processing system comprising: a monitor that displays an image processing result obtained by the image processing device so that a user can view the result on the treadmill, wherein the image processing device Inverted video data from either the left foot axis motion or the right foot axis motion from the beginning to the end of the left foot axis motion obtained from the original video data obtained by shooting past motions. An image processing system for displaying superimposed moving image data on a monitor in which moving image data and current moving image data obtained from the start of movement by the other axis foot are superimposed. Is the most important feature.

  The image processing system of the present invention preferably further includes an acoustic device that emits a guide sound for checking the walking rhythm in conjunction with the start of the left foot axis motion and the right foot axis motion.

Furthermore, in the image processing system of the present invention, it is more preferable that the image processing device overlaps the movement of the foot on the side that is not the axial foot of the current moving image data in which the inverted moving image data is half-length moving image data. It is characterized by matching.
Further, the present invention is one in which the image processing device displays the user's past moving image data or current moving image data in a color-coded manner and displays it on a monitor, or is performed beyond the range where the user's motion is allowed. In such a case, a warning for the user may be displayed on the monitor.
The image processing apparatus may display evaluation data indicating a degree of overlap between the inverted moving image data and the other original moving image data corresponding to the inverted moving image data on a monitor.

  The image processing device, when obtaining the moving image data from the start to the end of the left foot axis walking or right foot axis walking from the original moving image data obtained by photographing the user's movement, the image data constituting the original moving image data, Analyzing the exposed part of the skin near the user's ankle, the time when the exposed part of one foot is at the lowest point position is the start of walking with that foot as the axis, and The end time may be the time when the exposed portion of the other foot is at the lowest point position.

If the difference between the required time from the start to the end of the left foot axis walk and the required time from the start to the end of the right foot axis walk exceeds the allowable range, the present invention It may be an image processing system that issues a warning.
Furthermore, the image processing apparatus of the present invention is connected to a remote electronic computer, thereby transmitting superimposed moving image data to the electronic computer or displaying image data from the electronic computer on a monitor. The voice message to the user may be transmitted.

  The present invention relates to a treadmill, a photographing device for photographing a user's movement performed on the treadmill, an image processing device for processing moving image data indicating a user's movement obtained by the photographing device, An image processing method in an image processing system comprising: a monitor that displays an image processing result obtained by the image processing apparatus so that a user can view the result on the treadmill, wherein the image processing apparatus Invert the left and right of the video data for either one of the left foot axis motion or the right foot axis motion from the start to the end of the motion data of the left foot axis obtained from the original video data that captured the user's past motion Superimposed video data is superimposed on the current video data obtained from the start of the movement with the other axis foot on the monitor. It is intended to include an image processing method.

In the image processing method of the present invention, the image processing system further includes an acoustic device, and emits a guide sound for checking the walking rhythm in conjunction with the start of the left foot axis motion and the right foot axis motion. May be. Further, in the image processing method of the present invention, the inverted moving image data and the other moving image data by the image processing device may be superimposed on each other in color, and the image processing device A warning for the user may be displayed on the monitor when the exercise is performed beyond the allowable range.
In the image processing method of the present invention, the image processing apparatus may display evaluation data indicating a degree of overlap between the inverted moving image data and the other original moving image data corresponding to the inverted moving image data on a monitor. Good.
Furthermore, the present invention includes a program for executing the above-described image processing method on an electronic computer.

By adopting the above configuration, the image processing system, the image processing method, and the program for executing the image processing method of the present invention on an electronic computer can be repeatedly walked by taking advantage of a treadmill without taking up a place. It is possible to perform an exercise such as running, and it is possible to perform an exercise while accurately comparing the current state of the user's own exercise in real time while viewing the superimposed moving image data displayed on the monitor.
In addition, since the size and position of the user image on the treadmill are substantially constant, there is an effect that the accuracy and processing speed of the image processing can be improved.

Furthermore, by superimposing the half-length moving image data of the inverted moving image data by the image processing device, the comparison target can be narrowed down and displayed on the monitor, the displayed image is easy to see, and the processing speed is also higher. The effect of further improvement can be obtained.
In addition, when an acoustic device that emits a guide sound for checking the walking rhythm is provided, the user can hear from the ear whether or not his / her walking rhythm is correct by walking according to the guide sound. It can be grasped from the audible guide sound, and the effect as training to acquire correct walking can be enhanced, and the user can enjoy walking training while listening to music.
Therefore, in this case, the user can perform walking training while maintaining high motivation.
In addition, when the image processing apparatus is to display the user's past moving image data or current moving image data in a color-coded manner and display it on the monitor, it is possible to visually compare the movement more clearly and quickly. .

In the present invention, when the user's exercise exceeds the allowable range and the warning for the user is displayed on the monitor, it is possible to easily know the suitability of the exercise range by the user. .
Furthermore, when the evaluation data indicating the degree of overlap between the inverted moving image data and the other original moving image data corresponding to the inverted moving image data is displayed on the monitor, it is possible not only to know the suitability of the exercise accurately. You can also increase your motivation to continue exercise.

  In the present invention, the image processing apparatus analyzes an exposed portion of the skin near the user's ankle, and determines when walking starts and ends based on when the exposed portion of the foot is at the lowest point position. In this case, it becomes possible to superimpose moving image data without attaching or attaching a special marker to the user, and the image processing system of the present invention can be used easily.

In the present invention, when the difference between the required time from the start to the end of the left foot axis walking and the required time from the start to the end of the right foot axis walk exceeds the allowable range, If a warning is issued, it is possible to easily know whether or not the exercise rhythm of the user is appropriate.
Furthermore, in the present invention, by connecting to an electronic computer at a remote place, the superimposed moving image data is transmitted to the electronic computer, the image data from the electronic computer is displayed on the monitor, and the audio to the user is displayed. If the message can be transmitted, the image processing system can be exercised at home by receiving instructions from a supervisor or instructor from a remote location.

When the image processing system of the present invention, the image processing method, and the program for executing the image processing method on an electronic computer are used for a rehabilitating user, the movement of the healthy person is compared symmetrically. Because the physical characteristics are the same, it is easy to refer. In addition, there is an effect that it is easy to understand how the movements are different in order to overlap and display using image processing, and how to move the body.
Furthermore, since the present invention enables the user to walk for a long time while confirming the walking state by using the treadmill, the effect of rehabilitation can be further obtained for a user who is half-body paralyzed or a prosthetic leg. Can do. Further, by using a treadmill, the time required for image processing is reduced, and rehabilitation can be performed while directly comparing the state of motion in real time.

1 is a perspective view showing an image processing system of the present invention. The conceptual diagram which shows the outline | summary of the image processing system of this invention. The figure which shows the monitor screen of the image processing system of this invention. The figure which shows the mode of the image processing about the original moving image data by the image processing system of this invention, and each image data (frame). The figure which shows a mode that the image displayed on the monitor of the image processing system of this invention is produced | generated. The figure which shows a mode that the image displayed on the monitor of the image processing system of this invention is produced | generated. The figure which shows the image displayed on the monitor of the image processing system of this invention. The flowchart which shows the process of the image processing by the image processing method of this invention. The figure which shows the earthing | grounding time and stance time of a right-and-left leg | foot when a user performs walking training. The figure which shows the image displayed on the monitor of the image processing system of this invention. The figure which shows the display on the monitor for selecting the operation mode of the image processing system of this invention.

  The present invention relates to a treadmill, a photographing device for photographing a user's movement performed on the treadmill, an image processing device for processing moving image data indicating a user's movement obtained by the photographing device, An image processing system comprising: a monitor that displays an image processing result obtained by the image processing apparatus so that a user can view the result on the treadmill, wherein the image processing apparatus Of the left foot axis motion or the right foot axis motion from the beginning to the end of the left foot axis motion obtained from the original motion picture data obtained by taking the motion of (2) as a reference image. Image processing for displaying superimposed video data on the monitor, which is a combination of the inverted video data that has been inverted and the current video data obtained from the start of movement by the other axis foot. System, a program for an image processing method and image processing method executed by an electronic computer.

Next, an embodiment in which the image processing system and the image processing method of the present invention are embodied for gait rehabilitation support will be described with reference to the drawings.
As shown in FIGS. 1 and 2, an image processing system according to an embodiment of the present invention includes an electronic computer 1 that functions as an image processing device, a photographing device (video camera) 2, a monitor 3, and the photographing device 2. And a treadmill 4 installed in front of the monitor 3.
The position where the photographing apparatus 2 is installed is preferably a position in front of the treadmill 4 and having a height from the floor of about 80 to 100 cm.
Further, as the monitor 3, in the case of daily rehabilitation at home, it is preferable to use a TV installed at home, but it may be a monitor for an electronic computer (personal computer).

  The electronic computer 1 includes an electronic computer program for embodying the present invention, and performs image processing based on moving image data provided from the photographing apparatus 2, and the image processing result Various information such as R, evaluation data E, number of steps N, and elapsed time (remaining time) T are displayed on the monitor 3, and the user 5 repeatedly walks (exercises) on the treadmill 4 while viewing the information. Be able to.

Here, as the image processing result R, the original motion picture data obtained by photographing the last motion by the user is used as a reference image, and the current motion is photographed with the reversed motion image data obtained by inverting the left and right of the comparison target portion. The superimposed moving image data superimposed on the original moving image data can be suitably used.
This is adopted for the following reasons.
It can be said that the walking movement by a healthy person is the same movement on the left and right sides of the body, and that the same movement at the same timing on the left and right sides of the body is normal walking (see the image frame in FIG. 4A). That is, in the walking motion by a healthy person, walking on the right foot axis and walking on the left foot axis are symmetrical movements on the left and right sides of the body.

On the other hand, the walking of a person with hemiplegia is a movement with one side (for example, walking on the right foot axis) that is less affected by paralysis, but the movement on the other side (for example, walking on the left foot axis) is The movement is bad due to hemiplegia. Since hemiplegic walking alternates between a problem-free movement and a bad movement, the problem-free movement immediately before the bad movement is used as a reference movement, and the difference in movement is recognized by reversing and displaying it. And the effect of rehabilitation appears (see the image frame in FIG. 4B).
In addition, when a person with hemiplegia walks, the time required for walking on the right foot axis and walking on the left foot axis may be out of order.

  In this way, since the movements on the healthy side of the person who performs rehabilitation are compared symmetrically, it is easy to refer to them because the physical characteristics are the same. Further, it is possible to obtain an effect that it is easy to understand how the movements are different in order to superimpose and display using image processing, and how to move the body.

The evaluation data E in this embodiment indicates the degree of overlap between the original moving image data and the inverted moving image data obtained by inverting the comparison target portion of the original moving image data in which the motion of the user 5 is photographed. Although numerical data (digital display) is used as an embodiment, an analog display such as a bar graph, a line graph, and a meter may be used.
This evaluation data E is added when the matching rate of the image is high, and is subtracted when the matching rate is lowered, so that the quality of the walking state can be evaluated in real time and displayed to the user 5. .

Image processing for obtaining the superimposed data is performed in the following procedure.
That is, the electronic computer 1 as the image processing apparatus first extracts moving image data from the start to the end of the left foot axis walking or the right foot axis walking from the original moving image data obtained by photographing the user's movement.
Here, which axis foot is used as a reference is preferably selected in advance by the user. If the right foot axis walking is normal, the right foot axis walking is selected and the moving image data is used as a reference. It is specified by the electronic computer 1 so that
Then, inverted moving image data is generated by inverting the selected right foot axis walking moving image data (see FIG. 4B).

Next, the original video data at the start of the right foot axis walking video data including the video data from the start to the end of the right foot axis walking by the user 5 and at the start of the left foot axis walking indicating the current walking state By matching the timing with the start time, the timings of both data are matched, and the respective images of the corresponding moving image data are overlapped to obtain superimposed moving image data of the whole body (see FIG. 5B).
In this superimposed moving image data, the image data portion of the reversed moving image data is the portion where the image data of the reversed moving image data of the user 5 does not match the image data of the original moving image data indicating the current walking state. The image data portion of the original moving image data is preferably displayed in red on the monitor 3 (see FIG. 5C).
In the above, the method of displaying the inverted video data in blue and the original video data in red is used for the portion where the inverted video data and the current original video data do not match. In particular, since the reverse video is a correct motion, a display method in which only the non-matching portion of the current original video is displayed in red on the monitor 3 in the sense that the motion corresponding to the current original video is corrected. May be adopted.

In addition, in this superimposed moving image data, only the portion (skin color portion) where the skin of the user 5 is exposed is recognized separately, and the image data of the reversed moving image data and the current walking state are shown for that portion. A portion that matches the image data of the original moving image data shown may be displayed in white.
And about the part in which the image data in both moving image data do not correspond, the image data part of inversion moving image data is displayed in blue, and the image data part of original moving image data is color-coded and displayed in red.

That is, the superimposed moving image data includes moving image data indicating the current walking state by the user 5 (for example, moving image data of the left foot axis walking) and moving image data indicating the state of walking performed one step before (right foot axis in this case). Walking moving image data), and by inverting any one of the moving image data, it is possible to judge whether the current walking state is good or bad.
Therefore, the walking motion can be continued while easily comparing the correct walking state and the bad walking state directly in real time, and a high rehabilitation effect can be efficiently obtained.
Furthermore, if the skin-colored part is recognized, it is not necessary to attach a special marker before the exercise so that the image processing apparatus can recognize a specific position of the body, and the image processing of the present invention can be easily performed. The system can be utilized.

  The electronic computer 1 serving as the image processing apparatus is based on original moving image data (for example, moving image data including 30 image frames per second) obtained by capturing the motion of the user 5 from the start to the end of walking on the right foot axis. When the image data up to and the image data from the start to the end of walking on the left foot axis are respectively extracted, the image data constituting the original moving image data is exposed in the vicinity of the ankle of the user 5 (that is, the image The skin-colored part above) is analyzed, and when the exposed part is at the lowest point position, the start of walking with that foot as the axis and the end of walking with that foot as the axis is exposed. It is assumed that the portion is at the lowest point position (see FIG. 9).

  Further, in the example shown in FIGS. 5B and 5C, the superimposed moving image data of the whole body is obtained and displayed on the monitor 3. However, as shown in FIGS. Only the inverted moving image data of the inverted half of the original moving image data indicating the normal walking of the person 5 (for example, the left side) is superimposed on the original moving image data indicating the current walking state to obtain the superimposed moving image data. Good.

The outline of the data processing method by the image processing system using the electronic computer 1 as the image processing apparatus of the present invention will be described with reference to FIG.
The electronic computer 1 photographs the state of walking of the user 5 from the front by the photographing device 2 to obtain original moving image data (step S1).
Next, the computer 1 obtains image data from the start to the end of walking on the right foot axis by analyzing the exposed portion near the user's ankle for the image data constituting the original moving image data, Inverted moving image data obtained by inverting the left and right of each image data is obtained (step S2).

Next, the electronic computer 1 superimposes the reverse moving image data of the right foot axis walking and the original moving image data of the left foot axis walking (step S3).
Then, the electronic computer 1 analyzes the superimposed moving image data, and color-codes the non-overlapping portion on the monitor 3 (step S4).
Further, the electronic computer 1 digitizes the degree of coincidence of the images in the inverted moving image data of the right foot axis walking and the original moving image data of the left foot axis walking, and displays it on the monitor 3 together with the evaluation data E, the elapsed time T, and the number of steps N.

  Here, by connecting the electronic computer 1 to the Internet I, by connecting to the electronic computer 6 at a remote location, the image D of the instructing physician is displayed on the monitor 3 from the remote location, and from the speaker 7 through the speaker 7 Rehabilitation guidance can also be performed while conveying a guidance message to the user 5.

Furthermore, the image processing system according to the embodiment of the present invention changes the mode of the image processing to warn the user 5 when the user 5 exercises outside the allowable range. It can also be displayed on the monitor 3.
Examples of warnings are monitors such as “the head is shaking to the left. Do not move the head too much.”, “The right foot is shaking out. Be careful not to shake outside.” A display on three screens or a sound from the speaker 7 can be considered.
This is because the user's skin is specifically recognized (face and foot near the ground), for example, the right foot 8 of the user 5 is, for example, about 10 cm from the appropriate position shown in FIG. When it protrudes outward, a warning for the user 5 is displayed on the monitor 3.

The warning is displayed on the monitor 3 by displaying one of the right and left screens (zones Z1 and Z2) of the user 5 displayed on the monitor 3 screen, for example, changing the zone Z1 from light blue to light red. Can be simply and clearly shown that a bad movement has occurred on the right side of the body of the user 5 (see FIG. 7B).
In addition to the display on the monitor 3 described above, the user 5 can know the presence of a bad movement both visually and audibly by issuing a warning sound from the speaker 7 or the electronic computer 1.

Furthermore, in the image processing system according to the embodiment of the present invention, when the face 9 of the user 5 moves beyond an allowable range by particularly recognizing a portion where the user's skin is exposed, That is, when a large tilt occurs in the trunk, the color of the screen (zones Z3 and Z4) on the monitor 3 may be changed to display a warning to the user 5.
For example, as shown in FIG. 7C, zones Z3 and Z4 are set on both sides of the width of the waist of the user 5 (about 40 cm), and the face of the user 5 is shown in FIG. 7D. When 9 moves to the left from the width of the waist (about 40 cm), it is conceivable to change the color of the zone Z4 on the monitor 3 from light blue to light red.

Similarly, when the user's 5 arm moves beyond the allowable range, the color of the screen on the monitor 3 may be changed to display a warning to the user 5.
Further, as a result of the image processing, the electronic computer 1 allows a difference between the required time from the start to the end of the right foot axis walking by the user 5 and the required time from the start to the end of the left foot axis walking. If it exceeds the range, a warning sound may be emitted from the speaker 7 or the electronic computer 1 to the user 5, or a warning to that effect may be displayed on the monitor 3 screen.
Conversely, if the left and right movements of user 5's body are correctly matched, and if the difference between the time required for walking on the right foot axis and walking on the left foot axis is within the allowable range, “You can walk beautifully. “Please do your best in this condition” may be displayed on the screen of the monitor 3, or the speaker 7 may be notified of this by voice.

  As described above, in the present invention, an image processing system and an image that are particularly suitable for determining whether or not a person's walking (including fast walking such as a racewalk) and running are correctly performed. Gait rehabilitation is possible by providing a processing method and walking or exercising while directly seeing in real time whether or not a person's walking balance is appropriate or correct. It is possible to provide an image processing system and an image processing method suitable for support or sports training.

Next, another embodiment in which the image processing system and the image processing method of the present invention are embodied for gait rehabilitation support will be described with reference to FIGS.
FIG. 9 is a diagram showing the contact time of the left and right feet and the standing time of each foot when the user 5 performs walking training, and the Y-axis value is the lower end of the exposed portion of the foot skin. Indicates the position of the pixel corresponding to the position. As shown in FIG. 10, the value of the pixel position corresponding to the lower end position of the captured image by the image capturing apparatus 2 is 320, and the value of the pixel position corresponding to the upper end position of the captured image by the image capturing apparatus 2 is 0. It has become.

This image processing system further emits a guide sound for checking the walking rhythm from the speaker 7 as an acoustic device together with the start of the left foot axis motion and the right foot axis motion in the image processing system of the embodiment. I am doing so.
“Guide sound for checking walking rhythm” is a sound that composes music familiar to the user 5 such as nursery rhymes, simple melodies, etc., and grasps a certain tempo or timing during walking It is desirable that the sound is easy.
Specifically, in the image processing system according to the embodiment, the program of the electronic computer 1 determines when the left foot axis motion and the right foot axis motion start, and when the left foot axis motion and the right foot axis motion start. In addition, whether or not the walking rhythm is appropriate for the user 5 by sequentially generating as a guide sound one by one from the speaker 7 so that the user 5 can hear the sound that constitutes a nursery rhyme (for example, music of “frog song”) It is possible to point out whether or not.

Furthermore, in this embodiment, the rhythm of walking with the left foot and the right foot (as shown in FIG. 10, the stance time by each foot is displayed as the number of image frames on the monitor 3 screen). If it is correct, 100 points are displayed on the monitor 3 screen, and the score deducted by 5 points for every 1/30 second deviation is displayed on the monitor 3 screen as the evaluation data E. It makes it possible to recognize clearly.
Furthermore, when the number of image frames required for walking with the right foot matches the number of image frames required for walking with the left foot, the right side of the rhythm determination display unit 10 provided on the screen of the monitor 3 is illuminated in white, and walking with the right foot is performed. If the number of image frames required for walking and the number of image frames required for walking with the left foot do not match, the left side of the rhythm determination display unit 10 provided on the monitor 3 screen is lit red to indicate the state of walking. 5 can be easily confirmed visually.

By walking according to this guide sound, the user 5 can grasp whether or not his / her walking rhythm is correct by checking whether or not the guide sound heard from the ear is a correct melody. As a result, the user 5 can enjoy walking training while listening to music.
That is, in this embodiment, the user 5 can perform walking training while maintaining high motivation in accordance with music obtained by walking.

  The following table shows the results of the walking training in which the walking rhythm is pointed out by the image processing system and the image processing method of this embodiment and the case in which the walking training is performed unconsciously without pointing out the walking rhythm.

  From the above results, an acoustic device (speaker 7) that emits a guide sound for checking the walking rhythm in conjunction with the start of the movement of the left foot axis and the movement of the right foot axis is used to hear the walking rhythm from the user 5 In addition, it was confirmed that not only learning of the correct walking rhythm becomes easy, but also the motivation of the user 5 for walking training can be kept high.

  Furthermore, in the present invention, a plurality of photographing devices (video cameras) 2 are installed at different heights or different positions so that the position and state of the legs of the user 5 can be displayed and determined more accurately. Or a partial enlarged image can be separately displayed on the monitor 3, or the user's 5 movement can be grasped by using a photographing device capable of identifying light in a specific wavelength range, It is possible to grasp the changes in the body temperature of the user 5, etc., and it is possible to carry out so that the health condition of the user 5 by the instructor can be accurately judged and rehabilitated within an appropriate range. Such cases are also included in the scope of the present invention.

  The present invention is not limited to the above-described embodiment, and can be modified and implemented without departing from the spirit of the present invention. For example, the computer 1 functioning as an image processing apparatus The operation speed of the red mill 4 can be changed, the operation of the tread mill 4 can be stopped, a message for confirming the intention to change the operation mode is displayed on the monitor 3 screen, and decision making and operation mode selection can be performed. It is also possible to switch the operation mode or stop the operation by performing a display for the purpose and touching the display part on the monitor 3 screen.

As the message for confirming the intention of switching the operation mode, for example, “please hold your hand over the menu you want to implement” can be considered.
As a display for decision making and selection of an operation mode, the following can be considered, for example.
1) When it is desired to determine the state of walking motion as shown in FIGS. 5 and 6 from the superimposed moving image data, the characters “walking” are displayed on the monitor 3.
2) As shown in FIGS. 7A and 7B, when a zone is set on the screen of the monitor 3 and it is desired to judge a bad movement of the right foot outward, the character “right foot” is displayed on the monitor 3. To do.
3) As shown in FIGS. 7C and 7D, when it is desired to set a zone on the monitor 3 screen and determine the inclination of the trunk (face), the characters “trunk” are displayed on the monitor 3. (See FIG. 11).

Further, as shown to the user 5 with a moderate walking pitch (pace) by the electronic computer 1, a beep sound or rhythm is emitted from the speaker at regular intervals regardless of the actual walking state of the user 5. May be implemented.
The present invention can be implemented using a treadmill 4 having a handrail 4a as shown in FIG. 10, or can be applied to a game using a composite image.
Further, when there is moving image data before the user 5 is ill (hemiplegia), it is used as original moving image data (a reference image indicating a normal normal motion) by the image processing device (as it is a moving image) Data or moving image data that is reversed left and right can be used as a reference image), and superimposed with moving image data that shows the current state of motion (moving image data as it is or reversed moving image data) Even if it is displayed on the top 3, a good rehabilitation effect can be obtained.

  The present invention is an image processing system suitable for gait rehabilitation support or exercise training for a user who has hemiplegia or a prosthetic leg, an image processing method, and a program for executing the image processing method on an electronic computer. It can also be used in games played by moving the body.

1 Electronic computer (image processing device)
2 Shooting device 3 Monitor (TV)
4 treadmill 4a handrail 5 user 6 electronic computer 7 speaker (acoustic device)
8 Right foot 9 Face 10 Rhythm judgment display part I Internet D Image of supervising doctor E Evaluation data N Step count S1 to S6 Step T Elapsed time (remaining time)
Z1-Z4 zone

Claims (15)

With treadmill,
A photographing device for photographing a movement of a user performed on the treadmill;
An image processing device for processing moving image data indicating a user's movement obtained by the photographing device;
A monitor for displaying an image processing result obtained by the image processing apparatus so that a user can view the result on the treadmill;
An image processing system comprising:
The image processing device uses either one of the left foot axis motion or the right foot axis motion from the start time to the end time of the left foot axis motion obtained from the original motion picture data of the user's past motion. Inverted video data with the video data reversed left and right, and
An image processing system, wherein superimposed moving image data is displayed on a monitor by superimposing the current moving image data obtained from the start of movement by the other axis foot.   The image processing system according to claim 1, further comprising an acoustic device that emits a guide sound for checking a walking rhythm in conjunction with the start of the left foot axis motion and the right foot axis motion.   3. The image processing apparatus according to claim 1, wherein the inverted moving image data is half-length moving image data and is superimposed on a movement of a foot on a side that is not an axial foot of the current moving image data. Image processing system.   4. The image processing system according to claim 1, wherein the image processing apparatus is configured to display a user's past moving image data or current moving image data in a color-coded manner on a monitor. 5. .   The said image processing apparatus displays the warning with respect to a user on a monitor, when a user's exercise | movement is performed exceeding the range which is accept | permitted, The monitor as described in any one of Claim 1 thru | or 4 characterized by the above-mentioned. Image processing system.   6. The image processing apparatus according to claim 1, wherein the image processing apparatus displays evaluation data indicating a degree of overlap between the inverted moving image data and the other original moving image data corresponding to the inverted moving image data on a monitor. The image processing system according to one item. The image processing apparatus includes:
When obtaining the video data from the start to the end of the left foot axis walking or right foot axis walking from the original video data shooting the user's movement,
For the image data that makes up the original video data, analyze the exposed part of the skin near the user's ankle, and start walking when the exposed part of one foot is at the lowest point position The image processing system according to claim 6, wherein the end of walking with the foot as an axis is when the exposed portion of the other foot is at the lowest point position.   If the difference between the required time from the start to the end of the left foot axis walk and the required time from the start to the end of the right foot axis walk exceeds the allowable range, a warning is issued to the user. The image processing system according to claim 1, wherein the image processing system is one.   The image processing apparatus is connected to an electronic computer at a remote location, thereby transmitting superimposed moving image data to the electronic computer, displaying image data from the electronic computer on a monitor, and voice to the user. The image processing system according to claim 1, wherein the image processing system can transmit a message. With treadmill,
A photographing device for photographing a movement of a user performed on the treadmill;
An image processing device for processing moving image data indicating a user's movement obtained by the photographing device;
A monitor for displaying an image processing result obtained by the image processing apparatus so that a user can view the result on the treadmill;
An image processing method in an image processing system comprising:
According to the left foot axis motion or the right foot axis motion from the beginning to the end of the motion data of the left foot axis obtained from the original motion picture data obtained by photographing the past motion of the user by the image processing device. Inverted video data with the video data reversed left and right, and
An image processing method, comprising: superimposing moving image data superimposed on current moving image data obtained from the start of movement by the other axis foot on a monitor.   11. The image processing system according to claim 10, further comprising an audio device, and generating a guide sound for checking a walking rhythm in conjunction with the start of the left foot axis motion and the right foot axis motion. Image processing method.   The image processing method according to claim 10 or 11, wherein the overlapping of the inverted moving image data and the other moving image data by the image processing device is performed by color-coding.   The said image processing apparatus displays the warning with respect to a user on a monitor, when a user's exercise | movement is performed exceeding the range which is accept | permitted, The any one of Claim 10 thru | or 12 characterized by the above-mentioned. The image processing method according to item.   14. The image processing device displays evaluation data indicating a degree of overlap between the inverted moving image data and the other original moving image data corresponding to the inverted moving image data, on a monitor. An image processing method according to any one of the above.   A program for executing the image processing method according to any one of claims 10 to 14 on an electronic computer.

Images