JP7116696B2 - Learning support system and program - Google Patents

Description

The present invention relates to a learning support system, a learning support device, and a program used when learning a technique.

Conventionally, techniques in various technical fields have been passed down through actual training and work performed by technical learners under the guidance of instructors.

In order to convey advanced skills to learners, it is sometimes necessary to convey tips that are difficult to express in words, but it is not easy for instructors to efficiently convey these tips to learners. In recent years, in order to efficiently convey the skills acquired by the instructor to the learner, devices and methods have been proposed that store the instructor's skills as data and display the stored technical data to the learner (for example, Patent document 1).

Patent Literature 1 discloses a method of extracting and storing movements of a leader, such as the position of the leader's hand, the amount of movement, and the speed of movement, under various environments. In this method, the instructor performs the same work each time while changing the environment, and by measuring the instructor's movements each time using motion capture, etc., changes in the environment and movements are associated and saved. .

In addition, a technology has been proposed that visualizes the movement information of the instructor that has been measured, and displays the image superimposed on the actual work scene using augmented reality (AR) technology using glasses-type wearable devices (smart glasses). (for example, Patent Document 2).

For example, in the work support device disclosed in Patent Document 2, the information processing unit compares the surface shape of the work target object measured by the imaging unit with the surface shape of the work target object that is stored in advance. After calculating the amount to be thermally deformed at the point, the position to be heated and the amount of heat to obtain the amount of deformation are calculated, and an image showing the position to be heated is superimposed on the image of the work target object and displayed on the head-mounted display. it's shown.

Japanese Patent Application Laid-Open No. 2003-281287 JP 2009-69954 A

A conventional learning support system, such as Patent Document 2, which displays the data of the instructor's technique to the learner, can be used for learners with different proficiency levels and for learners with different characteristics such as dominant hand. Show technology like In such a system, learners with different proficiency levels and characteristics may not be able to learn efficiently.

An object of the present invention is to provide a learning support system that enables a learner to efficiently acquire skills.

In order to solve the above problems, the present invention provides a display unit worn by the learner, an imaging unit worn by the learner to capture the visual field image of the learner, and an instructor who serves as a model of the learner's actions. It comprises a storage section for storing a model video, which is a video of a working action, and a computing section. The calculation unit superimposes the model video on the visual field image captured by the imaging unit and displays it on the display unit. Change content.

According to the present invention, the displayed model moving image dynamically changes according to the work motion of the learner, so that the learner can train to acquire the technique at a pace suitable for each learner. This allows the learner to acquire the technology efficiently.

Explanatory drawing which shows the structure of the learning support system of 1st Embodiment. 1 is a block diagram showing the configuration of a control system 101 of the learning support system according to the first embodiment; FIG. 4 is a flowchart showing the flow of learning and training according to the first embodiment; An example of a screen when a positional difference is displayed on the display unit 19. FIG. An example of a screen when a positional difference is displayed on the display unit 19. FIG. A display example of the position difference. A display example of the position difference. A display example of the position difference. A display example showing the difference in action elements between the learner and the instructor. A display example showing the difference in pressure between the learner and the instructor. 2 is a block diagram showing the details of the configuration of the glasses-type device 201. FIG. An example of changing the display of operation elements with reduced differences. 1 is a block diagram showing a configuration example of a control system 101 of a learning support system; FIG. 4 is a flowchart showing the overall flow of the learning support system according to the first embodiment; FIG. 4 is a diagram showing input/output sequences of the control system 101, the glasses-type device 201, and the sensing device 204; (a), (b) Examples of UI displayed on the display unit when inputting basic information and training type. Explanatory diagram of difference determination, proficiency evaluation, and review. An example of a log stored in the information storage unit 17. FIG. 4 is a flow chart showing the flow during the level determination mode; Explanatory drawing which shows the structure of the learning support system of 2nd Embodiment. FIG. 2 is a block diagram showing the configuration of a glasses-type device worn by a leader; The block diagram which shows the structure of the control system 101 of the learning support system of 2nd Embodiment. 9 is a flow chart showing the operation flow of the learning support system of the second embodiment. The block diagram which shows the structure of the learning support system of 3rd Embodiment. 10 is a flow chart showing the flow of operation of the learning support system of the third embodiment; 11 is a flow chart showing the flow of operation of a learning support system according to a modification of the third embodiment; Explanatory drawing which shows the structure of the learning support system of 4th Embodiment. A display example of the display unit 19 of the learning support system of the fourth embodiment.

A learning support system according to an embodiment of the present invention will be described below.

<<Learning support system of the first embodiment>>
As a learning support system according to the first embodiment, an example of a system for a glasses-type device in which a learner learns how to operate a probe in an ultrasonic imaging apparatus will be described. As shown in FIG. 1, this learning support system includes a glasses-type device 201 worn by a learner 102, and glasses connected to the glasses-type device 201 via a network (eg, a wireless LAN (Local Area Network)) 30. and a control system 101 that controls the functions of the mold device 201 . The glasses-type device 201 includes a display unit 19 and an imaging unit (camera) 15 that captures the visual field image of the learner 102 .

As shown in FIG. 2, the control system 101 includes a computing unit 16 that performs various computations in the learning support system based on information received from the glasses-type device 201, an information storage unit 17 that stores various information, It has The calculation unit 16 superimposes a model moving image (for example, see FIG. 4), which is a moving image of the instructor's work motion, which is a model of the work motion of the learner 102, on the visual field image captured by the camera 15, and displays it on the display unit 19. In addition to displaying, the display content of the model moving image is dynamically changed according to the feature of the work motion of the learner 102 included in the visual field image. The information storage unit 17 stores at least model data including a model moving image and position information of the instructor's hand, which will be described in detail later.

The calculation unit 16 executes various calculations by executing a learning support program stored in advance in the information storage unit 17 . Specifically, the calculation unit 16 superimposes the model moving image on the video capturing unit 161 that captures the visual field video of the learner 102 from the camera 15 and the visual field video of the learner 102, and outputs the superimposed processed image to the display unit 19. a superimposed display unit 162, a difference calculation unit 163 for calculating the difference between the work action of the instructor included in the model moving image and the work action of the learner 102 included in the visual field image, and the difference calculated by the difference calculation unit 163 is equal to or greater than a predetermined threshold, and a display adjustment unit 16C changes the display content of the model moving image to be displayed based on the difference calculated by the difference calculation unit 163.

The flow of training motions by the learning support system of the first embodiment will be described with reference to FIG. In this flow, a work operation when the learner 102 performs training while holding the probe 202 in his/her hand will be described.

The training operation of the flow of FIG. 3 is controlled and executed by control system 101 . First, the camera 15 captures the visual field image of the learner 102 . The video capturing unit 161 captures the visual field video of the learner captured by the camera 15 via the network 30 (step S41). Next, the superimposed display unit 162 reads out the model image stored in the information storage unit 17, and superimposes the image of the learner's hand and the probe 202 included in the learner's visual field image onto the image of the instructor included in the model moving image. Positions are adjusted so that predetermined reference points of the hand and the probe overlap, and the position-adjusted model moving image is displayed on the display unit 19 (step S42). Next, as shown in FIG. 4, the difference calculation unit 163 calculates the difference between the position of the probe 202 held by the learner and the position of the probe held by the instructor. Specifically, first, the difference calculation unit 163 calculates the position of the center 202b of the surface of the probe 202 held by the learner that touches the surface of the inspection target, from the visual field image of the learner. Further, the difference calculation unit 163 reads out the position of the center 202c of the surface where the probe held by the instructor touches the inspection target from the model data stored in the information storage unit 17 . Then, the difference calculator 163 calculates the difference between the position of the center 202c of the probe held by the instructor and the position of the center 202b of the probe 202 held by the learner (step S43).

Next, the difference determination unit 16A determines whether or not the difference calculated by the difference calculation unit 163 in step S43 is equal to or greater than a predetermined threshold. If the difference is equal to or greater than the threshold, the result (difference determination result) is displayed. Output to the adjustment unit 16C (step S44). The display adjustment unit 16C performs image processing to shift the position of the probe center 202b of the model moving image to the position of the probe 202 held by the learner, and transmits the processed model moving image to the glasses-type device 201 via the network 30. output to The model moving image received by the glasses-type device 201 is displayed on the display unit 19 (step S45).

As a result, the learner can see the images in which the positions of the probes of the instructor and the learner are overlapped, so that the orientation and angle of the hand and probe can be matched to the orientation and angle of the instructor's hand and probe. , you can learn how to move your hands and probes efficiently.

After the model moving image processed in step S45 is displayed to the learner, the flow returns to step S41. The learning support system repeats the above flow until the learner's training is completed. Moreover, when the difference calculated by the difference calculation unit 163 in step S44 is less than the threshold value, the flow returns to step S41.

Note that the display adjustment unit 16C uses the center 202b of the learner's probe 202 and the center 202c of the instructor's probe as reference points, and aligns the position of the hand of the model video with the position of the hand in the visual field image of the learner. but other reference points may be used. For example, the display adjustment unit 16C detects the position of a part of the body of the learner and adjusts the display position of the model video, such as aligning the position of the fingertip of the learner 102 with the position of the fingertip of the instructor of the model video. may

[Display to inform the learner of the difference in position]
Another example of display adjustment performed by the display adjustment unit 16C when the position difference between the learner and the instructor calculated by the difference determination unit 16A is equal to or greater than the threshold will be described below.

The display adjustment unit 16C can display the positional difference between the center 202b of the probe 202 held by the learner and the center 202c of the probe held by the instructor by an arrow 102d as shown in FIG. The learner can recognize how much his or her motion deviates from the model by looking at the size and inclination of the arrow 102d. Also, the learner can easily bring the position of the probe 202 held by him/herself close to the position of the probe held by the instructor while looking at the arrow 102d.

In addition, the position of the instructor's elbow may be included in the model moving image, and the position information of the elbow may be stored in the information storage unit 17 . In this case, the difference calculation unit 163 can detect the position of the elbow of the learner from the visual field image of the learner, and calculate the difference between the position of the elbow of the learner and the position of the elbow of the instructor. When the difference between the position of the learner's elbow and the position of the instructor's elbow is greater than or equal to the threshold value, the display adjustment unit 16C, as shown in FIG. It is possible to display concretely in sentences how to move the body. In addition, the display adjustment unit 16C may display a hint by displaying a sign such as an arrow 19b on the part of the body that should be moved.

Further, the information storage unit 17 may store a skeleton diagram 19c showing the positions of the bones and joints of the instructor's hand 102b. In this case, as shown in FIG. 6, the display adjustment unit 16C can display a skeleton diagram 19c superimposed on the image of the instructor's hand 102b. By displaying the skeletal diagram 19c, the learner can easily correct the position of his/her own hand while viewing the displayed positions of the joints of the instructor's hand 102b.

Further, in the information storage unit 17, as shown in FIG. 7, there is a color distribution diagram in the shape of a working hand or arm, which indicates the magnitude of the force to be applied to each part of the hand at each time point of the work. 19d may be stored. In this case, the difference calculation unit 163 detects the position and shape of the learner's hands and arms from the learner's visual field image, and the display adjustment unit 16C displays the color distribution diagram 19d superimposed on the learner's hand 102c. can be done. For example, like thermography, warm colors are superimposed on parts of the body where the learner should apply a lot of force to the muscles, and cool colors are superimposed on parts of the body where the learner should apply a little force. , the learner can be made to recognize the degree of effort put in.

In addition, as shown in FIG. 8, the display adjustment unit 16C detects areas where the positions of the instructor's probe and the probe 202 held by the learner overlap and areas where the positions of the probes 202 are misaligned, and displays the overlapping area in white. , and the instructor's probe and the probe 202 can be displayed in different colors for areas where misalignment occurs. In this way, the area where the positions of the learner and the instructor are misaligned can be displayed in different colors for the learner and the instructor, so the difference in the positions of the learner and the instructor is emphasized. . Therefore, the learner can easily recognize the direction in which to move his or her hand.

Further, when the position of the probe 202 held by the learner deviates from the position of the probe held by the instructor, the display adjustment unit 16C may be configured to rewind and reproduce the model moving image to the timing when the position deviation starts. At this time, the model moving image to be played again may be divided into short-time segments, and the model moving image may be repeatedly reproduced for each segment. As a result, the learning effect can be enhanced because the learner can practice the work motions that the learner is not good at many times.

Note that the glasses-type device 201 may be provided with a vibration sensor. In that case, when the position of the probe 202 held by the learner and the position of the probe held by the instructor are deviated, the vibration sensor mounted on the glasses-type device 201 gives force feedback such as vibration or impact to the learner. may

By changing the display content as described above, the learner can be informed of the positional deviation, so that the learner can easily grasp the correct positions of the probe and the hand, and can easily correct the positional deviation.

[Display adjustment type]
<Operating elements other than position>
In the above description, an example has been described in which the difference calculation unit 163 calculates the difference (action element) between the positions of the probes or hands of the learner and the instructor. As the motion element for calculating the motion difference, one or more motion elements such as "speed, pressure, motion trajectory, gaze point, sound" may be detected in addition to "position". These action elements of the instructor are included in the model data and stored in the information storage unit 17 . A method of calculating the difference between the learner and the instructor in each action element and a method of displaying the difference will be described with reference to FIG.

・From the video imaged by the speed camera 15, the difference calculation unit 163 calculates the speed of the learner's hand movement, and the calculated speed of the learner's hand and the instructor's hand stored in the information storage unit 17. The difference between the movement speed of the learner and the instructor is calculated from the difference in the position of the hands of the learner and the instructor. When the difference determination unit 16A determines that the difference between the speed of the learner and the speed of the instructor is equal to or greater than a predetermined value, the display adjustment unit 16C adjusts the playback speed of the model video to match the speed of the learner. . For example, when the work motion of the learner 102 is slower than the work motion of the instructor by a predetermined value or more, the display adjustment unit 16C slows down the reproduction speed of the model moving image so as to match the speed of the learner 102 . The learner's and instructor's speeds are displayed by different color temperatures of the displayed images, barometers, numerical values, etc. so that the learners can recognize the difference between their own speed and the instructor's speed. You may do so.

Pressure Information on the pressure when the instructor presses the probe against the test object (hereinafter referred to as the instructor's pressure) is stored in advance in the information storage unit 17 . The superimposed display unit 162 reads the instructor's pressure information from the information storage unit 17, and superimposes a circle 103b2 indicating the magnitude of the pressure on the visual field image of the learner as shown in FIG. . Further, the difference calculation unit 163 calculates the pressure with which the learner presses the probe 202 against the test object from the color change of the hand, fingertip, or nail when the learner presses the probe 202 against the test object. When the difference determination unit 16A determines that the difference between the pressure of the learner pressing the probe 202 and the pressure of the instructor is equal to or greater than a predetermined value, the display adjustment unit 16C determines the difference between the pressure of the instructor and the pressure of the learner. The difference is displayed by, for example, the size of the circle, the difference in the color of the circle, and the like. The difference between the instructor's pressure and the learner's pressure is emphasized by, for example, filling in an area surrounded by a circle 103b1 indicating the pressure by the learner 102 and a circle 103b2 indicating the pressure by the instructor as shown in FIG. can be displayed. By moving the probe so as to reduce the area filled in with this circle, the learner can perform training in which the test object is pressed with a pressure close to that of the instructor.

- Motion trajectory Information on the route (motion trajectory) along which the instructor moves the probe is stored in advance in the information storage unit 17 . The superimposed display unit 162 reads information on the motion trajectory from the information storage unit 17, and superimposes and displays a flow line indicating the motion trajectory on the visual field image of the learner. At that time, as shown in FIG. 9, the color of the instructor's motion trajectory is set to the past trajectory (past motion trajectory) 103c or the future trajectory (future motion trajectory) from the current position of the probe 202 of the learner. 103d may be displayed in different colors. Further, the difference calculation unit 163 calculates the motion trajectory of the learner's probe 202 from the learner's visual field image captured by the camera 15, and the motion trajectory of the learner's probe 202 and the motion trajectory of the instructor's probe. Find the difference. When the difference determination unit 16A determines that the difference is equal to or greater than the predetermined value, the display adjustment unit 16C adds the motion trajectory of the learner's probe 202 to the already displayed instructor's probe motion trajectories 103c and 103d. A learner may be allowed to recognize the difference between the instructor's action trajectory and the learner's action trajectory by superimposing the indicated flow line.

- Gaze Point Information on the point (gazing point) that the instructor pays attention to during the work is stored in advance in the information storage unit 17 . The superimposed display unit 162 reads the point-of-regard information from the information storage unit 17, and superimposes a point 103a indicating the point-of-regard on the visual field image of the learner. The spectacles type device 201 includes a line-of-sight sensor 20 as shown in FIG. 11, which will be described later. The line-of-sight sensor 20 detects the gaze point of the learner. The difference calculation unit 163 calculates the difference between the learner's gaze point and the leader's gaze point 103 a stored in the information storage unit 17 . When the difference determination unit 16A determines that the difference between the gaze point 103a of the learner and the instructor is equal to or greater than the threshold value, the display adjustment unit 16C highlights and displays the instructor's gaze point 103a by blinking or the like. The display adjustment unit 16C may cause the display unit 19 to display the point of gaze of the learner detected by the line-of-sight sensor 20, other than the point of gaze 103a of the instructor, using a cross symbol or the like.

- Sound The glasses-type device 201 may be equipped with a recording device for recording work sounds. In this case, the recording device acquires the work sound of the learner, and the work sound of the learner recorded by the difference calculation unit 163 and the work sound of the model video stored in the information storage unit 17 are combined. Calculate the difference. When the difference determination unit 16A determines that the difference between the work sounds of the learner and the instructor is equal to or greater than the threshold value, the display adjustment unit 16C displays that the sounds are out of sync using waveforms, colors, characters, and the like.

It should be noted that the information storage unit 17 may store in advance the statistically likely deviation and its cause for each motion element. In that case, the difference calculation unit 163 may be configured to obtain the cause of the difference between the learner's work motion and the instructor's work motion for the motion element in which the learner's work motion and the instructor's work motion are deviated. good. Further, the display adjustment unit 16C may cause the display unit 19 to display the statistically likely deviation of each motion element and its cause.

[Glasses-type device 201]
Here, the configuration of the glasses-type device 201 will be specifically described. As shown in FIG. 11, the spectacles-type device 201 includes spectacles, a camera 15, a display unit 19, a wireless communication device 13, a CPU 12, a memory 18, and a line-of-sight sensor 20 mounted on a part of the spectacles. be done.

The camera 15 captures an image in a predetermined direction such as the line of sight of the user, and measures the work motion of the user.

The display unit 19 projects the model moving image adjusted by the display adjustment unit 16C under the control of the calculation unit 16 and the CPU 12 into the user's field of view.

As the display unit 19, it is also possible to use a structure that projects an image onto the user's retina. A wearable display may be used instead of glasses. In this case, the visual field image of the user captured by the camera 15 and the model moving image adjusted by the display adjustment unit 16C are superimposed and displayed on the wearable display.

A wireless communication device 13 performs communication between the CPU 12 and the control system 101 .

The CPU 12 controls the operation of each part of the glasses-type device 201 based on the information received from the calculation part 16 of the control system 101 . Specifically, an instruction is sent to the wireless communication device 13 to transmit the image data captured by the camera 15 to the image capture unit 161, or information received from the superimposed display unit 162 or the display adjustment unit 16C is sent to the display unit 19. It is reflected in the video that is sent to and displayed.

The memory 18 stores the image captured by the camera 15 and the calculation result of the CPU 12 as required. Also, the memory 18 can store images to be displayed on the display unit 19 .

The line-of-sight sensor 20 is a sensor that detects where the gaze point of the user is. The line-of-sight sensor 20 transmits data of the detected gaze point to the CPU 12 .

[Training mode]
In the above description, it is assumed that the display adjustment unit 16C aligns the position of the probe held by the instructor with the position of the probe 202 held by the learner at any time. In the information storage unit 17, there are a beginner mode for basic training in which the position of the instructor's hand is adjusted at any time to enable training to match the learner's pace, and a basic training mode in which the instructor's hand is displayed only when the difference exceeds a predetermined value. Multiple training modes (Table 1) may be stored, such as a normal mode in which the position of is modified. Now, the training modes stored in the information storage unit 17 will be described.

In the beginner mode, during the reproduction of the model moving image, the model moving image follows the work motion of the learner 102, and the superimposed position and movement speed of the model moving image are dynamically changed. In this mode, the model video is adjusted to the learner's movement pace, so the learner can observe the instructor's work closely and memorize the correct work patterns.

The normal mode is a training mode in which a model video is played back with an emphasis on the speed of the instructor's work so that the learner can improve the quality of the work while getting used to it. In this mode, basically, the model video is played back at the work speed of the instructor, and only when the work action of the learner 102 is delayed by a certain amount or more, the superimposed position and playback speed of the model video are adjusted to suit the learner 102. controlled.

The processing in each training mode is as shown in the flow shown in FIG. Dynamically change according to work operation. Specifically, based on the difference determination result, the display adjustment unit 16C causes the learner 102 to display the action elements whose difference is equal to or greater than the predetermined value in a manner that is easier for the learner 102 to perceive than the action elements whose difference is less than the predetermined value. In this manner, the transmittance of each action element of the model moving image is dynamically changed, and the presence/absence of display is dynamically switched.

Specifically, for example, as shown in FIG. 12(a), a circle 103b2 indicating the instructor's pressure, the instructor's gaze point 103a, and motion trajectories 103c and 103d are displayed, and the learner's pressure and gaze point are displayed. If the difference between the instructor's pressure and the gaze point changes to decrease, it means that the learner is working with appropriate pressure and gaze point. As shown in 12(a) to 12(c), transparency control is performed to increase the transparency of the displayed instructor's pressure 103b2 and the gaze point 103a. Conversely, the display adjustment unit 16C lowers the transparency and emphasizes and displays the action elements that change such that the difference between the work action of the learner and the work action of the instructor increases.

This kind of transparency control emphasizes the display of movement elements that the learner is not good at (here, moving the probe while drawing the correct trajectory), so the learner can focus on the movement elements that the learner is not good at during training. be able to.

[Program structure/flow of overall training]
An example of the configuration of the technical training program executed by the learning support system of the present embodiment and the overall flow of the training will be described below. The information storage unit 17 stores a plurality of modes as shown in Table 1, for example, as learning support programs. Specifically, in addition to the beginner mode and normal mode described above, the information storage unit 17 stores a level determination mode, a guidance mode, and a review mode.

When executing these modes, the calculation unit 16 of the control system 101 includes, as shown in FIG. A level evaluation unit 16B that evaluates the technical level of the learner 102 from the result of difference determination by the difference determination unit 16A, and a statistical calculation unit 16D that statistically calculates the skill data of the learner 102 are provided.

A well-known method can be used as a method for the superimposed display unit 162 to align and superimpose the view image of the learner and the model moving image. For example, the superimposed display unit 162 is attached to objects that come into the field of view of the learner 102 during skill acquisition, such as the body (hands, etc.) of the learner 102, an object of ultrasonic examination, a probe held by the learner, or the surroundings thereof. The mark is used as a reference marker to determine the display position of the model movie.

The functions of the calculation unit 16 are realized as software installed in the CPU or GPU. Part or all of the functions of the computing unit 16 can also be realized by hardware such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

In the information storage unit 17, in addition to a learning support program including a training mode for skill acquisition, basic information of the learner 102 (for example, age, dominant hand, habit, eyesight, difference in physical ability of each individual such as grip strength), training A data table of the learner 102 including model data related to work actions performed by the instructor according to the type of training, a log of the technical data of the learner 102, a weak point list of the learner 102, and the like is stored. The model data stored in the information storage unit 17 includes a model video, a plurality of motion elements of the instructor's work motion (position, speed, pressure, motion trajectory, gaze point, and sound), basic information of the model video ( (e.g., dominant hand, examination site to be examined by ultrasound examination in the model video, examination content, patient's body shape), method of displaying the difference when there is a difference between the action of the instructor and the learner for each action element, etc. . Further, the information storage unit 17 stores the image captured by the camera 15 and the calculation result of the calculation unit 16 as necessary. Furthermore, the information storage section 17 can also store images to be displayed on the display section 19 .

The overall flow of training by the learning support system will be described below with reference to FIGS. 14 and 15. FIG. In this flow, a case will be described in which six motion elements of position, speed, pressure, motion trajectory, gaze point, and sound are used as motion elements for judging the difference between the work motion of the learner and the work motion of the instructor.

Also, in this flow, instead of the probe 202, the learner will hold the sensing device 204 (see FIG. 1) for training. When training using the sensing device 204 , the sensing device 204 is connected to the control system 101 via the network 30 in addition to the glasses-type device 201 . Sensing devices 204 may include pressure sensors, tilt sensors, temperature sensors, humidity sensors, light sensors, tactile devices (eg, vibrators) that provide tactile feedback to learner 102, and the like. When the sensing device 204 is connected to the control system 101 , information acquired by the sensing device 204 can be used for computation by the computation unit 16 . The superimposed display unit 162 can convert information such as pressure and tilt detected by the sensing device 204 into an image that can be displayed on the display unit 19 and superimpose it on the visual field image of the learner 102 . If the sensing device 204 is provided with a sensor that acquires information such as temperature, humidity, and light intensity that cannot be acquired from the camera 15, the information acquired by the sensor can be used for display control of the model moving image by the display adjustment unit 16C. can. Even information that can be acquired from the camera 15 may be acquired with higher accuracy than the camera 15 by using the sensing device 204 . It should be noted that the sensing device 204 preferably has the same shape as the tool used in the training work (for example, the probe of the ultrasonic imaging apparatus).

[Step S1]
First, the control system 101 receives input of basic conditions necessary for training from the learner. Specifically, the control system 101 causes the display unit 19 to display a UI screen as shown in FIG. The learner inputs the learner's basic information and the type of training on the displayed UI screen. The input information is sent to the control system 101 via the network 30 and stored in the information storage unit 17 . As a result, the content for executing the training program is activated and delivered to the spectacles type device 201 .

[Step S2]
Next, the control system 101 delivers guidance mode content to the glasses-type device 201, and the guidance mode is implemented. In the guidance mode, the learner performs work actions according to the guide displayed on the display unit 19, thereby confirming the work actions to be performed by the learner 102 in the training mode to be executed later, confirming the contents displayed, and preparing for the training. You can learn how to use the content, such as motivation. In this mode, the skill level of the learner 102, the difference between the work motion of the learner and the work motion of the instructor, how the proficiency level determination is displayed, the difference between each training mode, etc. 102. For example, the learner 102 can experience and learn how the information is displayed when the learner's work motion deviates from the instructor's work motion during the training mode. Note that this guidance mode may be implemented after the level determination mode, or may be implemented repeatedly until the learner masters how to use the content.

[Step S3]
Next, the control system 101 distributes the contents of the level determination mode to the spectacles type device 201 to implement the level determination mode. A specific flow of this mode will be described later, but in this mode, the learner works without reproducing the model moving image. The difference determination unit 16A and the level evaluation unit 16B evaluate the current proficiency level of the learner 102, and set the difficulty level of the next training mode (beginner mode, normal mode, etc.) according to the evaluation of the proficiency level. . The proficiency evaluation method is the same as the level evaluation method in each training mode.

[Step S4]
Next, training is performed in the training mode set in the level judgment mode. The specific flow in the training mode is as described with reference to FIG. pressure, motion trajectory, gaze point, sound), the difference between the motion of the learner 102 and the motion of the instructor included in the model moving image is obtained. At this time, as shown in FIG. 17, the difference calculator 163 assigns six motion elements to each unit time, and obtains the difference of each motion element divided for each unit time. The difference determination unit 16A determines whether the difference calculated for each motion element by the difference calculation unit 163 as shown in the spider chart of FIG. It is determined whether or not it is 116 or more. 16 C of display adjustment parts adjust the display content of a model moving image, when the difference of a learner and an instructor is more than a threshold value. It should be noted that the action elements (five action elements other than sound in this figure) for which the difference is equal to or less than the threshold value 116 are the action elements that the learner performed well during the training, and the action elements for which the difference exceeds the threshold value 116. The elements (sounds in this figure) are action elements that the learner did poorly.

[Step S5]
When the training mode ends, the control system 101 activates content for judging proficiency, and the level evaluation unit 16B judges the technical proficiency of the learner during training. Specifically, the level evaluation unit 16B uses the difference determination result by the difference determination unit 16A to determine how many motion elements among the six motion elements have a difference equal to or less than the threshold value 116, as shown in the spider chart of FIG. Evaluate the learner's proficiency according to

The relationship between the number of motion elements having a threshold value of 116 or less and the proficiency level can be set arbitrarily. For example, among six action elements, the number of action elements whose threshold is 116 or less may be the same as the number of levels. The evaluation unit 16B may determine the proficiency level of this learner as "level 5".

Also, in accordance with the determined proficiency level, the learning mode and the content to be learned intensively can be changed in the next learning performed by the same learner 102 . Also, for example, if the learner's level exceeds a certain level, such as when the difference between all motion elements is less than the threshold 116 in the training result in the beginner mode, the next training will proceed to the normal mode of the next level. can be

[Step S6]
Next, the proficiency level determined in step S5 is displayed to the learner by the glasses-type device 201, and the review mode is executed according to the proficiency level. In the review mode, the learner's work motions in the training mode in step S4 are reproduced for the learner 102, allowing the learner to objectively grasp the current skill level. In this step, it is preferable to display to the learner how much the proficiency level has improved from the initial proficiency level determined in step S3.

For example, when the visual field image of the learner captured by the camera 15 during the training mode is stored in the information storage unit 17, the superimposed display unit 162 reads out the stored visual field image of the learner in this mode. A retrospective video is displayed by superimposing a model video on the visual field video. In particular, the work action of the learner 102 that causes the action element whose difference is greater than or equal to the threshold 116 (the learner did poorly) may be recalled and played back. As a result, the learner can objectively grasp how much his current technique differs from the model technique, or where his weak points are.

In this review mode, the statistical calculation unit 16D generates a statistical graph (histogram, radar chart as shown in FIG. 17, etc.) based on the difference determination result of each motion element determined by the difference determination unit 16A, and learns from the statistical graph. The skill level of the person may be displayed. Also, instead of the statistical graph, for example, skill data and a weak point list as shown in FIG. By displaying statistical graphs and lists, the learner 102 can easily check the action elements that he or she is good at or weak in. This list and statistical graph are stored in the information storage unit 17 .

The retrospective mode may be run after the level determination mode, and is preferably run after each training mode. By implementing the retrospective mode after implementing the level determination mode and the training mode each time, the learner 102 can perform the next training while recognizing his/her current skill level.
Through the above steps, the learner 102 is trained to master the technique.

Here, a specific flow of the level determination mode in step S3 will be described with reference to FIGS. 15 and 19. FIG.

[Step S31]
When the level determination mode starts, the content of the level determination mode is displayed on the display unit 19 of the glasses-type device 201, and the camera 15 captures the learner's 102 visual field image from the learner's 102 visual field image. A field-of-view image obtained from the camera 15 is transmitted to the control system 101 . The sensing device 202 also measures the work motion of the learner 102 with each sensor and transmits the measured data to the control system 101 .

In the level determination mode, the level determination may be performed by the same work as the training mode that the learner later performs, or the level determination may be performed by basic work that is simpler than the training mode. When level determination is performed in the same work as the training mode, the number of modes stored in the information storage unit 17 can be reduced. On the other hand, when level determination is performed with basic work that is easier than the work performed in the training mode, the number of modes stored in the information storage unit 17 increases, but it is determined whether the level of the learner 102 has reached the basic level. be able to.

[Step S32]
Based on the information obtained from the camera 15, the difference calculation unit 163 calculates the difference between the work motion of the learner 102 and the model work motion (not displayed in this mode) for the six motion elements.

[Step S33]
The difference determination unit 16A determines whether or not the difference between each motion element and the hand of the learner 102 is equal to or greater than a threshold value, and transmits the determined difference determination result to the level evaluation unit 16B.

[Step S34]
The level evaluation unit 16B determines the proficiency level of the learner according to the difference determination result, and determines whether to train in the normal mode or in the beginner mode.

The level evaluation unit 16B determines to perform training in the normal mode when the learner's level is equal to or higher than a predetermined level, and determines to perform training in the beginner mode when the learner's level is equal to or lower than the predetermined level.

[Step S35]
The level evaluation unit 16B transmits the level determination result determined in step S34 to the glasses-type device 201, and the learner's level is displayed. In addition, it is preferable to execute content for reviewing the work actions of the learner in the level determination mode, similar to the review mode performed after execution of the training mode.

[Step S36]
The display adjustment unit 16C constructs training content according to the mode determined in step S34, and distributes the training content to the glasses-type device 201. FIG. After distributing the training contents, the training flow by the learning support system returns to step S4 in FIG. 14, and the training mode is executed. The display adjustment unit 16C may change the playback speed of the model moving image, or adjust the display by other methods besides the playback speed of the model moving image, according to the training mode to be constructed.

[Other display adjustment examples]
An example of adjusting the display of a model moving image in accordance with each mode and each operation element using methods other than those described above will be described below.
For example, when it is found from the difference determination result of the difference determination unit 16A that the difference between the point of gaze of the learner 102 and the point of gaze of the instructor is large and deviates from the threshold, the display adjustment unit 16C causes the display unit 19 to display The learner 102 can acquire the correct gazing point by displaying the correct gazing point for a long time or highlighting it.

Also, in the beginner mode, we explained that the instructor's movement speed is adjusted to match the learner's movement speed. There are other ways to drop it. For example, the display adjustment unit 16C may construct content that, when the learner's action is delayed by a certain amount or more, pauses the model moving image, and rewinds and reproduces it so as to start over from the previous step. In addition, although the normal mode has been described as a mode in which the learner trains according to the movement speed of the instructor, even in the normal mode, if the learner's work movement is delayed by more You may make a learner's work catch up with a model animation by pausing reproduction|regeneration of.

The display adjustment unit 16C may construct content so that, when each action element of the work action of the learner matches the action element of the instructor, the match is displayed to the learner. As the display method, display, sound, or vibration feedback can be used. If the method of displaying a match is the same for each action element, the learner can easily recognize that he or she has been able to perform the corresponding action element according to the model.

The display adjustment unit 16C, when the discrepancy between the learner's action element and the instructor's action element seen for a certain period of time is improved by a predetermined value or more (when the learner's skill is improved by a certain amount or more) Alternatively, the content may be structured so that when the learner 102 completes a series of work actions, the learner 102 is given feedback that makes them realize their progress. For example, the display adjustment unit 16C can display to the learner that the skill level has been improved by feedback such as display, sound, or vibration, or by highlighting the improved action element.

In addition, the display adjustment unit 16C provides the learner 102 with visual and auditory feedback by display and sound, and by the sensing device 202, constructs content that gives the learner tactile feedback such as vibration and impact. It is also possible to make the user feel as if they are actually working.

As described above, the learning support system of the first embodiment can dynamically change the display of the model moving image according to the learner's work motion. Therefore, each learner, such as a learner with a different level or a learner with different characteristics, can train the technique at a learning pace that suits them, making it easier to acquire the skillful technique.

In addition, the learning support system of the first embodiment extracts subtle tricks that are difficult to express in words, which are necessary for skill acquisition, as movement elements, and calculates the difference between the skill of the instructor and the learner for each movement element. Since the difference judgment result is displayed, the learner can grasp the advanced and detailed tricks of the technique and how well he or she has grasped the tricks for each action element. Since the difference determination result is displayed in a display method that allows the learner to learn intuitively, the learner can acquire the technique through intuitive actions.

In addition, in the learning support system of the first embodiment, information such as pressure and line of sight, which is originally difficult to understand by appearance, is superimposed on the visual field image of the learner. However, learners can learn while checking.

In addition, in the learning support system of the first embodiment, an example was shown in which one learner trains using the learning support program stored in the control system 101. It is possible to train at the same time or at different timings by using the glasses-type devices connected via the respective devices. Therefore, in the learning support system of the first embodiment, the advanced skills acquired by the instructor can be efficiently transmitted to many learners.

In addition, in this embodiment, two training modes, a beginner mode and a normal mode, are used to improve technique, but the training mode may be one mode or three or more modes. . Furthermore, the information storage unit 17 may store a mode effective for a learner's skill acquisition in addition to the modes described above.

Also, the internal configuration of the control system 101 (the calculation unit 16 and the information storage unit 17) may be stored in a cloud connected to the glasses-type device 201 via the network 30. FIG.

<<Learning support system of the second embodiment>>
Differences between the learning support system of the second embodiment and the learning support system of the first embodiment will be described below. Like the learning support system of the first embodiment, the learning support system of the second embodiment includes a glasses-type device 201 and a control system 101 connected to the glasses-type device 201 via a network. However, in the learning support system of the first embodiment, display adjustment of the model moving image displayed is performed using the teacher's model data already stored in the information storage unit 17 of the control system 101. In the learning support system of (1), new model data is generated from a teacher's model video that is not currently stored in the information storage unit 17, and the newly generated model data is used for display adjustment. In this respect, the learning support system of the second embodiment differs from the learning support system of the first embodiment.

As shown in FIG. 20, the learning support system of the second embodiment includes a glasses-type device 201 that can be worn by a learner 102 and a device that can be worn by a leader 103 who is away from the learner (for example, glasses and a glasses-type device 201 and a control system 101 connected to the glasses-type device 203 via a network 30 .

The spectacles-type device 203 has the same configuration as the spectacles-type device 201, and as shown in FIG. A display section 19B for displaying an image is provided. The glasses-type device 203 may also include a device for recording work sounds, a speaker for reproducing the work sounds of the instructor, and the like, if necessary.

In the second embodiment, the control system 101 has the same configuration as the control system 101 in the first embodiment. and an information storage unit 17 that stores the information.

Further, in the second embodiment, as shown in FIG. 22, the calculation unit 16 includes an image capture unit 161 that captures the visual field image of the learner 102 and the visual field image of the instructor 103, a superimposed display unit 162, and a difference calculation unit 163. , a difference determination unit 16A, a display adjustment unit 16C, and a model generation unit 164 that generates model data from the instructor's visual field image.

The difference between the flow of the training operation of the learning support system of the second embodiment and the flow of the training operation of the learning support system of the first embodiment will be described below with reference to FIG. First, the learner performs work for training. Camera 15 captures a visual field image of learner 102 (step S41). The image capture unit 161 receives the visual field image captured by the camera 15, and outputs the visual field image of the learner to the display unit 19B of the glasses-type device 203 of the instructor 103 (step S41B1). The instructor sees the training motion of the learner displayed on the display unit 19B, grasps the characteristics of the work motion of the learner, and performs a model work in a style that matches the characteristics of the work motion of the learner. . The camera 15B captures the visual field image of the instructor 103 while the instructor 103 is doing the model work. (Step S41B2). The image capturing unit 161 receives the visual field image captured by the camera 15B. The model generating unit 164 generates a model moving image from the visual field image of the instructor 103, and generates model data from the instructor 103 moving image (step S41B3). ). The model data newly generated from the model moving image of the instructor 103 is stored in the information storage unit 17, and used when the display adjustment unit 16C adjusts the display content according to the learner's work motion. In the second embodiment, steps after step S42 (steps S42 to S45) in which the display adjustment unit 16C superimposes and displays the model moving image and the visual field image of the learner are the same as in the first embodiment.

In the learning support system of the second embodiment, in addition to the effects obtained in the learning support system of the first embodiment, it is possible to utilize the instructor's technique, which is not stored in the information storage unit, for learning support of the learner. You can get the effect that you can. Further, in the learning support system of the second embodiment, it is possible to make the learner learn the skill of the instructor who is away from the learner.

In addition, in the learning support system of the second embodiment, it is assumed that a work video of the instructor that is not stored in the information storage unit is recorded to create a model video, and that the created model video is utilized for learning support of the learner. However, examples of enabling a learner to receive training support from a remote instructor are not limited to the above examples.

For example, the glasses-type device on the instructor's side and the glasses-type device on the learner's side may be sent to each other's video and audio. Specifically, on the instructor's glasses-type device, an image of the learner using the glasses-type device and performing training movements is displayed, and while the instructor watches the learner's training movements, Gives advice to the learner about work tips and the like through actions and voice. The instructor's action video data and voice data are sent to the learner's glasses-type device, and the instructor's action video is displayed on the learner's glasses-type device, and the instructor's voice is played back. be done. In this way, the learner can acquire skills more efficiently by transmitting advice from an instructor remote from the learner to the learner during training.

<<Learning support system of the third embodiment>>
Differences between the learning support system of the third embodiment and the learning support system of the first embodiment will be described below. In the learning support system of the third embodiment, artificial intelligence (AI) that has learned the skills of multiple instructors selects a model video that is closest to the characteristics of the learner's work motion as the learner's work motion progresses. , The model video displayed changes every hour.

Like the learning support system of the first embodiment, the learning support system of the third embodiment includes a glasses-type device 201 and a control system 101 connected to the glasses-type device 201 via a network.

In the third embodiment, the control system 101 has the same configuration as the control system 101 in the first embodiment, and includes a calculation section 16 and an information storage section 17 as shown in FIG. Further, the configuration of the calculation unit 16 in the third embodiment is similar to that of the calculation unit 16 in the first embodiment, and includes an image acquisition unit 161, a superimposed display unit 162, a difference calculation unit 163, and a difference determination unit 16A. , and a display adjustment unit 16C.

The information storage unit 17 in the third embodiment includes a storage unit 171 containing the learner's basic information, skill data, and a weak point list, and an artificial intelligence that changes the display of the model video using the technical features of a plurality of instructors. and a portion 172 . The artificial intelligence unit 172 has a learned model 172A that selects combination data having features closest to the features of the learner's work motion, and a result of selecting a model video to be displayed from the combination data selected by the learned model 172A. and a display unit 172B.

The learned model 172A is a model in which a large number of combined data of learner's characteristics and instructor's characteristics are learned in addition to model moving images. The trained model 172A is composed of, for example, a neural network. Combinations of the learner's features and the instructor's features learned by the learned model 172A include six motion elements of work motion (position, speed, pressure, motion trajectory, gaze point, and sound), basic information (for example, dominant hand , examination site for ultrasonic examination training, examination contents, patient's body type), etc. are included.

The operation flow of the learning support system of the third embodiment will be described below with reference to FIG.

First, the camera 15 captures the visual field image of the learner 102 . Next, the video capturing unit 161 captures the visual field video of the learner captured by the camera 15 via the network 30 . (Step S41). Next, the trained model 172A extracts the features of the work motion of the learner from the visual field image of the learner, and compares the features of the learner's motion with a large number of combined data learned by the trained model 172A. By doing so, combination data having features closest to those of the learner are selected (step S411). The result display unit 172B selects a model moving image having features closest to those of the learner from the combination data selected by the learning model 172A, and transfers it to the superimposed display unit 162. FIG. The superimposed display unit 162 superimposes the received model moving image on the visual field image of the learner (step S42). After displaying the model moving image that most closely resembles the characteristics of the learner's work motion, the flow of the motion returns to step S41, and this flow is repeated for each unit of time.

As described above, in the learning support system of the third embodiment, the model video closest to the features of the learner's work motion is changed and displayed at any time. be done. As a result, it is possible to provide training content that matches the characteristics of the learner at any time during training, and the learner can learn techniques more efficiently.

<<Modified example of the third embodiment>>
Another example of adjusting a model video using artificial intelligence (AI) that has learned the techniques of a plurality of instructors will be described with reference to FIG. 26 . In this modified example, the artificial intelligence unit changes the method of displaying the difference between the work motions of the learner and the instructor at any time according to the magnitude of the difference between the action elements for which the difference is greater than the threshold.

The learning support system of the modified example has the same configuration as the learning support system of the third embodiment, but differs in the contents of the combination data learned by the trained model 172A. In the learning support system of the modified example, the trained model 172A includes many combinations of the magnitude of the difference for each action element and the optimum method for displaying the magnitude of the difference. For example, regarding the difference between the elbow positions of the learner and the instructor, if the difference is less than a predetermined value, the text "Lift more elbows!" is displayed as shown in FIG. It is indicated by a sign such as 19b, and when the difference is large, a skeletal diagram 19c as shown in FIG. learned by the finished model 172A.

The learned model 172A also includes a model video, learner's characteristics, instructor's characteristics, and the like. Features of learners and instructors include six motion elements of work motion (position, speed, pressure, motion trajectory, gaze point, and sound), basic information (e.g., dominant hand, ultrasonic examination training) site, examination content, patient's body type, etc.).

The flow of operation of the learning support system of the modified example of the third embodiment will be described below.

In the flow of the learning support system of the modified example, steps S41 and S42 from when the camera 15 captures the visual field image of the learner 102 until the model moving image is superimposed on the visual field image are the same as those in the first embodiment ( Fig. 3). When the model video is superimposed on the visual field image of the learner, the difference calculation unit 163 extracts the six motion elements of the learner from the visual field video of the learner, and reads the six motion elements of the instructor from the model 172A. . Next, the difference calculation unit 163 calculates the difference between the actions of the learner and the instructor for each action element. (Step S43C).

Next, the difference determination unit 16A determines whether or not the difference calculated by the difference calculation unit 163 is equal to or greater than a threshold for each motion element, and notifies the artificial intelligence unit 172 of the determination result (step S44C).

Next, the learned model 172A captures the visual field image of the learner, and extracts the motion elements whose difference is equal to or greater than the threshold and the magnitude of the difference. The learned model 172A compares the extracted action elements and the magnitude of the difference between them with the combination data learned by the learned model 172A, thereby obtaining combination data including the method of optimally displaying the difference for each action element. Select (step S45C1). The result display unit 172B determines, from the combined data selected by the learning model 172A, the most effective method for training the learner as a method of displaying the difference between each action element, and performs display adjustment using that method. Then, an instruction is sent to the display adjustment unit 16C as follows (step S45C2). The display adjustment unit 16C adjusts the display method of the model moving image according to the instruction received from the result display unit 172B and causes the display unit 19 to display the model moving image (step S45). After the model moving image processed in step S45 is displayed to the learner, the flow returns to step S41.

As described above, in the learning support system of the modified example of the third embodiment, the difference between the learner's motion and the instructor's motion is displayed by the optimum display method for each motion element. You can easily see the diff and understand how you should change your behavior.

<<Learning support system of the fourth embodiment>>
Differences between the learning support system of the fourth embodiment and the learning support system of the first embodiment will be described below. As shown in FIG. 27, the learning support system of the third embodiment includes a spectacles-type device 201 and a control system 101 connected to the spectacles-type device 201 via a network 30. The network 30 further includes ultrasound waves. An imaging device 104 is connected.

The ultrasonic imaging device 104 may have the same configuration as a conventional ultrasonic imaging device. The images captured by the ultrasonic imaging apparatus are displayed on the display unit 19 of the spectacles-type device 201, for example, as shown in FIG. preferably.

When using the ultrasonic imaging apparatus, it is assumed that the user's viewpoint is not the hand holding the probe but the screen on which the captured image is displayed. Since the user cannot see the action at hand and the captured image at the same time, it is difficult for the learner who is unfamiliar with the technique to see the screen while operating the probe, which is one of the reasons why it is difficult to master the technique.

In the fourth embodiment, by displaying the image captured by the ultrasonic imaging device around the image at the learner's hand by the glasses-type device 201, the movement of the learner's 102 viewpoint is minimized. It is possible to master the movement of the hand and the resulting change in the photographed image while more intuitively grasping it.

In the second to fourth embodiments, the sensing device 204 described in the first embodiment may be connected to the control system 101, and the learner 102 may use the sensing device 204 during training. In addition, in the fourth embodiment, the probe of the ultrasonic imaging apparatus 104 incorporates a sensor similar to the sensor included in the sensing device 204 so that the learner 102 can learn techniques for handling the probe while using a real probe. may

<<Learning support device>>
In the above embodiment, an example in which the control system 101 is connected to the glasses-type device 201 via the network 30 has been described, but the configuration of the control system 101 is not limited to the above example. The control system 101 may reside within the memory 18 within the eyewear device 201 . The control system 101 may also be installed in a memory that can be externally attached to the glasses-type device 201 .

In this embodiment, the glasses-type device 201 is used as the learning support device, but the learning support device may include at least a camera and a display unit. It may be a combination of displays that are located nearby or a portable display.

Further, in the present embodiment, the case where the technique learned by the learner is the probe operation of the ultrasonic imaging apparatus was exemplified, but the present invention is applicable to learning in all technical fields. For example, it may be used for surgical methods, rehabilitation, sports, how to move the body in traditional performing arts, and nursing care methods. may be Furthermore, the present invention may be used for technical learning in industrial fields such as wiring work and welding work.

DESCRIPTION OF SYMBOLS 12... CPU, 13... Wireless communication device, 14... Motion sensor, 15... Camera, 16... Calculation part, 16A... Difference determination part, 16B... Level evaluation part, 16C... Display adjustment part, 16D... Statistical calculation part, 17... Information storage unit 18 Memory 19 Display unit 20 Line-of-sight sensor 30 Network 101 Control system 201, 203 Glasses type device 204 Sensing device

Claims (11)

Stores a display unit worn by the learner, an imaging unit worn by the learner that captures the visual field image of the learner, and a model video that is a video of the instructor's work motion that serves as a model for the work motion of the learner. and a computing unit,
The calculation unit executes a level determination mode and a training mode in which the model moving image is superimposed on the visual field image captured by the imaging unit and displayed on the display unit according to the determination result of the level determination mode. can be,
The calculation unit includes a difference calculation unit that calculates at any time a difference between the work motion of the instructor included in the model video and the work motion of the learner included in the visual field image, and the model video is displayed according to the difference. a display adjustment unit that changes the content,
The computing unit determines whether the training mode is performed in a beginner mode or in a normal mode, depending on whether or not the difference is equal to or greater than a threshold in the level determination mode. When the training mode is performed, the display adjustment unit changes the playback speed of the model animation so that the speed of the work movement of the instructor in the model animation follows the speed of the work movement of the learner, When the training mode is performed in the normal mode, the model moving image is reproduced at the speed of the work movement of the instructor, and the work movement of the learner lags behind the work movement of the instructor in the model animation by a certain amount or more, and the Only when the difference calculated from time to time by the difference calculation unit reaches or exceeds a threshold value, the playback speed of the model video is adjusted so that the speed of the work motion of the instructor of the model motion matches the speed of the work motion of the learner. to control
A learning support system characterized by: The difference calculation unit calculates the difference for each of a plurality of predetermined motion elements,
2. The learning support system according to claim 1 , wherein the display adjustment unit changes the display content of the model moving image according to the difference for each of the action elements. The difference calculation unit calculates the difference for each of a plurality of predetermined motion elements,
The display adjustment unit adjusts the display content of the model moving image so that, among the plurality of motion elements, the motion elements whose calculated difference exceeds a predetermined value are emphasized over the motion elements whose difference is equal to or less than a predetermined value. 2. The learning support system according to claim 1 , wherein the learning support system is changed. The difference calculation unit calculates the difference for each of a plurality of predetermined motion elements,
2. The display adjustment unit increases the transmittance of the model moving image displayed on the display unit for an operation element, of the plurality of operation elements, for which the difference is smaller than a predetermined value. The learning support system described in . The storage unit stores the learner's visual field image captured by the imaging unit,
2. The learning support system according to claim 1, wherein the calculation unit reproduces a retrospective video in which a model video is superimposed on the visual field video of the learner stored in the storage unit. further comprising a glasses-type device and a control system connected to the glasses-type device via a network;
The display unit and the imaging unit are provided in a glasses-type device worn by the learner,
2. The learning support system according to claim 1, wherein said calculation unit and said storage unit are provided within said control system. further comprising a sensing device connected to the control system via a network;
7. The learning support system according to claim 6 , wherein the calculation unit dynamically changes display content of the model moving image based on information detected by the sensing device. Further comprising a device that captures a leader's visual field image, connected to the control system via a network,
7. The learning support system according to claim 6 , wherein the calculation unit generates the model moving image from a visual field image of the instructor captured by the device. The storage unit stores an artificial intelligence unit that has learned a plurality of features of the work motion of the learner and features of the work motion of the instructor,
The artificial intelligence unit selects at any time a model video of the work motion of the instructor having features similar to the features of the work motion of the learner, and changes the model video to be superimposed on the visual field image at any time. Item 1. The learning support system according to Item 1. An ultrasound imaging device is further connected to the network,
7. The learning support system according to claim 6 , wherein the computing unit causes the display unit to display an ultrasonic image acquired by the ultrasonic imaging device. the computer,
A model video, which is a video of the instructor's working motion that serves as a model of the learner's motion, is superimposed on the visual field image captured by the imaging unit that is worn by the learner and captures the learner's visual field image, thereby learning. A program for displaying on a display unit worn by a person and functioning as a calculation means for dynamically changing the display contents of the model moving image according to the characteristics of the learner's work motion included in the visual field image, ,
The calculation means includes difference calculation means for calculating at any time a difference between the working motion of the instructor included in the model moving image and the working motion of the learner included in the visual field image, and displaying the model moving image according to the difference. and display adjustment means for changing the content,
The computing means executes a level determination mode and a training mode in which the model moving image is superimposed on the visual field image captured by the imaging unit and displayed on the display unit according to the determination result of the level determination mode,
The computing means determines whether the training mode is performed in a beginner mode or in a normal mode depending on whether or not the difference is equal to or greater than a threshold value in the level determination mode. When the training mode is performed, the display adjustment unit changes the playback speed of the model animation so that the speed of the work movement of the instructor in the model animation follows the speed of the work movement of the learner, When the training mode is performed in the normal mode, the model moving image is reproduced at the speed of the work movement of the instructor, and the work movement of the learner lags behind the work movement of the instructor in the model animation by a certain amount or more, and the Only when the difference calculated from time to time by the difference calculation unit reaches or exceeds a threshold value, the playback speed of the model video is adjusted so that the speed of the work motion of the instructor of the model motion matches the speed of the work motion of the learner. to control,
program.

Images