JP2020115202A - Information processing apparatus, presentation system, and information processing program - Google Patents

Abstract

To achieve an information processing apparatus that can support learning of an operation related to serving customers performed by an operation learner.SOLUTION: An information processing apparatus (100) comprises: a face information acquisition unit (111) that acquires face information including information on at least part of the face of an operation learner who learns to serve customers; a voice information acquisition unit (114) that acquires voice information including information on a speech uttered by the operation learner; a difference information derivation unit (118) that derives first difference information indicating the difference between first reference information and the state of the at least part of the face indicated by the face information acquired by the face information acquisition unit (111), and second difference information indicating the difference between second reference information and the voice information acquired by the voice information acquisition unit (114); and a presentation information creation unit (119) that creates presentation information according to at least any one of the first difference information and second difference information.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an information processing method, a presentation system, and an information processing program.

In recent years, systems have been developed to assist workers in learning work content. For example, Patent Literature 1 discloses a work education system that displays a difference in the line of sight or work motion between a model worker and a learning worker based on image information.

JP, 2018-18090, A

On the other hand, in the customer service work and the like, the above-described conventional technique using only image information has a problem that it is not possible to appropriately support the acquisition of work content.

One aspect of the present invention is to realize an information processing apparatus capable of supporting the learning of a work learner regarding customer service work.

In order to solve the above problems, an information processing apparatus according to an aspect of the present invention includes a face information acquisition unit that acquires face information including information of at least a part of a face of a work learner who learns customer service, The difference between the voice information acquisition unit that acquires voice information including the information about the utterance by the work learner and the state of at least a part of the face indicated by the face information acquired by the face information acquisition unit and the first reference information First difference information indicating the first difference information and a difference information deriving unit deriving second difference information indicating a difference between the voice information acquired by the voice information acquiring unit and the second reference information, and the first difference information. And a presentation information generation unit that generates presentation information according to at least one of the second difference information.

According to one aspect of the present invention, it is possible to realize an information processing apparatus that can assist a work learner in learning about customer service work.

It is a block diagram which shows the component of the presentation system which concerns on one Embodiment of this invention. It is a figure which shows an example of the image which the display of the head mounted display which concerns on one Embodiment of this invention displays. 6 is a table showing an example of a learner's view target, utterance, action, and concentration degree specified by a view target specifying unit or a decision making information specifying unit according to an embodiment of the present invention. It is a table which shows an example of the data of a veteran customer previously stored beforehand in the storage part concerning one embodiment of the present invention. It is a table which shows an example of the data of the difference information which the difference information derivation part concerning one embodiment of the present invention derived. It is a flowchart which shows the process of the presentation system which concerns on one Embodiment of this invention. It is a figure which shows an example of the presentation information which the presentation information generation part which concerns on one Embodiment of this invention produced|generated. It is a figure which shows an example of the presentation information which the presentation information generation part which concerns on one Embodiment of this invention produced|generated. It is a block diagram which shows the component of the presentation system which concerns on one Embodiment of this invention. It is a flowchart which shows the process of the presentation system which concerns on one Embodiment of this invention. It is a figure which shows an example of the information regarding a visual recognition target object.

Hereinafter, an embodiment of the present invention will be described in detail. When the configuration in the following specific items (embodiments) is the same as the configuration described in other items, the description may be omitted. Further, for convenience of description, members having the same functions as the members shown in each item are denoted by the same reference numerals, and the description thereof will be appropriately omitted.

[Embodiment 1]
1. Presentation System The presentation system 800 according to the present embodiment provides learning support to a work learner who learns customer service work. As a form of the learning support device, for example, an experiential learning device using VR or the like can be mentioned. The presentation system 800 according to the present embodiment includes information of a work learner (for example, a novice part-time job; hereinafter, may be simply referred to as a learner) and a model worker (for example, a veteran attendant) stored in advance. By presenting the difference (also called "difference information") from the information (also called "reference information") to the learner as the presentation information, the work learner is supported for independent and experiential learning. It is possible to Hereinafter, a case where the presentation system 800 is used in a situation in which a learner who works in a restaurant or the like that needs to serve customers learns to serve customers will be described as an example.

The learner's information and reference information include, for example, an object to be visually recognized, a degree of concentration on the object to be visually recognized, utterance content, tone of voice, action, and the like.

FIG. 1 is a block diagram showing components of a presentation system according to an embodiment of the present invention. As shown in FIG. 1, the presentation system 800 according to the present embodiment includes an information processing device 100, a head mounted display 200, a wearable interface 300, an information management server 400, and a display unit 500. With such a configuration, the presentation system 800 according to the present embodiment can provide learning support to the learner.

<Head mounted display>
As shown in FIG. 1, the head-mounted display 200 includes a camera 210 (also referred to as an “imaging unit”) that captures at least a part of a learner's face, and a microphone 220 (“collecting sound” that collects a voice uttered by the learner. Unit)), a speaker 240 and a display 270 (both may be referred to as “presentation unit”), a motion sensor 230, a control unit 250, and a communication unit 260.

The control unit 250 of the head mounted display 200 centrally controls each unit of the head mounted display 200. Each function of the control unit 250 and all the functions included in the head mounted display 200 may be realized by the CPU executing a program stored in a storage unit (not shown) or the like, for example.

The communication unit 260 is controlled by the control unit 250 and communicates with the information processing device 100. The form of communication is not particularly limited, and may be communication using Bluetooth (registered trademark), WiFi (registered trademark), other short-range wireless communication technology, or other communication.

The camera 210 acquires an image of the learner's face as face information. The camera 210 may acquire an image of at least a part of the learner's body at the same time as the face image. The microphone 220 acquires the voice of the learner as voice information. The motion sensor 230 is, for example, an acceleration sensor, and acquires the learner's motion as motion information.

The speaker 240 outputs the sound input to the head mounted display 200 via the communication unit 260. The display 270 outputs the image input to the head mounted display 200 via the communication unit 260. The image output by the display 270 may display a virtual space, and the display example of the display 270 may be the image shown in FIG. FIG. 2 is a diagram showing an example of an image displayed on the display of the head mounted display according to the embodiment of the present invention. An image 200A shown in FIG. 2 is a view of the inside of a restaurant from the viewpoint of a learner who is a store clerk, and the image 200A shows customers, his/her own hands, a clerk other than himself, a table, and the like.

In addition, FIG. 3 illustrates the learner's visual target, the utterance (speech content and voice tone), the movement and the degree of concentration (blinking, the state of the pupil, the gaze retention time, and the visual target) of the learner identified by the visual target specifying unit or the decision making information specifying unit. It is a table showing an example of (expression).

<Wearable interface>
As shown in FIG. 1, the presentation system 800 includes a wearable interface 300. In one example, the wearable interface 300 includes a motion sensor 310 and a button 320, and inputs an operation performed by a learner to the information processing device 100. Examples of the operation performed by the learner include an operation of raising and lowering the motion sensor 310 and an operation of pressing the button 320.

In one example, the presentation system 800 can include more detailed information about the learner's hand movement and the like in the learner's movement information acquired by the movement information acquisition unit 115 in the information processing apparatus 100. Examples of the movement of the learner's hand in the present embodiment include a movement of holding a cup of water or placing it on a table, and a movement of taking an order for a meal.

Further, the information such as the learner's hand movement input via the wearable interface 300 can be reflected as the learner's hand movement or the like in the image output by the display 270 in the head mounted display 200.

Known sensors and buttons can be used as the motion sensor 310 and the button 320, and the motion sensor 310 is, for example, an acceleration sensor.

<Information processing device>
As shown in FIG. 1, the information processing apparatus 100 according to this embodiment includes a control unit 110, a storage unit 140, and a communication unit 130.

The control unit 110 of the information processing apparatus 100 according to the present embodiment includes a face information acquisition unit 111 that acquires face information including information on at least a part of the face of the work learner, and voice information that includes information about the utterance by the work learner. And the information stored in the storage unit 140 and the state of at least part of the face indicated by the face information acquired by the face information acquisition unit 111 (also referred to as “first reference information”). Difference information (also referred to as “first difference information”) indicating the difference between the voice information acquired by the voice information acquisition unit 114 and the information stored in the storage unit 140 (also referred to as “second reference information”). A difference information derivation unit 118 that derives difference information (also referred to as “second difference information”) indicating a difference between the call information and a presentation information generation unit 119 that generates presentation information according to the difference information. ..

The information processing apparatus 100 is mounted on, for example, a terminal device (for example, a smartphone, a tablet, a personal computer, a television receiver, or the like) that can be connected to a local network or a global network.

[Control part]
As illustrated in FIG. 1, the control unit 110 of the information processing apparatus 100 according to the present embodiment includes a face information acquisition unit 111, a state detection unit 112, a visual target identification unit 113, a voice information acquisition unit 114, a motion information acquisition unit 115, The decision information specifying unit 116, the learner information acquiring unit 117, the difference information deriving unit 118, and the presentation information generating unit 119 are provided. The control unit 110 centrally controls each unit of the information processing device 100. Each function of the control unit 110 and all the functions included in the information processing apparatus 100 may be realized by the CPU executing a program stored in a storage unit (not shown) or the like, for example.

(Face information acquisition unit)
The face information acquisition unit 111 acquires face information including information on at least a part of the learner's face at each time point. The face information acquisition unit 111 can acquire the learner's face information from the image acquired from the camera 210 included in the head mounted display 200, for example. The learner's face information is information indicating the face feature amount. The feature amount of the face indicates, for example, position information indicating the position of each part of the face (for example, eyes, nose, mouth, eyebrows, etc.), shape information indicating the shape, size information indicating the size, and the like.

In particular, it is particularly useful because the object visually recognized by the learner can be specified from the eye information. Examples of the eye information include the corner points of the inner and outer corners of the eyes, the edges of the iris and the pupil, and the like. In addition, the face information acquisition unit 111 may appropriately perform correction processing such as noise reduction and edge enhancement on the image acquired from the imaging unit. The face information acquisition unit 111 transmits the extracted face information to the state detection unit 112.

(Status detector)
The state detection unit 112 detects the state of the learner from the face information of the learner acquired by the face information acquisition unit 111. For example, the state detection unit 112 determines the object visually recognized by the learner at each time point (“visual recognition target” in FIG. 3 ), the number of blinks per minute at each time point (“blinking” in FIG. 3 ), each time point. The opening/closing of the pupil (“pupil state” in FIG. 3), the time during which the visual target is visually recognized (“gaze retention time” in FIG. 3), the facial expression (facial expression in FIG. 3), and the like are detected.

The state detection unit 112 detects the state of the learner based on the face information extracted by the face information acquisition unit 111. After detecting the state of the learner, the state detection unit 112 transmits the detection result to the visual target specifying unit 113.

As an example, the state detection unit 112 generates position information and shape indicating the position of each part of the face (for example, eyes, nose, mouth, cheeks, eyebrows, etc.) that is the feature amount of the face acquired by the face information acquisition unit 111. Referring to the shape information and the size information indicating the size, the learner's state, for example, the learner's visual recognition target, pupil state, number of blinks, eyebrow movement, cheek movement, eyelid movement , At least one of a lip movement and a jaw movement is detected.

As described above, by providing the state detection unit, the information processing device can preferably detect the state of the learner.

The method of detecting the visual target is not particularly limited, but the information processing apparatus 100 is provided with a point light source (not shown), and a corneal reflection image of the light from the point light source is captured by the image capturing unit for a predetermined time, thereby learning. There is a method of detecting a visual recognition target of a person. The type of point light source is not particularly limited, and examples thereof include visible light and infrared light. By using an infrared LED, for example, a visually recognizable target can be detected without giving a learner an unpleasant feeling.

As the state of the learner, the degree of concentration can be mentioned in addition to the visual recognition target. Generally, when a person is concentrating, he or she blinks at a stable interval with a low frequency, and the pupil tends to open. Therefore, by learning the number of blinks and the state (size) of the pupil, learning is performed. It is possible to evaluate the degree of concentration of personnel. For example, if the number of blinks is detected for a predetermined time and the blinks are performed at stable intervals within the predetermined time, it can be said that the learner is likely to be gazing at a target. Further, if the size of the pupil is detected for a predetermined time and the pupil is growing for a long time within the predetermined time, it can be said that there is a high possibility that the learner is gazing at an object.

The method of detecting the number of blinks is not particularly limited, for example, a method of irradiating the learner's eyes with infrared light and detecting the difference in the amount of infrared light reflection between when the eyes are open and when the eyes are closed. Etc.

The method of detecting the state of the pupil is not particularly limited, but for example, a method of detecting a circular pupil from the image of the eye by using Hough transform may be used. With regard to the size of the pupil, a threshold may be set, and when the size of the pupil is equal to or larger than the threshold, it may be evaluated as “open”, and when the size of the pupil is smaller than the threshold, evaluation may be made as “closed”.

Further, it is possible to detect that the visual recognition target does not change for a predetermined time or longer as an evaluation of the degree of concentration. For example, it is possible to detect the visual recognition target of the learner by using the above-described technique and measure the time during which the learner visually recognizes the object (“gaze retention time” in FIG. 3).

The state detection unit 112 detects at least one of the visual recognition target of the learner, the state of the pupil and the number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and jaw movement. However, it is preferable to combine these. By combining the detection methods in this way, the state detection unit 112 can appropriately evaluate the degree of concentration of the learner while visually recognizing a certain object.

Other than the eye condition, for example, the movement of the eyebrow such as lifting the inside of the eyebrow or raising the outside of the eyebrow, the movement of the eyelid such as raising the upper eyelid, or tightening the eyelid, the movement of the nose such as wrinkling the nose , The state of each part of the face such as the movement of the lips such as lifting the upper lip and the purging of the lips, the movement of the cheeks such as lifting the cheeks, the movement of the jaws such as lowering the chin. As the state of the learner, the states of a plurality of parts of the face may be combined.

The state detection unit 112 may calculate the degree of concentration of the learner by determining the learner's mood from the facial expression of the learner. The state related to the learner's mood, for example, as described above, refers to the facial feature amount, and as the state of the learner, for example, the learner's visual recognition target, the state of the pupil, the number of blinks, the movement of the eyebrows, At least one of the movement of the cheek, the movement of the eyelid, the movement of the lips, and the movement of the jaw can be detected, and the determination can be made based on the detected state of each part of the learner's face.

As a method of determining the degree of concentration of the learner from the facial expression, for example, when the eyebrows move upward, it can be determined that the learner is concentrating because he or she is gazing at the visual target. Further, for example, when the cheek moves upward while visually recognizing a person's face, it can be determined that the person is concentrating on the visually recognizable object because the facial expression is being made to the opponent.

When detecting the learner's facial expression, the state detection unit 112 can also calculate information regarding the learner's facial expression by machine learning. The specific configuration of the learning process for acquiring information about facial expressions does not limit the present embodiment, but for example, any of the following machine learning methods or a combination thereof can be used.

・Support Vector Machine (SVM)
・Clustering
・Inductive Logic Programming (ILP)
・ Genetic Algorithm (GP)
・Bayesian Network (BN)
・Neural network (NN)
When using a neural network, the data may be preprocessed and used for input to the neural network. For such processing, in addition to one-dimensional arraying of data or multi-dimensional arraying, for example, a method such as data argumentation can be used.

When a neural network is used, a convolutional neural network (CNN) including convolution processing may be used. More specifically, a convolutional layer that performs a convolutional operation is provided as one or more layers included in the neural network, and a filter operation (sum of products operation) is performed on input data input to the layer. It may be configured. Further, when performing the filter calculation, processing such as padding may be used together, or a stride width set appropriately may be adopted.

Further, as the neural network, a multi-layer or super multi-layer neural network having tens to thousands of layers may be used.

(Visual target specifying part)
The visual recognition target identification unit 113 identifies the target visually recognized by the learner with reference to the face information acquired by the face information acquisition unit 111 or the detection result acquired from the state detection unit 112. The object to be visually recognized (visually recognized object) may be a person or an object.

In addition, the visual recognition target specifying unit 113 specifies the degree of concentration of the work learner based on the detection result acquired from the state detecting unit 112.

As described above, the information processing apparatus 100 can accurately determine the target visually recognized by the learner and the degree of concentration by including the visual target specifying unit 113.

Specific processing of the visual recognition target identification unit 113 will be described. The visual recognition target identification unit 113 identifies the visual recognition target by determining the position coordinates of the tip of the learner's line of sight from the detection result acquired from the state detection unit 112. For example, the visual recognition target identification unit 113 identifies that the position coordinates of the learner's line of sight are within the coordinates of the foot of the customer, as shown in FIG. Further, in addition to the information on the line of sight, as described in (State detection unit), the state of the pupil, the number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and jaw movement. By referring to the motion detection result, it is possible to more appropriately specify which object the learner is concentrating and visually recognizing.

The visual recognition target identification unit 113 may identify which object the learner is concentrating and visually recognizing by using the machine learning method described in (state detection unit).

The visual recognition target identification unit 113 transmits the identified result to the difference information derivation unit 118.

(Voice information acquisition unit)
The voice information acquisition unit 114 acquires voice information including information about the utterance by the learner at each time point. The voice information acquisition unit 114 can acquire the voice information of the learner from the microphone of the head mounted display 200, for example. As the learner's voice information, for example, a voice frequency, a voice volume, etc. are acquired. A known voice recognition technique can be used to specify the voice information.

The voice information acquired by the voice information acquisition unit 114 is input to the decision information specifying unit 116.

(Motion information acquisition unit)
The motion information acquisition unit 115 acquires motion information indicating the motion of at least a part of the body of the work learner at each time point. The motion information acquisition unit 115 acquires from the motion sensor 230 of the head mounted display 200, the motion sensor 310 of the wearable interface 300, or the button 320, as shown in FIG. 1, for example. In addition, the camera 210 of the head mounted display 200 obtains the learned body from the captured image.

The motion information acquired by the motion information acquisition unit 115 is input to the decision making information specifying unit 116.

(Decision Information Identification Department)
Based on the voice information acquired by the voice information acquisition unit 114 and the motion information acquired by the motion information acquisition unit 115, the decision information specifying unit 116 specifies the decision information indicating the learner's intention at each time point. In the present embodiment, the decision-making information includes, for example, the utterance content of the learner, the tone of the voice, and the content of the action.

As an example, the decision making information specifying unit 116 specifies the utterance content and the tone of the voice uttered by the learner. For example, as shown in FIG. 3, the decision-making information identifying unit 116 identifies the utterance content as text indicating the utterance content such as "Welcome to you" or "What do you want to order?" The tone is specified as one of high, medium and low as an example.

Further, the decision making information specifying unit 116 refers to the motion information acquired by the motion information acquiring unit 115 and specifies the motion performed by the learner at each time point. For example, as shown in FIG. 3, the decision-making information identification unit 116 identifies an action such as "bow" or "put water".

Note that the decision making information specifying unit 116 may specify the decision making information by using the machine learning method described in (State detecting unit).

The decision information specified by the decision information specifying unit 116 is input to the difference information deriving unit 118.

(Difference information derivation part)
The difference information deriving unit 118 refers to the visual recognition target acquired from the visual recognition target specifying unit 113, the decision making information acquired from the decision making information specifying unit 116, and the information stored in the storage unit 140 with reference to the degree of concentration. The difference is derived as difference information. An example of deriving the difference information will be described with reference to FIGS.

FIG. 3 shows learner data identified by the visual recognition target identification unit 113 and the decision making information identification unit 116. FIG. 3 shows, as an example of the learner's data, the timing of visually recognizing the object, the visually recognizable object, the utterance content, the tone of the voice, the action, and the degree of concentration.

FIG. 4 is a table showing an example of data of veteran customers who is stored in the storage unit 140 in advance. FIG. 4 shows the timing of visually recognizing the object, the visually recognizable object, the utterance content, the tone of the voice, the motion, and the degree of concentration, as an example of the data of the experienced customer. The “first reference information”, the “second reference information”, and the “third reference information” in FIG. 4 correspond to the visual recognition target, the utterance, and the motion of the veteran customer, respectively.

In one example, the information processing apparatus 100 collects a series of decision-making (utterances and actions, etc.) that is performed within a predetermined time from the timing when the learner or the veteran customer visually recognizes the object (“event” in FIGS. 3 to 5). )). As shown in FIGS. 3 to 5, in one example, an event includes timing, a visual recognition target, an utterance, a motion, and a degree of concentration, and each event can be identified by an ID. As shown in FIGS. 3 to 5, the event of the learner and the event of the veteran customer can be associated using the event ID as an example. In the present specification, for example, as shown in FIG. 3, the data of the event ID 1 in the learner's data is displayed within a fixed time after the visual target and the visual object visually recognized at the timing of 00;00;03 are visually recognized. Including decision making.

Further, the “timing” data in FIGS. 3 and 4 is, for example, data indicating a time point when an object is visually recognized. ) May be an intermediate time point between the end time point and the end time point), or a combination of a plurality of these points may be used as data (not shown). Further, in one example, the difference information derivation unit 118 refers to the timing information acquired from the visual recognition target specifying unit 113 and the decision making information specifying unit 116, respectively, to determine the progress from visual recognition to decision making (utterance or action). The time may be calculated and included as data (not shown).

The difference information derivation unit 118 derives difference information for each event ID based on the learner data and the experienced customer service data shown in FIGS. 3 and 4, respectively. FIG. 5 is a table showing an example of difference information data derived by the difference information deriving unit 118. In FIG. 5, the difference information is information derived for each event ID, and indicates the difference in the timing of visually recognizing the object, the visually recognizable object, the utterance content, the tone of the voice, the action, and the concentration degree in the corresponding event ID. Show.

The “first difference information”, the “second difference information”, and the “third difference information” in FIG. 5 correspond to the differences in the visual recognition target, the utterance, and the action between the learner and the veteran, respectively.

As the difference information, information on “deviation” between the learner data shown in FIG. 3 and the veteran customer data shown in FIG. 4 (“+/−” or “+/−” in the table is used (learner -Veteran customer)) quantitative information) or information on whether or not it matches the data of veteran customer (qualitative information indicated by "○/X" in the table). In one example, as shown in FIG. 5, when the learner's data and the veteran customer's data do not match, the difference information derived by the difference information deriving unit 118 is the veteran customer's data (“ Correct answer”) may be included.

In one example, according to the event ID 1 in FIG. 3 showing the learner's data, at the timing “00;00;03”, the visual target identification unit 113 determines that the visual target of the learner is “the foot of the customer”. Has been identified. Further, according to the information of the veteran customer who is the information stored in advance in the storage unit 140, which is shown as the event ID 1 in FIG. 4, the visual recognition target of the veteran customer at the timing “00;00;03” is “customer”. The eyes". The difference information deriving unit 118 determines based on both data that the learner's visual recognition target does not match the veteran customer attendant, and as the difference information regarding the visual recognition target item, as shown in event ID 1 in FIG. Derive. Further, the difference information derivation unit 118 derives the “customer's eyes”, which is the visual target of the veteran customer, as the correct answer.

Further, as shown in event ID1 of FIG. 3, at timing “00;00;03”, the decision-making information identifying unit 116 speaks “Welcome” to the learner and performs a “bow” action. Have specified that. Further, as shown in event ID 1 in FIG. 4, according to the information stored in advance in the storage unit 140, the experienced customer utters "Welcome" at the timing "00;00;03" and "Given a bow." The operation of "is performed. The difference information deriving unit 118 determines that the utterance content and the action of the learner match those of the veteran customer based on the both data, and as shown in the event ID 1 of FIG. 5, the difference between the utterance content item and the action item. “O” is derived as information.

Then, as shown in event ID1 of FIG. 3, at the timing “00;00;03”, for example, the decision-making information identifying unit 116 identifies that the tone of the learner's voice is “medium”. .. Further, as shown in event ID1 of FIG. 4, according to the information stored in advance in the storage unit 140, the tone of the voice of the veteran customer at the timing “00;00;03” is “high”. The difference information derivation unit 118 determines that the tone of the learner's voice is lower than that of the veteran customer based on the both data, and as the difference information regarding the item of the voice tone, as shown in event ID1 of FIG. −” is derived.

Further, at the timing "00;00;03", the visual recognition target identification unit 113 identifies that the learner's blink (times/minute) is "5" as shown in event ID1 in FIG. .. Further, as shown in event ID1 in FIG. 4, according to the information stored in advance in the storage unit 140, the blink of the experienced customer at the timing “00;00;03” is “10”. Based on both data, the difference information derivation unit 118 determines that the learner's blink is 5 less than the experienced customer attendant, and as shown in event ID 1 in FIG. 5, “-5” is set as the difference information regarding the item of the blink. Derive.

At timing “00;00;05”, the visual recognition target identification unit 113 identifies that the state of the learner's pupil is “closed”, as indicated by event ID2 in FIG. Further, as shown in event ID2 in FIG. 4, according to the information stored in advance in the storage unit 140, the blink of the experienced customer at the timing “00;00;05” is “open”. The difference information derivation unit 118 determines based on both data that the pupil state of the learner does not match the veteran customer attendant, and as the difference information regarding the item of the pupil state, as shown in event ID2 of FIG. X" is derived.

At timing “00;00;05”, the visual target specifying unit 113 specifies that the learner's line-of-sight hold time (seconds) is “0.0”, as indicated by event ID2 in FIG. .. Further, as shown in event ID2 of FIG. 4, according to the information stored in advance in the storage unit 140, the line-of-sight retention time of the veteran customer at the timing “00;00;05” is “0.8”. .. The difference information derivation unit 118 determines that the learner's line-of-sight stop time is 0.8 less than that of the veteran customer based on both data, and as shown in event ID2 of FIG. 5, the difference information regarding the line-of-sight stop time item. As a result, “−0.8” is derived.

Further, at the timing “00;00;05”, the visual target specifying unit 113 confirms that the eyebrow movement and the cheek movement are not detected in the learner's facial expression, as shown in event ID2 of FIG. It is specified as "-". On the other hand, as shown in event ID2 of FIG. 4, according to the information stored in advance in the storage unit 140, the facial expression of the experienced customer attendant at the timing “00;00;05” is that the movement of the eyebrows is “−”, The movement of the cheek is "upward". In this case, the difference information derivation unit 118 determines that the learner's eyebrow movement matches that of the veteran customer based on both data, and as shown in event ID2 of FIG. 5, the eyebrow movement item. “O” is derived as the difference information regarding. Further, the difference information deriving unit 118 determines that the learner's cheek movement does not match the experienced customer attendant based on both data, and as shown in event ID2 of FIG. 5, the difference information regarding the cheek movement item. "X" is derived as.

Further, according to the visual recognition target specifying unit 113 and the decision making information specifying unit 116, the learner visually recognizes the "customer table" at the timing "00;00;40" as shown in event ID3 of FIG. , "What would you like to order?" On the other hand, as shown in event ID3 in FIG. 4, according to the information stored in advance in the storage unit 140, the "customer table" is visually recognized and "What are you ordering?" is uttered. The timing of performing the "put water" operation is "00;00;33". Based on both data, the difference information deriving unit 118 determines that the series of data of the learner is 00:00;07 later than the veteran customer, and as shown in event ID3 of FIG. “+00;00;07” is derived as the difference information regarding.

As shown in event ID 4 of FIG. 4, according to the information stored in the storage unit 140 in advance, the experienced customer sees “own hand” at the timing “00;00;47”, I will confirm your order" and perform the action of "repeat your order". On the other hand, as shown in FIG. 3, neither the visual recognition target specifying unit 113 nor the decision making information specifying unit 116 specifies the learner's visual recognition target and decision making information at the timing “00;00;47”. In addition, regardless of the timing, neither the visual recognition of "your own hand", the utterance "I confirm the order", or the operation "repeat order contents" is not specified. In such a case, the difference information derivation unit 118 determines, based on both data, that there is no data corresponding to the veteran customer data in the learner's data, and as shown at 600 in FIG. As information, "no data", "x", and correct answer information are derived.

From the table of FIG. 5, it is determined that the learner does not look at the eyes of the customer and tends to take the order later. Further, from the table of FIG. 5, it is determined that the learner has not confirmed the order with the customer.

The difference information derivation unit 118 may derive the difference information using the machine learning method described in (State detection unit).

The difference information derived by the difference information deriving unit 118 is input to the presentation information generating unit 119.

(Learner information acquisition section)
The learner information acquisition unit 117 acquires, for example, learner information managed by the information management server 400 as learner information. As an example, the learner information includes attribute information indicating the learner's attributes and learner identification information for distinguishing other learners from the target learner. The learner's attribute information is, for example, the learner's age, sex, work history, and the learning period up to now using the presenting system 800. The learner identification information is, for example, a learner's ID or a learner's email address.

In this way, since the learner information includes the attribute information and the learner identification information, the presentation information generation unit 119, which will be described later, makes an effective presentation for each learner according to the attribute of each learner. Information can be presented. Details will be described in the item of (presentation information generation unit).

The learner information acquisition unit 117 inputs the learner information acquired from the information management server 400 to the presentation information generation unit 119.

(Presentation information generator)
The presentation information generation unit 119 uses the difference information derived by the difference information derivation unit 118 (the visual recognition target and the degree of concentration identified by the visual identification target identification unit 113 from the face information, and the utterance and movement identified by the decision making information identification unit 116 from the voice information) The presentation information is generated based on the information derived from the action specified from the information). In one example, the presentation information generated by the presentation information generation unit 119 is an image.

The presentation information generation unit 119 generates presentation information in which the target visually recognized by the veteran customer is highlighted when the target visually recognized by the learner is different from the visually recognized target. FIG. 8 is a diagram showing an example of the presentation information generated by the presentation information generation unit 119. An image 200C shown in FIG. 8 is an image in which an object to be visually recognized by the learner is highlighted, and an arrow pointer 200C1 is added as a symbol to the image 200A shown in FIG. In FIG. 8, the arrow pointer 200C1 has an arrow shape, but may have any shape and color.

In addition, as examples of highlighting of the visual target, in addition to surrounding the visual target, changing the background color around the visual target, displaying the visual target larger than other images, and For example, displaying an image as a pop-up window.

The presentation information generated by the presentation information generation unit 119 may always include highlighted display, and, for example, the presence or absence of emphasis and the degree thereof may be appropriately changed according to the difference information derived by the difference information derivation unit 118. Good.

The presentation information generation unit 119 may calculate a score according to the difference information derived by the difference information derivation unit 118, and may include the calculated score in the presentation information.

FIG. 7 is a diagram showing an example of the presentation information generated by the presentation information generation unit 119. The image 200B shown in FIG. 7 includes an instruction window 200B1 and a score window 200B2 as text.

The instruction window 200B1 in the image 200B is a window for displaying a text indicating a utterance content and an action to be performed by the learner. The content of the guide is based on the data of the experienced customer who is stored in the storage unit 140 in advance. As shown in FIG. 7, the text may indicate all of the utterance contents and actions that the learner should perform, for example, “A guest raised his face from the menu, what should you do? ", etc. may be used to give a hint to the learner.

The score window 200B2 in the image 200B is a window for displaying the score indicating the proficiency level of the learner. As a score calculation method, for example, (i) the difference information determined by agreement or disagreement and the difference information determined by the degree of deviation are added or subtracted, and a predetermined score is given for the disagreement item. For items that have been deducted and deviated, the method of changing the deduction points according to the degree of deviation, and (ii) not only the difference information for judging the degree of deviation, but also the difference information for judging whether they match or not If it does not deviate from the specified range (for example, a veteran customer's visual target is the "customer's eye", whereas the learner's visual target is the "customer's nose", the utterance of the veteran customer The content is "What do you want to order?", but the content of the learner's utterance was "Ask for your order, etc.)" Examples of the method include, but are not limited to.

In one example, the presentation information generated by the presentation information generation unit 119 includes sound (not shown). The voice may be a voice or a sound effect. In one example, the voice reads the text displayed in the instruction window 200B1. Also, as an example of the sound effect, a sound such as a sound indicating a correct answer or an incorrect answer (such as "ping-pong" or "boo") depending on the content of the difference information, and the learner's attention along with the highlighting of the visual recognition target Sounds that urge you can be mentioned.

The presentation information generation unit 119 refers to the learner information acquired by the learner information acquisition unit 117 when generating the presentation information. By referring to the learner information, the presentation information generation unit 119 can present the presentation information such that each learner can effectively learn according to the attribute of each learner.

For example, when the learner in the present embodiment has never experienced hospitality business, the presentation information generation unit 119 generates the presentation information in which the visual recognition target is more emphasized, or a guide with more specific content. The presentation information including is generated. Further, for example, when the learner's previous learning period is longer than the predetermined period, the presentation information generating unit 119 generates the presentation information including the score indicating the learner's proficiency level calculated under a more severe condition. You can Further, in one example, the presentation information generation unit 119 can generate presentation information including a voice such as an anime character that the learner likes.

The presentation information generation unit 119 transmits the generated presentation information to the head mounted display 200. The presentation information generation unit 119 also transmits the presentation information to the display unit 500.

<Display>
As shown in FIG. 1, the presentation system 800 includes a display unit 500. The display unit 500 displays the presentation information generated by the presentation information generation unit 119, mainly for an instructor who teaches a learner. The image displayed by the display unit 500 may include the table showing the learner's data shown in FIG. 3 and the table showing the difference information shown in FIG. 5. Furthermore, the image of the veteran customer shown in FIG. It may include a table showing the data. The image displayed by the display unit 500 may include a part of the image displayed by the display 270 of the head mounted display 200.

2. Processing Example of Presentation System FIG. 6 is a flowchart showing processing of the presentation system 800. The processing of the presentation system 800 will be described based on FIG.

First, the use of the presentation system is started (step S100), and the process is started. The process proceeds to step S (hereinafter, “step” is omitted) 102.

In S102, the learner information acquisition unit 117 in the control unit 110 of the information processing apparatus 100 acquires the learner information (learner information acquisition step). Details of the processing are as described in (learner information acquisition unit). The learner information acquisition unit 117 transmits the learner information to the presentation information generation unit 119, and proceeds to S104.

In S104, the face information acquisition unit 111 acquires from the camera 210 of the learner's head-mounted display 200 and extracts each part of the face from the face image (face information acquisition step). Details of the processing are as described in (Face information acquisition unit). The face information acquisition unit 111 transmits the extracted result to the state detection unit 112, and proceeds to S106.

In S106, the state detection unit 112 detects the learner's visual recognition target, blink, the state of the pupil, and the state of each part of the face (state detection step). The details of the processing are as described in (State detection unit). The detection result is transmitted to the visual recognition target specifying unit 113, and the process proceeds to S108.

In S108, the visual recognition target identification unit 113 identifies a visual recognition target visually recognized by the learner (visual recognition target identification step). The details of the processing are as described in (visual identification target specifying unit). At this time, the visual recognition target specifying unit 113 may specify the degree of concentration while visually recognizing at the same time as specifying the visual recognition target. The visual recognition target specifying unit 113 transmits the specified result to the difference information deriving unit 118, and proceeds to S110.

In S110, at least one of the voice information acquisition unit 114 and the motion information acquisition unit 115 acquires the decision-making information of the learner from the microphone 220 and the motion sensor 310 included in the head-mounted display 200 and the like (decision-making information acquisition). Step). Details of the processing are as described in (Voice information acquisition unit) and (Motion information acquisition unit). It proceeds to S112.

In S112, at least one of the voice information acquisition unit 114 and the motion information acquisition unit 115 refers to whether voice information, motion information, or the like (hereinafter, sometimes referred to as “decision-making information”) is acquired, The presence or absence of decision information is determined. If there is decision-making information, the voice information acquisition unit 114 and the movement information acquisition unit 115 transmit the acquired decision-making information to the decision-making information specifying unit 116, and the process proceeds to S113. If there is no decision-making information, the process proceeds to S114.

In S113, the decision making information specifying unit 116 specifies the decision making information such as the learner's utterance content, voice tone, and motion (decision making information specifying step). Details of the processing are as described in (Decision making information specifying unit). The decision making information specifying unit 116 transmits the specified result to the difference information deriving unit 118, and proceeds to S114.

In S114, the difference information derivation unit 118 obtains reference information such as the visual recognition target of the veteran customer and decision-making information that is stored in advance in the storage unit 140 (reference information obtaining step). Proceed to S116.

In S116, the difference information deriving unit 118 refers to the visual recognition target specified by the visual recognition target specifying unit 113, the decision making information specified by the decision making information specifying unit 116, and the reference information acquired from the storage unit 140, and makes a difference. Deriving information (difference information deriving step). Details of the processing are as described in (difference information deriving unit). The difference information derivation unit 118 transmits the derived difference information to the presentation information generation unit 119, and proceeds to S118.

In S118, the presentation information generation unit 119 generates presentation information by referring to the difference information derived by the difference information derivation unit 118 and the learner information acquired from the learner information acquisition unit 117 (presentation information generation step). .. Details of the processing are as described in (presentation information generation unit). The control unit 110 of the information processing device 100 causes the communication unit 130 to transmit the presentation information generated by the presentation information generation unit 119 to the head mounted display 200 and the display unit 500, and proceeds to S120.

In S120, the control unit 250 of the head mounted display 200 causes the display 270 or the like to display the presentation information generated by the presentation information generation unit 119 (presentation information display step), the process proceeds to S122, and the process ends. At this time, the control unit 250 of the head mounted display 200 may output the presentation information generated by the presentation information generation unit 119 to the speaker 240 or the like as sound (not shown).

It should be noted that all the presentation information such as the instruction window 200B1 and the score window 200B2 in FIG. 7, and the arrow pointer 200C1 in FIG. 8 may not necessarily be displayed in real time during learning of the ongoing customer service, and for example, the current customer service The configuration may be such that the presentation information generated in the learning is displayed at the time of learning the customer service work after the next time. Further, the information processing apparatus 100 may have a function of recording the video displayed on the display 270 or the like in the storage unit 140, and may display each presentation information when the video is reproduced later. The same applies to Embodiment 2 described later.

[Embodiment 2]
1. Presentation system The presentation system 800a according to the present embodiment has a function equivalent to that of the presentation system 800 shown in FIG. Present to learners. As a result, it is possible to more appropriately support the work learner in independent and experiential learning. In the following, similar to the first embodiment, a case where the presentation system 800a is used in a situation where a learner working at a restaurant or the like that needs service is learning the service work will be described as an example.

FIG. 9 is a block diagram showing components of the presentation system according to the embodiment of the present invention. As shown in FIG. 9, the presentation system 800a according to the present embodiment further includes a difference analysis unit 120, which is different from the presentation system 800 shown in FIG. 1, in that the control unit 110a analyzes difference information. Further, the presentation information generation unit 119a according to the present embodiment generates presentation information by referring to the determination result input from the difference analysis unit 120 in addition to the difference information input from the difference information derivation unit 118. Further, the storage unit 140a according to the present embodiment has a judgment rule database 141 which is a database regarding judgment rules. Note that, as another aspect, the determination rule database 141 may not be included in the storage unit 140 but may be separately stored in a memory or the like (not shown).

(Difference Analysis Department)
The difference analysis unit 120 performs a determination process in which the determination rule read from the determination rule database 141 is applied to at least one of the first to third difference information derived by the difference information derivation unit 118.

Of the difference information shown in FIGS. 3 to 5, all the information about the “deviation” between the learner's data and the veteran customer's data, and the information on whether or not the data match the veteran customer's data are presented as the learner's information. On the contrary, it is difficult for the learner to understand which data should be focused on. Therefore, if the difference between the two data is within a predetermined threshold value or if a predetermined judgment criterion is satisfied, it may be preferable not to present the difference to the learner. The determination rule included in the determination rule database 141 may be understood as a data set that defines the threshold value or the determination criterion for each difference.

For example, at a certain timing, when the visually recognized object of the veteran customer is the customer's eyes, and the determination rule indicates within a predetermined range including the customer's eyes, the difference analysis unit 120 causes the learner at the timing. Even if the visual recognition target is not exactly the eyes of the customer, it may be determined that the visual recognition target is allowed as long as it is within the predetermined range, and the learner does not present the difference in the visual recognition target. On the other hand, when the learner's visual recognition target is his or her hand in the above case, the difference analysis unit 120 may determine that the learner should be presented with the difference of the visual recognition target. Note that the above-mentioned predetermined range that is allowed not to be presented to the learner may be a plurality of ranges that are not continuous.

Also, for example, at a certain timing, if the utterance content of a veteran customer is "What do you want to order", even if the utterance content of the learner is "I will ask your order", The difference analysis unit 120 may judge that the difference in the utterance content is not presented to the learner, as indicating the same content. On the other hand, when the learner's utterance content is “I will confirm your order” in the above case, the difference analysis unit 120 may determine that the learner should be presented with the difference in the utterance content. Good. Further, as in the above-described example, the difference analysis unit 120 may determine that the utterance content indicates the same content if the predetermined portions of the content match. Further, the determination rule database 141 may include a data set indicating the utterance content of the learner permitted for the utterance content of the veteran customer. Further, the threshold value of whether or not to present the learner about the level of the voice tone is not limited to the utterance content itself.

Further, for example, at a certain timing, when the motion of the veteran customer is “to show the front”, even if the motion of the learner is to “state the table number”, the difference analysis unit 120 does not change the difference analysis unit 120. May judge that the learner does not present the difference in the action. On the other hand, when the learner's action is “ask for an order” or the like in the above case, the difference analysis unit 120 may determine to present the learner with the difference in action. Further, the determination rule database 141 may include a data set indicating a learner's motion that is allowed in addition to a certain motion of an experienced customer.

In addition, the present invention is not limited to the above-described examples, and for example, the determination rule database 141 includes a threshold value for the difference in timing between the two actions, and the difference analysis unit 120 notifies the learner of the difference in timing that exceeds the threshold value. You may judge as what you present.

Further, the difference analysis unit 120 provides information indicating which visual target object the veteran customer and the learner are viewing at each time, and information regarding the time when each visual target object is viewed within a predetermined time. To generate. An example of each of the above information will be described later.

2. Processing Example of Presentation System FIG. 10 is a flowchart showing processing of the presentation system 800a. The processing of the presentation system 800a will be described based on FIG.

In the processing based on the flowchart of FIG. 10, the same processing as that of the first embodiment is executed from S100 to S116. After the processing in S116 is performed, the process proceeds to S117.

In S117, the difference analysis unit 120 performs the determination process in which the determination rule read from the determination rule database 141 is applied to at least one of the first to third difference information derived by the difference information derivation unit 118. .. Details of the processing are as described in (Difference analysis section). Further, the difference analysis unit 120 provides information indicating which visual target object the veteran customer and the learner are viewing at each time, and information regarding the time when each visual target object is viewed within a predetermined time. To generate. The difference analysis unit 120 transmits the determination result of the determination process and each of the above information to the presentation information generation unit 119a, and then proceeds to S118'.

In S118', the presentation information generation unit 119a refers to the determination result by the difference analysis unit 120 to generate the presentation information. In other words, the presentation information generation unit 119a generates presentation information for presenting the difference information that the difference analysis unit 120 has determined to present to the learner. In addition, the presentation information generation unit 119a may include the information regarding the visual recognition target object input from the difference analysis unit 120 and illustrated in FIG. 11 in the presentation information. FIG. 11A is a diagram showing which visual recognition target object the learner was visually recognizing at each time. Further, in FIG. 11, O1 to O5 each represent a visual recognition target object, for example, O1 corresponds to “a customer's eyes” or the like. In addition, FIG. 11B is a diagram showing the ratio of the time during which the learner visually recognizes each visual target object, which is information regarding the time during which each visual target object is visually recognized within a predetermined time period. The presentation information generation unit 119a may also generate the information illustrated in FIG. 11 for the visual target object visually recognized by the veteran customer. The presentation information generation unit 119a may also perform the same processing as the presentation information generation unit 119 in S118 of the first embodiment. Proceed to S120'.

In S120', the control unit 250 of the head mounted display 200 causes the display 270 or the like to display the presentation information generated by the presentation information generation unit 119a (presentation information display step), and proceeds to S122 to end the processing. At this time, the control unit 250 of the head mounted display 200 may output the presentation information generated by the presentation information generation unit 119a to the speaker 240 or the like as sound (not shown). Note that the presentation information displayed on the display 270 or the like may not be all information displayed on one screen, and for example, the information displayed may be switchable according to a user's instruction. In addition, in S118′, when the presentation information generation unit 119a generates the information illustrated in FIG. 11 also regarding the visual target object visually recognized by the veteran customer, the control unit 250 causes the veteran customer and the learner to learn. The corresponding information illustrated in FIG. 11 may be displayed on the display 270 or the like in a comparable manner. As a result, the learner can easily grasp the visual target object to which he or she should pay more attention, as compared with the data by the experienced customer.

[Example of software implementation]
The control block of the information processing device 100 (100a) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

In the latter case, the information processing apparatus 100 includes a computer that executes instructions of a program that is software that realizes each function. The computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes the program to achieve the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-transitory tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (Random Access Memory) for expanding the program may be further provided. The program may be supplied to the computer via any transmission medium (communication network or broadcast wave) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The information processing apparatus according to each aspect of the present invention may be realized by a computer. In this case, the information processing apparatus is converted into a computer by operating the computer as each unit (software element) included in the information processing apparatus. The control program of the information processing device realized by the above, and a computer-readable recording medium recording the program are also included in the scope of the present invention.

The present invention is not limited to the above-described embodiments, but various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in the different embodiments Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

100, 100a Information processing device 110, 110a Control part 111 Face information acquisition part 112 State detection part 113 Visual recognition target identification part 114 Voice information acquisition part 115 Motion information acquisition part 116 Decision information specific part 117 Learner information acquisition part 118 Difference information Derivation unit 119, 119a Presentation information generation unit 120 Difference analysis unit 130 Communication unit 140, 140a Storage unit 141 Judgment rule database 200 Head mounted display 210 Camera 220 Microphone 230 Motion sensor 240 Speaker 250 Control unit 260 Communication unit 270 Display 300 Wearable interface 310 Motion Sensor 320 Button 400 Information Management Server 500 Display

Claims (12)

A face information acquisition unit that acquires face information including at least part of the face of a work learner who learns customer service,
A voice information acquisition unit for acquiring voice information including information on the utterance by the work learner;
First difference information indicating a difference between at least a part of the state of the face indicated by the face information acquired by the face information acquisition unit and the first reference information, and the voice information acquired by the voice information acquisition unit and the first difference information. A difference information derivation unit that derives second difference information indicating a difference from the second reference information;
An information processing apparatus, comprising: a presentation information generation unit that generates presentation information according to at least one of the first difference information and the second difference information. With reference to the face information, the work learner further comprises a visual target specifying unit that specifies a target to be visually recognized,
The information processing apparatus according to claim 1, wherein the difference information derivation unit derives the first difference information with reference to the visual recognition target specified by the visual recognition target specifying unit. The visual recognition target specifying unit further specifies the degree of concentration of the work learner on the target visually recognized by the work learner,
The information processing apparatus according to claim 2, wherein the difference information deriving unit derives the first difference information with reference to the degree of concentration specified by the visual recognition target specifying unit. When the target visually recognized by the work learner is different from the target indicated by the first reference information, the presentation information generation unit generates presentation information in which the target indicated by the first reference information is highlighted. The information processing apparatus according to claim 2 or 3, characterized in that. The information processing apparatus according to any one of claims 1 to 4, wherein the presentation information generation unit generates presentation information according to a difference in voice indicated by the second difference information. 6. The presentation information generation unit calculates a score according to the first difference information and the second difference information, and includes the calculated score in the presentation information. The information processing apparatus according to item 1. Further comprising a motion information acquisition unit for acquiring motion information indicating the motion of at least a part of the body of the work learner,
The difference information deriving unit derives third difference information indicating a difference between the motion information and third reference information,
The information processing apparatus according to any one of claims 1 to 6, wherein the presentation information generated by the presentation information generation unit includes information according to the third difference information. A difference analysis unit that performs a determination process applying a determination rule to at least one of the first difference information, the second difference information, and the third difference information,
The information processing apparatus according to any one of claims 1 to 7, wherein the presentation information generation unit generates the presentation information with reference to a determination result by the difference analysis unit. The difference analysis unit generates information indicating which visible target object is visually recognized at each time,
The information processing apparatus according to claim 8, wherein the presentation information generation unit includes, in the presentation information, information indicating which visual recognition target object is visually recognized at each time. The difference analysis unit, within a predetermined time, to generate information about the time each visual recognition object is visually recognized,
The information processing apparatus according to claim 8 or 9, wherein the presentation information generation unit includes, in the presentation information, information regarding a time when each visually-recognized object is visually recognized within the predetermined time. A presentation system comprising the information processing device according to claim 1 and a head mounted display,
The head mounted display is
An imaging unit for imaging at least a part of the face of the work learner;
A sound collecting unit that collects the voice uttered by the work learner,
A presentation system comprising: a presentation unit that presents the presentation information. A program for causing a computer to function as the information processing apparatus according to claim 1.

Images