WO2020162272A1 - Information processing device and information processing method - Google Patents

Abstract

The present invention achieves a learning support system that determines whether or not it is necessary to guide a learner. An information processing device (100) comprises: a face information acquisition unit (1) for acquiring the face information of a learner; a viewed-object identification unit (2) for identifying viewed objects viewed by the learner by referring to the face information; and a guidance determination unit (3) for determining whether the learner requires guidance or determines the guidance requirement level of the learner, in accordance with the combination of viewed objects.

Description

Information processing apparatus and information processing method

The present invention relates to an information processing device and an information processing method.

A system that is used in educational institutions to support learners' learning is known as conventional technology. For example, Patent Literature 1 describes a technique of comparing a result of monitoring a readback frequency and a gaze stop of a learner with other learners and presenting the comparison result to the learner and the instructor.

Japanese Patent Laid-Open Publication "Japanese Patent Application Laid-Open No. 2005-338173 (published on December 8, 2005)"

However, the technique described in Patent Document 1 is for evaluating the degree of understanding of the learner, and whether or not the learner needs the instruction needs to be examined by the learner or the instructor himself based on the evaluation result. There is.

An aspect of the present invention is to realize a learning support system that visualizes a learning process of a learner and determines necessity of instruction for the learner to extract a learner who needs instruction by the system. And

In order to solve the above problems, an information processing apparatus according to an aspect of the present invention refers to a face information acquisition unit that acquires face information including information of at least a part of a learner's face, and the face information. , The necessity or guidance of the instruction to the learner according to the combination of the visual identification target specifying unit that identifies a plurality of visual recognition target objects visually recognized by the learner and the visual identification target object specified by the visual recognition target identification unit. It is characterized in that it is provided with a guidance judging unit for judging a necessity level.

In order to solve the above problems, the information processing method according to one aspect of the present invention refers to a face information acquisition step of acquiring face information including information of at least a part of a learner's face, and the face information. Then, depending on the combination of the visual identification target identifying step of identifying the plurality of visual identification objects visually recognized by the learner, and the combination of the visual identification objects identified in the visual identification target identifying step, it is necessary or unnecessary to instruct the learner. And a guidance determining step of determining a guidance necessity level.

Further, in order to solve the above problems, an information processing program according to an aspect of the present invention is an information processing program for causing a computer to function as any one of the information processing devices, the face information acquisition unit, A computer is caused to function as the visually recognizable object specifying unit and the guidance judging unit.

Further, in order to solve the above problems, an information processing apparatus according to an aspect of the present invention refers to a face information acquisition unit that acquires face information including information of at least a part of a learner's face, and the face information. Then, the visual recognition target specifying unit that specifies a plurality of visual recognition target objects visually recognized by the learner, and a visualization information generation unit that generates visualized information indicating the visual recognition target object specified by the visual recognition target specifying unit. It is characterized by having.

According to one aspect of the present invention, it is possible to determine the necessity of instruction or the level of instruction required for a learner according to the combination of the visual recognition objects visually recognized by the learner.

It is a block diagram which shows the component of the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows the example of the screen which the display part with which the information processing apparatus which concerns on one Embodiment of this invention was equipped displays. It is a figure showing an example of reference data which a guidance judgment part of an information processor concerning one embodiment of the present invention acquires. It is a figure which shows an example of the guidance necessity data which the guidance determination part of the information processing apparatus which concerns on one Embodiment of this invention outputs. It is a flow chart which shows an example of a flow of processing of an information processor concerning one embodiment of the present invention. It is a block diagram showing the important section composition of the guidance judgment part of the information processor concerning one embodiment of the present invention. It is a figure which shows an example of the target information which the instruction|indication judgment part of the information processing apparatus which concerns on one Embodiment of this invention acquires. It is a block diagram which shows the component of the information processing apparatus which concerns on other embodiment of this invention. It is a figure which shows an example of the information which the visualization information production|generation part of the information processing apparatus which concerns on other embodiment of this invention produces|generates.

[First Embodiment]
[Information processing device]
Hereinafter, an embodiment of the present invention will be described in detail. The information processing apparatus 100 is a learning support system that visualizes a learning process of a learner and determines whether or not guidance is required for the learner, thereby extracting a learner who needs guidance by the system. FIG. 1 is a block diagram showing components of an information processing device 100 according to an embodiment of the present invention. As illustrated in FIG. 1, the information processing device 100 includes a face information acquisition unit 1, a visual identification target identification unit 2, and a guidance determination unit 3. The information processing device 100 may include a state detection unit 4, a learner information acquisition unit 5, a storage unit 6, a display unit 7, and an image acquisition unit 8. Here, the visual recognition target identification unit 2, the guidance determination unit 3, and the state detection unit 4 are realized as one configuration of the control unit 9 of the information processing device 100.

With such a configuration, the information processing apparatus 100 can visualize the learning process of the learner and determine whether the learner needs the instruction.

The information processing apparatus 100 is mounted on, for example, a terminal device (for example, a smartphone, a tablet terminal, a personal computer, a television receiver, etc.) that can be connected to a local network or a global network.

(Face information acquisition unit)
The face information acquisition unit 1 acquires the learner's face information from an image acquired from an imaging unit such as a camera, for example. The learner's face information is information indicating the face feature amount. The feature amount of the face indicates, for example, position information indicating the position of each part of the face (for example, eyes, nose, mouth, eyebrows, etc.), shape information indicating the shape, size information indicating the size, and the like. In particular, it is particularly useful because it is possible to evaluate the learner's interest, the degree of interest, and the like from the eye information to the object the learner is gazing. The eye information includes, for example, the corner points of the inner and outer corners of the eye, the edges of the iris and the pupil, and the like. In addition, the face information acquisition unit 1 may appropriately perform correction processing such as noise reduction and edge enhancement on the image acquired from the imaging unit. The face information acquisition unit 1 transmits the extracted face information to the state detection unit 4.

(Status detector)
The state detection unit 4 detects the state of the learner based on the face information extracted by the face information acquisition unit 1. After detecting the learner's state, the state detection unit 4 transmits information about the detection result to the visual recognition target identification unit 2. The state of the learner detected by the state detection unit 4 is, for example, the state of each part of the face of the learner, and the line of sight of the learner, the state of the pupil, the number of blinks, the movement of the eyebrows, the movement of the cheek, At least one of eyelid movement, lip movement, and jaw movement.

As an example, the state detection unit 4 calculates the position information and the shape indicating the position of each part of the face (for example, eyes, nose, mouth, cheeks, eyebrows, etc.) which is the feature amount of the face acquired by the face information acquisition unit 1. Referring to the shape information and the size information indicating the size, etc., as the state of the learner, for example, the learner's line of sight, the state of the pupil, the number of blinks, eyebrow movement, cheek movement, eyelid movement, At least one of lip movement and jaw movement may be detected.

By thus providing the state detection unit 4, the information processing apparatus 100 can preferably detect the state of the learner.

The method for detecting the line of sight is not particularly limited, but for example, by providing a point light source (not shown) in the information processing apparatus 100 and capturing a corneal reflection image of light from the point light source for a predetermined time by the image capturing unit, learning is performed. There is a method of detecting the moving destination of the line of sight of the person. The type of point light source is not particularly limited, and examples thereof include visible light and infrared light. By using, for example, an infrared LED, it is possible to detect the line of sight without making the learner feel uncomfortable. When detecting the line of sight, if the line of sight does not move for a predetermined time or more, it can be said that the user is gazing at the same place.

Also, the method of detecting the state of the pupil is not particularly limited, but for example, a method of detecting a circular pupil from the image of the eye by using Hough transform can be cited. In general, humans tend to open their eyes when they are concentrated. Therefore, by detecting the size of the pupil, the degree of concentration of the learner can be evaluated. For example, it can be said that there is a high possibility that the learner is gazing at an object during the time when the size of the pupil is detected for a predetermined time and the pupil is growing within the predetermined time. With regard to the size of the pupil, a threshold may be set, and when the size of the pupil is equal to or larger than the threshold, it may be evaluated as “open”, and when the size of the pupil is smaller than the threshold, evaluation may be made as “closed”.

The method of detecting the number of blinks is not particularly limited, for example, by irradiating the learner's eyes with infrared light, to detect the difference in the amount of infrared light reflection between when the eyes are open and when the eyes are closed. And the like. Generally, when a person is concentrating, he or she tends to blink at a stable interval at a low frequency, so that the degree of concentration of the learner can be evaluated by detecting the number of blinks. For example, if the number of blinks is detected for a predetermined time and the blinks are performed at stable intervals within the predetermined time, it can be said that the learner is likely to be gazing at a target.

The state detection unit 4 detects at least one of the learner's line of sight, the state of the pupil and the number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and jaw movement. However, it is preferable to combine these. By combining the detection methods in this way, the state detection unit 4 can appropriately evaluate the degree of concentration of the learner while visually recognizing a certain object.

Other than the eye condition, for example, the movement of the eyebrow such as lifting the inside of the eyebrow or raising the outside of the eyebrow, the movement of the eyelid such as raising the upper eyelid and the tension of the eyelid, and the movement of the nose such as wrinkling the nose. , The state of each part of the face such as the movement of the lips such as lifting the upper lip and the purging of the lips, the movement of the cheeks such as lifting the cheeks, the movement of the jaws such as lowering the chin. As the state of the learner, the states of a plurality of parts of the face may be combined.

(Visual target specifying part)
The visual recognition target identification unit 2 identifies the visual recognition target object visually recognized by the learner based on the detection result acquired from the state detection unit 4. Specific processing of the visual recognition target specifying unit 2 will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a screen displayed by the display unit 7 included in the information processing apparatus 100. At least one of a question sentence and a question sentence is contained in a visual recognition target object. The visual recognition target object includes at least one specific part in the question sentence and the question sentence. Here, the specific part includes an underlined portion added to the question sentence or the question sentence. In the example shown in FIG. 2, the visual recognition target objects visually recognized by the learner are a question sentence, a question sentence, an underlined portion, and an option.

As shown in FIG. 2, the display unit 7 includes, for example, a question sentence display area 10, a question sentence display area 11, and an option display area 12. Then, the question sentence display area 10 includes an underlined area 13 in which an underline is displayed. The visual target specifying unit 2 first acquires the coordinates of these areas on the display screen from the image acquiring unit 8 described later.

The visual recognition target identification unit 2 collates the acquired coordinates of each region with the line-of-sight information acquired from the state detection unit 4, and identifies which region the coordinates of the learner's line-of-sight position are within. .. For example, as shown in FIG. 2, when the coordinates of the learner's line-of-sight position 14 is specified to be within the coordinates of the underlined area 13 of the question sentence display area 10, the visual recognition target specifying unit 2 determines that the learner is The counting of the time in which the coordinates of the line-of-sight position of stays within the coordinates of the underlined area 13 is started. In this way, the visual recognition target identification unit 2 identifies which visual recognition target object the learner is visually recognizing by collating the coordinates of the gaze position of the learner with the coordinates of each area of the display unit 7. .. Further, the visual target specifying unit 2 continuously specifies the visual target while the one problem is displayed on the display screen, and thereby specifies the combination of the visual target visually recognized by the learner regarding the problem. be able to.

In addition to the information on the line of sight, as described in (State Detection Unit), the state of the pupil, the number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and jaw movement. By referring to the motion detection result, it is possible to more appropriately specify which region the learner is concentrating and visually recognizing.

The visual target specifying unit 2 transmits information regarding the specified visual target to the guidance determining unit 3.

(Teaching judgment section)
The instruction determination unit 3 determines the instruction necessity or the instruction necessity level for the learner according to the combination of the visual target objects visually recognized by the learner, which are identified by the visual target specifying unit 2.

When the learner answers the question as shown in Fig. 2, there is a method of reading the question sentence and then the question sentence before answering, or a method of reading the question sentence first and then reading the question sentence before answering. In addition, there are various learning processes until the learner gets an answer. In addition, the learner may recognize an underlined portion in the question sentence or question sentence as an important portion, and may read the underlined portion before reading the entire question sentence or question sentence. In multiple choice questions that select and answer alternatives, reading from alternatives can be considered. It is also possible that the learner selects an option without reading the question sentence.

As described above, since the learning process by the learner is various, for example, the instruction need is high for an undesirable learning process, and the instruction is low for a desirable learning process. Needs to judge the necessity of teaching and the teaching method according to the learning process. The information processing device 100 visualizes the learning process of the learner by identifying the visual target object visually recognized by the learner, and determines the necessity of instruction or the level of instruction necessity for the learner according to the combination of the visual target objects. To do. As described above, by determining the necessity of instruction or the level of instruction necessity according to the combination of the visual recognition objects visually recognized by the learner, the learner needing the instruction can be extracted by the system to support the learning.

The instruction determination unit 3 determines the necessity of instruction or the level of instruction required for the learner according to the combination of the visual target objects identified by the visual target identification unit 2. As an example, the instruction determination unit 3 refers to reference data stored in advance in the information processing device 100 to determine the instruction necessity or the instruction necessity level. Further, the guidance determination unit 3 may determine the guidance necessity or the guidance necessity level according to a predetermined rule.

The manner in which the guidance determination unit 3 refers to the reference data and determines the guidance necessity or the guidance necessity level will be described. FIG. 3 is a diagram showing an example of reference data acquired by the guidance determination unit 3. The reference data is stored in the storage unit 6. In FIG. 3, “subject” is a subject learned by the learner, “level” is the level of necessity of learning by the learner, “personality” is the personality of the learner, and “permutation” is The combination order of the visually recognized objects is shown.

Depending on the subject to be studied, the learning process with high or low instructional needs may differ, and the level of instructional need may differ. Therefore, the reference data shown in FIG. 3 includes data for each subject. In the same learning process, if the instruction necessity level does not differ for each subject, the reference data may not include the item of the subject.

“The level of instruction requirement” indicates the degree of instruction required for the learner according to the learning process of the learner. For example, the necessity level of guidance for the learner is shown in five stages, and the necessity level of guidance is set to be lowest at level 1 of necessity of guidance, and the level of necessity of guidance is set to be highest at level 5 of necessity of guidance. It should be noted that the accumulated determination result of the instruction determination unit 3 may be fed back to update the instruction necessity level associated with the permutation. It should be noted that the reference data may include, instead of the “guidance necessity level”, only the guidance necessity of the guidance necessity or the guidance necessity, and may include an item of the “guidance necessity” which is not divided into levels.

The “Personality” item represents groups divided into groups according to the characteristics of the learner's personality. The instruction determination unit 3 further refers to the learner information of the learner to determine the instruction necessity or the instruction necessity level for the learner. Depending on the learner's personality, desirable or undesirable learning processes may differ. Therefore, the reference data shown in FIG. 3 includes data for each personality of the learner. For example, learners belonging to the group A of learners having a discreet personality and learners belonging to the group B of learners having a distracted personality may have different degrees of instructional needs according to the learning process.

More specifically, referring to FIG. 3, the learner with the personality A is at the level 1 of the necessity of instruction and needs the instruction when learning is performed in the order of question sentence-problem sentence-underlined part-option. Although the degree is low, learners with personality B are at the level 3 of necessity of instruction, and the need for instruction is higher. Even in the same learning process, the necessity of instruction and the necessity of instruction are determined according to the character of the learner. Fail level is different. If the learner's personality is not considered in the determination of the instruction necessity level, or if the instruction need level is not affected by the learner's personality, the reference data may not include the personality item. ..

“Permutation” is data in which the combinations of visible objects are arranged by distinguishing the difference in the visual order. Note that the reference data may include a “combination” item that represents a combination of visual recognition target objects and does not distinguish the difference in the visually recognized order, instead of the “permutation”.

When there are four question sentences, question sentences, underlined parts, and options as the visual target, the “permutation” may be only the data consisting of these four visual target objects, or the number of the visual target objects. It may be omitted, and the data may include only a part of the visual recognition target. For example, the “permutation” may be a permutation in which the underlined part is omitted, as in the case where only the question sentence, the question sentence, and the options are viewed and the underlined part is not seen. Further, the “permutation” may be data in which some of the visual recognition target objects are duplicated. For example, the “permutation” may be a permutation in which the question sentences are duplicated, as in the case of looking at the question sentence after seeing the question sentence and then looking at the question sentence again.

Next, the judgment of the guidance judgment unit 3 referring to the reference data will be explained. FIG. 4 is a diagram illustrating an example of guidance necessity data output by the guidance determination unit 3. In FIG. 4, "subject" is a subject learned by the learner, "problem" is a problem learned by the learner, "ID" is information for identifying the learner, and "personality" "Is the personality of the learner, "Permutation" is the permutation of the visual recognition target specified by the visual recognition target specifying unit 2, and "Teaching necessity level" is the guidance necessity level determined by the guidance determining unit 3. Is shown.

The instruction determination unit 3 determines the instruction necessity level of each learner for each problem. For example, the instruction necessity data shown in FIG. 4 shows the result of determining the instruction necessity level for each of the learners 0001, 0002, and 0003 for the problem 0001.

The items of “personality” and “permutation” of the guidance necessity data shown in FIG. 4 are linked to the items of “personality” and “permutation” of the reference data shown in FIG. That is, the teaching determination unit 3 identifies the personality (A) of the learner 0001 from the learner information acquired from the learner information acquisition unit 5, and learns from the information regarding the visual recognition target object acquired from the visual recognition target specification unit 2. Person 0001 specifies the permutation (underlined part-choice-question sentence-question sentence) when the person 0001 learned question 0001, refers to the reference data acquired from the storage unit 6, and finds the data with which the specified personality and permutation are associated. It is extracted and the level of instruction necessity is determined to be 5.

Similarly, the teaching determination unit 3 specifies the personality (B) and permutation (question sentence-question sentence-underlined portion-choice) for the learner 0002, refers to the reference data, and sets the teaching necessity level to 2. judge. In addition, the teaching determination unit 3 specifies the personality (A) and the permutation (question sentence-question sentence-underlined portion-choice) for the learner 0003, refers to the reference data, and determines the teaching necessity level to be 1. To do.

In this way, the instruction determination unit 3 refers to the reference data and determines the instruction necessity or the instruction necessity level according to the order in which the visually recognized objects are visually recognized.

Next, the judgment of the guidance judgment unit 3 according to the predetermined rule will be described. The predetermined rule is, for example, a point-decreasing method or a point-adding method rule such as -10 points when the underlined portion is first seen and -5 points when the question sentence is first seen. According to this rule, if the score is -10 or more, the level of instruction need is 5 with the highest need for instruction, and if the score is 0, the level of instruction is 1 with the need for instruction. Is also predetermined.

Therefore, for the learner identified by the learner information acquisition unit 5, the teaching determination unit 3 applies the predetermined rule acquired from the storage unit 6 to the permutation of the visual recognition target identified by the visual recognition target identification unit 2. It is applied and graded, and the level of instruction necessity is determined based on the graded result. For example, the teaching determination unit 3 sets the score of -10 for the learner 0001 because the underlined portion is first seen, and determines that the level is 5 for the necessity of teaching.

The guidance determination unit 3 may further determine the following items to determine the guidance necessity or the guidance necessity level, for example.

When answering in a time shorter than the minimum time required to read the entire sentence of question and answer options, the learner answered without reading the question sentence and option sentence It is more likely that there is a greater need for guidance. In this case, the guidance determination unit 3 may perform processing such as raising the guidance necessity level.

(Storage unit)
The storage unit 6 stores a program for operating the control unit 9 and reference data and various other data referred to by the guidance determination unit 3.

(Learner information acquisition section)
The learner information acquisition unit 5 acquires, for example, learner information of a specific learner registered in advance in a database or the like, and transmits the acquired learner information to the teaching determination unit 3. As an example, the learner information includes attribute information indicating learner attributes and learner identification information for distinguishing other learners from the target learner. The learner's attribute information is, for example, the learner's age, sex, personality (generally written, distracted, etc.). The learner identification information is, for example, the learner's ID, the learner's email address, or the like.

As described above, the learner information includes the learner identification information, so that it is possible to distinguish the other learner from the target learner, and it is possible to determine whether or not the learner needs the instruction or the instruction necessity level. , And learner information can be linked.

(Display)
The display unit 7 displays the visual target object visually recognized by the learner. Further, the display unit 7 may display the determination result of the guidance necessity or the guidance necessity level by the guidance determination unit 3. In the present embodiment, the configuration in which the information processing device 100 includes the display unit 7 has been described as an example, but the display unit may be configured separately from the information processing device 100.

When the display unit 7 displays the judgment result of the guidance necessity or the guidance necessity level by the guidance determination unit 3, even if the guidance necessity data output by the guidance determination unit 3 as shown in FIG. 4 is displayed as it is. Alternatively, display data including an object in which the guidance necessity data is visualized may be displayed. A graph is an example of the display data. The display data may indicate the instruction necessity level of one learner, or may display the average instruction need level of a plurality of learners together. Thereby, the level of instruction need of the learner can be grasped at a glance and the level of instruction need of the learner can be visualized.

Further, the display unit 7 may display the guidance necessity or the guidance necessity level by the guidance determination unit 3 and the suggestion information based on the guidance necessity or the guidance necessity level. Proposal information is, for example, information for prompting guidance or information for proposing a guidance method for a problem with a high guidance necessity level. Specifically, the suggestion information may be a message such as "It is necessary to teach about relative pronouns in English grammar.", "Please explain about pages 8 to 10 of the textbook again."

(Image acquisition unit)
The information processing apparatus 100 may further include an image acquisition unit 8 that acquires a captured image including the field of view of the learner from an imaging device such as a camera. Here, the imaging device is not particularly limited as long as it can detect the state of the learner, and examples thereof include a camera mounted on a glasses-type wearable device and a camera mounted on a head-mounted device. The types of these devices may be transmissive or non-transmissive. In the case of the non-transmissive type, for example, the camera (imaging device) and the display (display unit) to which the learner is visually recognizing may be provided in the same device.

The captured image including the learner's field of view is, for example, an image of the scenery in front of the learner, a display (display unit), and the like.

The visual recognition target identification unit 2 may identify the visual recognition target object visually recognized by the learner with reference to the captured image acquired by the image acquisition unit 8 and the result detected by the state detection unit 4. .. Specifically, the visual recognition target identification unit 2 collates the coordinates of the captured image area with the learner's line-of-sight information detected by the state detection unit 4, and the coordinates of the learner's line-of-sight position are determined in the captured image. Determine if it corresponds to the coordinates. The visual recognition target specifying unit 2 may be configured to recognize the target object included in the captured image acquired by the image acquiring unit 8 using, for example, machine learning.

With this, it is possible to suitably identify the visual target that the learner is visually recognizing.

(Control unit)
The control unit 9 centrally controls each unit of the information processing device 100. The control unit 9 includes the visual identification target specifying unit 2, the guidance determining unit 3, and the state detecting unit 4. Each function of the control unit 9 and all the functions included in the information processing apparatus 100 may be realized by the CPU executing a program stored in the storage unit 6 or the like, for example.

[Processing example of information processing device]
Next, the processing of the information processing apparatus 100 will be described based on the flowchart of FIG.

First, use of the information processing apparatus 100 is started, and processing is started (step S20). The process proceeds to step S (hereinafter, “step” is omitted) 21.

In S21, the learner information acquisition unit 5 acquires the learner information. Details of the processing are as described in (learner information acquisition unit). The learner information acquisition unit 5 transmits the learner information to the teaching determination unit 3, and proceeds to S22.

In S22, the face information acquisition unit 1 acquires the learner's face image from the imaging unit, and acquires the learner's face information from the face image (face information acquisition step). Details of the processing are as described in (Face information acquisition unit). The face information acquisition unit 1 transmits the acquired face information to the state detection unit 4, and proceeds to S23.

In S23, the state detection unit 4 detects the learner's line of sight, blink, pupil state, and state of each part of the face based on the face information extracted by the face information acquisition unit 1. The details of the processing are as described in (State detection unit). The detection result is transmitted to the visual target specifying unit 2, and the process proceeds to S24.

In S24, the visual recognition target specifying unit 2 specifies the visual recognition target object visually recognized by the learner based on the detection result acquired from the state detecting unit 4 (visual recognition target specifying step). The details of the processing are as described in (visual identification target specifying unit). The visual recognition target specifying unit 2 transmits the information regarding the specified visual recognition target object to the guidance determining unit 3, and proceeds to S25.

In S25, the instruction determination unit 3 acquires reference data from the storage unit 6 and proceeds to S26.

In S26, the instruction determination unit 3 refers to the information regarding the visual recognition target object acquired from the visual recognition target identification unit 2 and the reference data acquired from the storage unit 6 to determine the necessity of instruction or the level of instruction necessity for the learner. Yes (instruction determination step). The details of the processing are as described in the (teaching determination unit). The control unit 9 may store the guidance necessity or guidance necessity level determined by the guidance determination unit 3 in the storage unit 6. The guidance determination unit 3 transmits the determined guidance necessity or guidance necessity level to the display unit 7, and proceeds to S27.

In S27, the display unit 7 displays the guidance necessity level or the guidance necessity level acquired from the guidance determination unit 3, proceeds to S28, and ends the process.

[Specific configuration of instruction determination unit 3]
A specific configuration of the guidance determination unit 3 will be described with reference to FIGS. FIG. 6 is a block diagram showing a main configuration of the guidance determination unit 3 of the information processing apparatus 100 according to the embodiment of the present invention, and FIG. 7 is a guidance of the information processing apparatus 100 according to the embodiment of the present invention. It is a figure which shows an example of the target information which the determination part 3 acquires. The teaching determination unit 3 includes a feature amount conversion unit 31, an estimation result calculation unit 32, and a teaching method classification unit 33.

(Feature conversion unit)

The feature amount conversion unit 31 calculates the feature amount regarding the learner by referring to the order in which the visually recognized objects are viewed and the learner information of the learner. The feature amount conversion unit 31 calculates a distribution of attention of the learner as a feature amount. The order of visually recognizing the visually recognizable objects corresponds to the “permutation” shown in FIGS. 3 and 4. The learner information of the learner includes the “personality” shown in FIGS. 3 and 4. The feature amount calculated by the feature amount conversion unit 31 corresponds to the “level” shown in FIG.

The feature amount conversion unit 31 may calculate the feature amount by referring to the order in which the visually recognized objects are visually recognized and the sensing data including the learner information of the learner. The sensing data includes the learner's line of sight, the learner's personality, the learner's educational history, and the learner's labeling (evaluation) by the instructor. The sensing data is obtained from a video image during the simulation test, coordinate data of the learner's line-of-sight position, and the like.

The feature amount calculated by the feature amount conversion unit 31 represents the degree of understanding and motivation of the learner. When the feature amount conversion unit 31 calculates the feature amount, for example, a point or the like that the learner has a low understanding level is extracted and recorded. The feature amount conversion unit 31 calculates, as a feature amount, the distribution of the attention of the learner with respect to each visual target object in question, based on the line-of-sight stagnation time of each visual target object in question of the learner. For example, when the choice is longer than the question sentence, the feature amount conversion unit 31 determines that the learner's degree of understanding of this problem is low, and calculates the feature amount representing the determination result. .. The feature amount may be, for example, a five-level evaluation in which the level of understanding is set to level 1 and the level of understanding is set to level 5.

The order in which the visual target is visually recognized is specified by the visual target specifying unit 2 and sent to the feature amount conversion unit 31. The learner information is sent from the learner information acquisition unit 5 to the feature amount conversion unit 31. The feature amount conversion unit 31 may further acquire the data stored in the storage unit 6 and use it for calculating the feature amount. The feature amount conversion unit 31 may send the calculated feature amount and sensing data to the estimation result calculation unit 32 and also to the storage unit 6.

(Estimation result calculation unit)
The estimation result calculation unit 32 calculates the estimation result with reference to the feature amount calculated by the feature amount conversion unit 31. The estimation result calculation unit 32 may calculate the estimation result by referring to the sensing data together with the feature amount. The estimation result calculated by the estimation result calculation unit 32 is a prediction of the learner's performance in the next test, the learner's willingness in future classes, the probability of leaving school, the number of students who take the test, and the like.

For example, if the feature amount is level 1, the estimation result calculation unit 32 estimates that the learner's grade in the next test is good, and if the feature amount is level 5, the learner's grade in the next test is high. Estimated to be bad. The estimation result calculation unit 32 may send the estimation result to the teaching method classification unit 33 as well as to the storage unit 6.

(Teaching Method Classification Department)
The teaching method classification unit 33 refers to the estimation result calculated by the estimation result calculation unit 32 and the target information, and determines the necessity of teaching or the level of necessity of teaching the learner. The teaching method classification unit 33 may refer to the sensing data together with the estimation result and the target information, and determine the necessity of the guidance or the level of the necessity of the guidance for the learner. The teaching method classification unit 33 may send the determination result of the guidance necessity or the guidance necessity level to the display unit 7 and also to the storage unit 6. Since the transmission of information from the guidance determination unit 3 to the display unit 7 is shown in FIG. 1, the display unit 7 is not shown in FIG.

The instruction necessity or instruction necessity level determined by the instruction method classification unit 33 corresponds to the “instruction necessity level” shown in FIG. The instruction necessity level represents a determination result such as requiring more strict instruction, changing an instructor, or intensively instructing a specific subject. The instruction necessity level is changed according to the target information.

The target information is included in the target information data stored in the storage unit 6 in advance. An example of the target information data is shown in FIG. The goal information data includes the target grades of each subject for each goal set by the learner. For example, if the goal 1 is "passing XX university", the target score of "national language" required to achieve "passing XX university" is 100, and the goal of "mathematics" is 200. The goal of "English" is 150 points.

The teaching method classification unit 33 refers to the learner's goal included in the sensing data and the learner's estimation result, and if the difference between the score of the estimation result and the target score is large, the instruction necessity level is increased and the estimation is performed. If the difference between the result score and the target score is small, it may be determined that the instruction necessity level is lowered.

For example, when the learner's estimation result is “Japanese” 50 points, “Math” 80 points, and “English” 100 points, and the learner's goal is “Goal 1”, in particular, The difference is large. Therefore, the teaching method classifying unit 33 determines a teaching method that intensively improves the mathematics results and the selection of a teacher who is excellent in teaching mathematics. In addition, since the difference between the score of the English estimation result and the target score is small, it is judged that it is more efficient to give up the grade in the national language and mathematics and improve the grade in English, and improve the grade in English intensively. The selection of such a teaching method may be determined.

[Example of software implementation]
Even if the control blocks of the information processing apparatus 100 (particularly the state detection unit 4, the visual recognition target identification unit 2, and the guidance determination unit 3) are realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. It may be realized by software.

In the latter case, the information processing device 100 includes a computer that executes the instructions of a program that is software that realizes each function. The computer includes, for example, one or more processors and a computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes the program to achieve the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-transitory tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (Random Access Memory) for expanding the above program may be further provided. The program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[Second Embodiment]
Another embodiment of the present invention will be described below. For convenience of description, members having the same functions as the members described in the above embodiment will be designated by the same reference numerals, and the description thereof will not be repeated.

[Information processing device (2)]
FIG. 8 is a block diagram showing components of the information processing apparatus according to this embodiment. As shown in FIG. 8, the information processing apparatus 101 is different from the information processing apparatus 100 of the first embodiment in that the control unit 90 includes a visualization information generation unit 91.

The information processing apparatus 101 refers to the face information acquisition unit 1 that acquires face information including at least a part of information about a learner's face, and refers to the face information to identify a plurality of visual target objects visually recognized by the learner. The visual identification target specifying unit 2 for specifying the visual identification target specifying unit 2 and the visualization information generation unit 91 for generating the visualized information indicating the visual identification target specified by the visual identification target specifying unit are provided. The visualization information generation unit 91 acquires in time series the result of the visual target specifying unit 2 specifying the visual target. The visualization information generation unit 91 sends the generated visualization information to the display unit 7 and the storage unit 6.

The visualized information generated by the visualization information generation unit 91 may include information indicating which visible target object is visually recognized at each time. In addition, the visualized information generated by the visualization information generation unit 91 may include information regarding the time when each visual target is visually recognized within a predetermined time.

FIG. 9 is a diagram showing an example of visualized information generated by the visualization information generation unit 91. The visualized information is, for example, as shown in (a) of FIG. 9, with the time as the horizontal axis, which shows which question in the test (questions Q1 to Q5) was visually observed at each time. It is a graph of information. In FIG. 9A, the problem Q1 is visually recognized from time t ₁ to t ₂ , the problem Q2 is visually recognized from time t ₃ to t ₄ , the problem Q3 is visually recognized from time t ₅ to t ₆ , and the time t _{7 is shown.} viewing the problem Q4 to ~ _{t 8,} it is shown that had been viewing the problem Q5 to time _t 9 _{~ t 10.}

Further, the visualized information is, for example, as shown in FIG. 9B, information indicating how long each question in the test was visually observed during the total answer time of the test. In FIG. 9, (b) shows a pie chart showing the ratio of the time during which each question was visually recognized during the total answer time of the test.

In addition, the visualized information is time-series information indicating which visual target of the question sentence, question sentence, underlined portion, and option was seen at each time while solving one question, The information may be information indicating the proportion of time during which the question sentence, the question sentence, the underlined portion, and the options are visually recognized during the total answer time of one question.

[Summary]
An information processing apparatus according to one aspect of the present invention refers to a face information acquisition unit that acquires face information including information on at least a part of a learner's face, and the face information, and the learner visually recognizes the face information. The instruction determination unit that determines the necessity of instruction or the level of instruction necessity for the learner according to the combination of the visual target specifying unit that specifies a plurality of visual target objects and the visual target specified by the visual target specifying unit. It is characterized by having and.

According to the above aspect, it is possible to determine the necessity of instruction or the level of necessity of instruction for the learner according to the combination of the visual recognition objects visually recognized by the learner.

In the information processing device according to the one aspect, the instruction determining unit determines the instruction necessity or the instruction necessity level for the learner according to the order of visually recognizing the visual target.

According to the above aspect, it is possible to determine the necessity of instruction or the level of instruction necessity for the learner according to the order in which the learner visually recognizes the visually recognized object.

In the information processing device according to the above aspect, the visual recognition target object includes at least one of a question sentence and a question sentence.

According to the above aspect, it is possible to determine the necessity of instruction or the level of instruction necessity for the learner based on the information that the learner visually recognized the question sentence or the question sentence.

In the information processing device according to the above aspect, the visual recognition target object includes at least one specific part in the question sentence and the question sentence.

According to the above aspect, it is possible to determine the necessity of instruction or the level of instruction necessary for the learner based on the information that the learner visually recognizes a specific part in the question sentence or the question sentence.

In the information processing apparatus according to the one aspect, the teaching determination unit further refers to the learner information of the learner and determines the guidance necessity or the guidance necessity level for the learner.

According to the above aspect, by referring to the learner information of the learner, it is possible to more accurately determine the instruction necessity or the instruction necessity level for the learner.

In the information processing device according to the one aspect, the instruction determination unit calculates a feature amount regarding the learner by referring to an order in which the visual recognition target object is visually recognized and learner information of the learner. With reference to the conversion unit, the estimation result calculation unit that calculates the estimation result by referring to the feature amount, the estimation result, and the target information, the necessity or the necessity level of the instruction for the learner is determined. It has a teaching method classification department.

An information processing method according to an aspect of the present invention is a face information acquisition step of acquiring face information including information on at least a part of a learner's face, and the learner visually recognizes the face information with reference to the face information. A visual identification target identifying step of identifying a plurality of visual identification objects, and a guidance determination step of determining whether or not the learner needs guidance or a guidance necessity level according to a combination of the visual identification objects identified in the visual identification target identification step. It is characterized by including and.

According to the above aspect, it is possible to determine the necessity of instruction or the level of necessity of instruction for the learner according to the combination of the visual recognition objects visually recognized by the learner.

An information processing program according to an aspect of the present invention is an information processing program for causing a computer to function as any one of the information processing devices, and includes the face information acquisition unit, the visual recognition target identification unit, and the guidance determination unit. Make the computer function as.

According to the above aspect, it is possible to determine the necessity of instruction or the level of necessity of instruction for the learner according to the combination of the visual recognition objects visually recognized by the learner.

An information processing apparatus according to one aspect of the present invention refers to a face information acquisition unit that acquires face information including information on at least a part of a learner's face, and the face information, and the learner visually recognizes the face information. It is characterized by comprising a visual recognition target specifying unit that specifies a plurality of visual recognition target objects, and a visualization information generation unit that generates visualized information indicating the visual recognition target object specified by the visual recognition target specifying unit.

In the information processing device according to the above aspect, the visualized information includes information indicating which visible target object is visually recognized at each time.

In the information processing device according to the one aspect, the visualized information includes information regarding a time when each visible target object is visually recognized within a predetermined time.

The present invention is not limited to the above-described embodiments, but various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in the different embodiments Is also included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Face information acquisition unit 2 Visual recognition target identification unit 3 Guidance determination unit 31 Feature amount conversion unit 32 Estimated result calculation unit 33 Guidance method classification unit 91 Visualization information generation unit 100, 101 Information processing device

Claims (11)

1. A face information acquisition unit that acquires face information including at least a part of the learner's face;
   With reference to the face information, a visual identification target specifying unit that identifies a plurality of visual recognition objects visually recognized by the learner,
   An information processing apparatus, comprising: a guidance determination unit that determines whether or not guidance is required or a guidance necessity level for the learner according to a combination of the visual identification objects identified by the visual identification target identification unit.
2. The information processing apparatus according to claim 1, wherein the guidance determining unit determines a guidance necessity or a guidance necessity level for the learner according to an order in which the visual recognition target object is visually recognized.
3. The information processing apparatus according to claim 1, wherein the visual recognition target object includes at least one of a question sentence and a question sentence.
4. The information processing device according to any one of claims 1 to 3, wherein the visual target includes a specific part of at least one of a question sentence and a question sentence.
5. 5. The instruction determination unit further determines the necessity of instruction or a level of instruction necessity for the learner by further referring to the learner information of the learner. The information processing device described.
6. The teaching determination unit,
   A feature amount conversion unit that calculates a feature amount regarding the learner by referring to the order in which the visually recognized objects are visually recognized, and learner information of the learner,
   An estimation result calculation unit that calculates an estimation result with reference to the feature amount,
   The information processing according to claim 1, further comprising: a teaching method classifying unit that refers to the estimation result and the goal information to determine whether or not the learner needs guidance or a guidance necessity level. apparatus.
7. A face information acquisition step of acquiring face information including at least a part of the learner's face;
   A visual target specifying step of identifying a plurality of visual target objects visually recognized by the learner with reference to the face information;
   An information processing method, comprising: a guidance determination step of determining whether or not guidance is required or a guidance necessity level for the learner in accordance with a combination of the visual identification objects identified in the visual identification target identification step.
8. An information processing program for causing a computer to function as the information processing device according to any one of claims 1 to 6, wherein the computer is used as the face information acquisition unit, the visual recognition target identification unit, and the guidance determination unit. Information processing program to make it function.
9. A face information acquisition unit that acquires face information including at least a part of the learner's face;
   With reference to the face information, a visual identification target specifying unit that identifies a plurality of visual recognition objects visually recognized by the learner,
   An information processing device, comprising: a visualization information generation unit that generates visualized information indicating a visual target identified by the visual target identification unit.
10. The information processing apparatus according to claim 9, wherein the visualized information includes information indicating which visible target object is visually recognized at each time.
11. The information processing apparatus according to claim 9 or 10, wherein the visualized information includes information regarding a time period during which each visual target is visually recognized within a predetermined time period.

Images