JP2014238792A - Learning support device and learning support method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a learning support device capable of efficiently performing learning by using a personal computer.SOLUTION: A learning support device 1 includes a display 4, a PC 3 for performing processing of an image to be displayed in the display 4, and a camera 2 for imaging the front of the display 4. The PC 3 displays an image picked up by the camera 2 in the display 4, and also displays a learning image 6 in at least one region of a plurality of regions formed by partitioning the image displayed in the display 4. Then, the PC 3 determines whether a hand of a user 5 is located in a region with the learning image 6 on the basis of the image picked up by the camera 2. Information based on the learning image 6 is provided when the hand of the user 5 is determined to be located in the region in which the learning image 6 is displayed.

Description

  The present invention relates to a learning support apparatus and a learning support method.

Conventionally, various apparatuses and methods using a personal computer have been proposed in order to effectively acquire language and the like.
For example, Patent Literature 1 discloses a learning support device that manages learners' learning histories individually and presents problem data to each learner according to the level of understanding of each learner.

JP 2004-157616 A The Institute of Electronics, Information and Communication Engineers, IEICE Technical Report ET97-87 (1997-12) Examination of the relationship between physical exercise and learning efficiency in computational and memory tasks

In the invention described in Patent Document 1, the learning support device analyzes the degree of understanding of each learner based on the correct answer rate of each learner and selects a question to be set according to the degree of understanding. Learning using such a device is merely to repeatedly solve a problem in a region that is not well understood, and is not significantly different from conventional learning methods. For this reason, there is still room for improvement as a learning method using a personal computer.
Such a problem is not limited to the invention described in Patent Document 1, and is generally common to apparatuses that perform learning efficiently using a personal computer.

  The present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to provide a learning support apparatus and a learning support method that can perform learning efficiently using a personal computer.

  A first aspect of the present invention is made to solve the above-described problem, and includes a display screen that displays an image, and an imaging unit that captures an imaging space including a user facing the display screen as a display image. A display unit that displays the display image on the display screen and displays the learning image in at least one of a plurality of regions formed by partitioning the display image; Determination means for determining based on the display image whether or not a predetermined part of the user is located in the area where the learning image is displayed, and the predetermined part is located in the area where the learning image is displayed. When it is determined through the determination means that the display means, the display means provides information based on the learning image.

  In the same configuration, when the user moves his / her body and positions a predetermined part in a region where the learning image is displayed, information based on the learning image is provided to the user. For example, sound corresponding to the learning image can be generated, or a visual change corresponding to the learning image can be generated. For this reason, according to this configuration, the user moves the body according to the display area of the learning image so that the predetermined position is located in the area where the learning image is displayed, and the user can move the body appropriately so that the predetermined part is moved to the area. When positioned, the user can obtain visual changes and auditory changes according to the learning image. Since the user can move the body to obtain a visual or audible change, the user can learn information related to the learning image from the visual or auditory sense while moving the body. In such learning while moving the body, the memory is easily fixed, so that the user can efficiently learn while having fun.

  A second aspect of the present invention is made to solve the above-described problem, and is a learning support apparatus according to the first aspect, wherein the determination means outputs from the imaging means before displaying the learning image. The display image to be stored is stored as a reference image, the display image output from the imaging unit after the learning image is displayed is stored as an evaluation image, the gradation value of the region in the reference image and the region in the evaluation image It is characterized in that it is determined whether or not the predetermined part is located in the region based on a two-dimensional correlation coefficient with a gradation value.

  A third aspect of the present invention is made to solve the above problem, and is a learning support device according to the second aspect, wherein the region is divided into a central part and an outer part excluding the central part. Whether or not the predetermined part is located in the region based on a two-dimensional correlation coefficient between the gradation value of the outer portion of the region in the reference image and the gradation value of the outer portion of the region of the display image It is characterized by judging.

  According to this configuration, since whether or not a predetermined part is located in a predetermined region is determined based on the gradation value of the outer portion, the sensitivity is higher than in the case of determining based on the gradation value of the entire region. . Therefore, it is possible to more accurately determine whether or not the predetermined part is located in the predetermined region as compared with the case of determining based on the gradation value of the entire region.

  A fourth aspect of the present invention is made to solve the above-described problem, and is a learning support apparatus according to any one of the first aspect to the third aspect, wherein the display unit is configured to display the display by a user. The learning image is displayed on the display screen so that the exercise intensity of the user when the body is moved in accordance with the learning image displayed on the screen falls within a predetermined range.

  When the heart rate during physical exercise increases due to light physical exercise, the learning efficiency tends to improve. However, when the heart rate increases excessively due to excessive physical exercise, it is becoming clear that the learning efficiency decreases. According to this configuration, since the learning image is displayed so that the exercise intensity falls within a predetermined range, the learning image can be displayed so that the learning efficiency is in a high range. Therefore, the user can learn efficiently.

  A fifth aspect of the present invention is made to solve the above-mentioned problem, and is a learning support apparatus according to any one of the first to fourth aspects, wherein a plurality of the imaging means are provided. The determination unit calculates a distance between the predetermined part and the display screen based on each display image captured through each imaging unit, and a predetermined distance between the predetermined part and the display screen. In this case, it is determined whether or not a predetermined part of the user is located in an area where the learning image is displayed.

  According to this configuration, it is possible to suppress misperception of a part other than the user as a predetermined part of the user and execution of the determination process.

  A sixth aspect of the present invention is made to solve the above-described problem, and captures an imaging space including a display unit that displays an image on a display screen and a user facing the display screen as a display image. A learning support apparatus including a plurality of imaging units, wherein the display unit displays a display image captured through one imaging unit of the plurality of imaging units on the display screen, and displays the display image on the display screen. Dividing the display image into a plurality of areas, selecting at least one of the plurality of areas, setting a predetermined range in coordinates extending in a direction perpendicular to the display screen, and selecting the selected area The learning image is displayed in a size corresponding to the set predetermined range, and based on each display image captured through the plurality of imaging means, a predetermined part of the user and the display screen And determining whether or not the predetermined part of the user is located in an area where the learning image is displayed when the distance between the predetermined part and the display screen is within the predetermined range. A process is executed.

  According to this configuration, it is possible to calculate the distance between the predetermined part of the user and the display screen. For this reason, the learning image can be displayed so that the user moves not only in the vertical and horizontal directions but also in the longitudinal direction.

  A seventh aspect of the present invention is made to solve the above-described problem, and is a learning support apparatus according to any one of the first to sixth aspects, wherein the imaging means captures infrared light. It is characterized by doing.

  According to this configuration, it is possible to suppress misperception of a part other than the user as a predetermined part of the user and execution of the determination process.

  An eighth aspect of the present invention is made to solve the above-described problem, and includes a display process for displaying an image, and an imaging process for capturing an imaging space including a user facing the display screen as a display image. A display step of displaying the display image on the display screen and displaying the learning image in at least one of a plurality of regions formed by dividing the display image; A determination step of determining whether or not a predetermined portion of the user is located in a region where the learning image is displayed based on the display image, and the display step displays the learning image at every elapse of a predetermined time. When the determination step determines that the predetermined part is located in a region where the learning image is displayed, the display step displays the learning image on the learning image. And providing a brute information.

  ADVANTAGE OF THE INVENTION According to this invention, the learning assistance apparatus which can learn efficiently using a personal computer can be obtained.

1 is a schematic configuration diagram of a learning support apparatus according to a first embodiment of the present invention. 6 is a flowchart showing an image processing procedure according to the embodiment. (A) is an enlarged view of a selected region in the reference image according to the embodiment, and (b) is an enlarged view of a selected region in the evaluation image according to the embodiment. The schematic block diagram of the learning assistance apparatus concerning 2nd embodiment of this invention. 6 is a flowchart showing an image processing procedure according to the embodiment. The schematic block diagram of the learning assistance apparatus concerning 3rd embodiment of this invention. 6 is a flowchart showing an image processing procedure according to the embodiment. The flowchart which shows the process sequence for calculating the three-dimensional coordinate of the user concerning the embodiment. The schematic diagram of the user and display concerning the embodiment. The schematic block diagram of the learning assistance apparatus concerning 4th embodiment of this invention. 6 is a flowchart illustrating an image processing method according to the embodiment. The schematic block diagram of the learning assistance apparatus concerning other embodiment.

(First embodiment)
Hereinafter, a first embodiment embodying the present invention will be described with reference to FIGS.

  First, a learning support apparatus 1 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 shows a schematic configuration of a learning support apparatus 1 according to the first embodiment of the present invention. This embodiment demonstrates the case where the learning assistance apparatus 1 is used for English learning.

  As shown in FIG. 1, the learning support apparatus 1 includes a camera 2 (corresponding to “imaging means” of the present invention) and a PC 3 (a “display means” of the present invention) as a personal computer connected to the camera 2. And a display 4 having a vertical width H and a horizontal width W (corresponding to a “display screen” of the present invention) connected to the PC 3.

  The connection method between the camera 2 and the PC 3 and the connection method between the PC 3 and the display 4 are not particularly limited, and for example, they may be connected via a wired or wireless network.

  The camera 2 is disposed at the upper center of the display 4 and images an imaging space including the user 5 facing the display 4. The imaging space of the camera 2 may be in a range in which the whole body of the user 5 standing up against the display 4 can be accommodated. In addition, the camera 2 images the imaging space by the camera 2 every elapse of the predetermined time Te.

  The PC 3 controls the operation of each part of the circuit by a central processing unit (CPU), and the control program is stored in advance in a ROM (not shown) of a memory mounted on the PC 3. The PC 3 executes various controls of the learning support apparatus 1 through this control program.

  An image of the imaging space captured at every elapse of the predetermined time Te by the camera 2 is output to the PC 3 each time as a display image and stored in the memory of the PC 3. The PC 3 displays the display image output from the camera 2 over the entire display 4 each time. That is, the latest image output from the camera 2 is displayed on the display 4. The PC 3 also includes a sound generation unit 3a that generates sound stored in advance in the memory of the PC 3 according to various controls.

  Such a PC 3 causes the learning image 6 to be displayed on the display 4 so as to be superimposed on the image captured by the camera 2. The learning image 6 is displayed at a predetermined point (for example, the lower left corner) of the image (display image) displayed on the display 4 as the origin, the horizontal direction is the X axis, and the direction perpendicular to the X axis is Y. Set on a coordinate system expressed as an axis.

Hereinafter, the processing procedure of the image output to the PC 3 will be described in detail with reference to FIG.
As shown in FIG. 2, when this process is started, first, the PC 3 starts measuring the learning time Tp (step S100). Next, a reference image is acquired, and an evaluation image is acquired and displayed on the display 4 (step S101). The reference image refers to an image that is captured by the camera 2 and output to the PC 3 at the time closest to the start of this process, among the images output from the camera 2 to the PC 3 as display images. The evaluation image indicates the latest display image, that is, the image displayed on the display 4 among the images output from the camera 2 to the PC 3 as display images.

  As described above, a predetermined coordinate system is set for the display image. Since both the reference image and the evaluation image are display images, the same coordinate system is set for the reference image and the evaluation image.

  Next, coordinates (X, Y) for displaying the learning image 6 are set (step S102). Specifically, the PC 3 selects one area from a plurality of areas (see FIG. 1) formed by dividing the evaluation image, and coordinates the center coordinates of the selected area (hereinafter, area B). Set as (X, Y).

  As described above, the same coordinate system is set for the reference image and the evaluation image. The PC 3 sets a region corresponding to the region B in the reference image (a region centered on the set coordinates (X, Y) in the reference image) as the region A.

In the present embodiment, when the display image is partitioned, each region is partitioned so as to be a rectangular region having a length in the X-axis direction of “m1” and a length in the Y-axis direction of “n1”. (See FIG. 3). The length “m1” in the X-axis direction is preferably in the range of 1/26 to 10/26 (W × 1/26 to W × 10/26) of the lateral width W of the display 4 (see FIG. 1). A range of 4/26 to 6/26 (W × 4/26 to W × 6/26) of the lateral width W is more preferable. The length “n1” in the Y-axis direction is preferably in the range of 1/20 to 8/20 of the vertical width H of the display 4 (H × 1/20 to H × 8/20). The range of 3/20 to 4.5 / 20 (H × 3/20 to H × 4.5 / 20) is more preferable.
Further, the regions A and B are divided into center regions Ac and Bc and outer regions Ae and Be excluding the center regions Ac and Bc, respectively. The center regions Ac and Bc are set in a rectangular shape having a length in the X-axis direction of “m2” and a length in the Y-axis direction of “n2”. The length “m2” in the X-axis direction is preferably in the range of 1/2 to 1/20 of m1 (m1 × 1/2 to m1 × 1/20), and is in the range of 1/4 to 1/10 of m1 ( m1 × 1/4 to m1 × 1/10) is more preferable. The length “n2” in the Y-axis direction is preferably in the range of 1/2 to 1/20 of n1 (n1 × 1/2 to n1 × 1/20), and is in the range of 1/4 to 1/10 of n1 ( n1 × 1/4 to n1 × 1/10) is more preferable.

  When the PC 3 displays the evaluation image on the display 4, as shown in FIG. 1, the image corresponding to the English word to be learned (here, the dog image) is coordinated (X, Y) in the region B. Is superimposed on the evaluation image. This dog image corresponds to the learning image 6 in the present embodiment.

In this way, the PC 3 displays the learning image 6 in the region B, and starts measuring the display time Td that is an elapsed time since the display of the learning image 6 is started (step S103).
Next, the PC 3 acquires an evaluation image and displays it on the display 4 (step S104). In this way, the latest image is used as the evaluation image every time step S104 is executed. For the evaluation image, the same coordinate system is set each time, and is equivalent to the area B and the outer area Be set in step S102. The areas are set as the area B and the outer area Be, respectively.

Next, the PC 3 calculates a two-dimensional correlation coefficient r between the outer region Be of the region B in the evaluation image and the outer region Be of the region A in the reference image (step S105).
In the present embodiment, the two-dimensional correlation coefficient r is calculated using the following formula 1.

  FIG. 3A shows an enlarged view of the region A in the reference image. FIG. 3B shows an enlarged view of the region B in the evaluation image. FIG. 3B shows a case where the hand of the user 5 (corresponding to the “predetermined part” of the present invention) is located in the region B.

  When the gradation value of the outer area Ae of the area A in the reference image is different from the gradation value of the outer area Be of the area B in the evaluation image, the period from when the reference image is captured until the evaluation image is captured In the meantime, some change occurred in the area, for example, as shown in FIG. 3B, a part of the body of the user 5 (in this case, a hand) enters the area where the learning image 6 is displayed. It is possible that a change has occurred.

  Therefore, in the present embodiment, a two-dimensional correlation coefficient r between the gradation value of the outer area Ae of the area A in the reference image and the gradation value of the outer area Be of the area B in the evaluation image is calculated (step S105). Then, it is determined whether or not the two-dimensional correlation coefficient r is equal to or less than the reference value C (step S106).

Thereby, it is determined whether a part of the body of the user 5 is located in this region. As can be seen from Equation 1 described above, the closer the gradation value of the outer region Ae and the gradation value of the outer region Be, the higher the two-dimensional correlation coefficient r.
The reference value C is determined in advance through experiments or the like. Specifically, the reference value C is preferably in the range of 0.5 to 1.0. More preferably, the range of 0.8 to 0.99 is preferable. If the reference value C exceeds 1.0 or is less than 0.5, it is difficult to appropriately determine whether or not the hand of the user 5 is located in the area where the learning image 6 is displayed. Become.

  When it is determined that the two-dimensional correlation coefficient r is equal to or less than the reference value C (step S106: YES), that is, when it is determined that the hand of the user 5 is located in the region B in the evaluation image, the learning image 6 Is provided, and the display time Td is cleared (step S108). In the present embodiment, an English word corresponding to a dog (learning image 6 in the present embodiment) and a dog cry are generated from the sound generation unit 3a, and an animation that causes the displayed dog to walk away is displayed. Alternatively, the image of the dog is changed to an English word “Dog” corresponding to the dog. Thus, when providing information based on the learning image 6, the learning image 6 is deleted.

  On the other hand, when it is determined that the two-dimensional correlation coefficient r exceeds the reference value C (step S106: NO), that is, when it is determined that the hand of the user 5 is not located in the region B in the evaluation image, Next, it is determined whether or not the display time Td of the learning image 6 exceeds the longest display time Tdmax (step S107).

  The longest display time Tdmax is the longest time for which the learning image 6 is continuously displayed on the display 4 when the display time Td is not cleared in the process of step S108, and is a predetermined time that is an evaluation image acquisition cycle. It is set in advance to a time longer than Te.

When it is determined that the display time Td is equal to or shorter than the longest display time Tdmax (step S107: NO), the processing after step S104 is executed.
On the other hand, when it is determined that the display time Td has exceeded the longest display time Tdmax (step S107: YES), the learning image 6 currently displayed on the display 4 is erased and the display time Td is cleared (step S109). ).

  Next, the PC 3 determines whether or not the learning time Tp has exceeded the predetermined time Tpc (step S110). The predetermined time Tpc is a predetermined time that is predetermined as an appropriate time for continuously using the learning support device 1 and is set in advance to a time longer than the longest display time Tdmax of the learning image 6. . The predetermined time Tpc is preferably in the range of 0.5 to 10 seconds. More preferably, the range of 2 to 5 seconds is preferable.

  When it is determined that the learning time Tp has exceeded the predetermined time Tpc (step S110: YES), this process is terminated. On the other hand, when it is determined that the learning time Tp is equal to or shorter than the predetermined time Tpc (step S110: NO), the processing after step S101 is executed.

By the way, as described in Non-Patent Document 1, in recent years, learning efficiency improves as the heart rate during physical exercise increases due to light physical exercise, while learning efficiency increases when the heart rate increases excessively due to excessive physical exercise. It is becoming clear that this will decline. Therefore, in the present embodiment, the longest display time Tdmax and the coordinates (X, Y) are set so that the exercise intensity falls within a predetermined range.
Specifically, for example, when expressed in terms of oxygen intake, it is preferable to set the longest display time Tdmax and coordinates (X, Y) so that the oxygen intake of the user 5 is 2 to 4 METs. It is more preferable to set the longest display time Tdmax and the coordinates (X, Y) so that becomes 3METs.
More specifically, the longest display time Tdmax is preferably set in the range of 20 seconds to 300 seconds, and more preferably in the range of 30 seconds to 120 seconds.

According to the said embodiment, there can exist an effect described below.
(1) According to the above embodiment, when the user 5 moves his / her body and positions a predetermined part in the area where the learning image 6 (in this embodiment, a dog image) is displayed, information based on the learning image 6 is displayed. Provided to the user. In other words, the user 5 can obtain an English word corresponding to a dog, a voice related to the dog's cry, an animation in which the displayed dog image walks away, and the like. Therefore, when the user 5 moves the body according to the display area of the learning image 6 so that the predetermined part is located in the area where the learning image 6 is displayed, and the predetermined part is located in the area, the user 5 learns. Visual changes and auditory changes based on the image 6 can be obtained. For this reason, the user 5 can learn English through auditory and visual changes while moving the body, and can efficiently learn while having fun.

  (2) Since whether or not a predetermined part is located in the selected area is determined based on the gradation value of the outer area Ae of the area A and the gradation value of the outer area Be of the area B, the gradation of the area A is determined. The accuracy is higher than in the case where the determination is made based on the value and the gradation value of the region B. In other words, it can be determined whether or not the predetermined part is located in the region A more accurately than in the case where the determination is based on the gradation value of the region A and the gradation value of the region B.

  (3) Since the longest display time Tdmax and the coordinates (X, Y) are set so that the exercise intensity falls within a predetermined range, the user 5 can perform physical exercise and learn efficiently. it can.

(Second embodiment)
Hereinafter, a second embodiment embodying the present invention will be described based on FIG. 4 and FIG. 5 with a focus on differences from the first embodiment. 4 and 5, the same elements as those described in the first embodiment are denoted by the same reference numerals, and redundant description of these elements is omitted.

  As shown in FIG. 4, the learning support device 1 of the second embodiment includes a central camera 7 (an “imaging” of the present invention) disposed at the upper center of a display 4 (corresponding to a “display screen” of the present invention). In addition to the “means”, an end camera 8 (corresponding to the “imaging means” of the present invention) provided on one upper side of the display 4 is provided.

  The central camera 7 and the end camera 8 image an imaging space including the user 5 facing the display 4. The imaging space of the central camera 7 and the end camera 8 may be in a range in which the whole body of the user 5 standing up against the display 4 can be accommodated. Further, the central camera 7 and the end camera 8 capture images of an imaging space including the user 5 facing the display 4 every elapse of a predetermined time Te, respectively, and the PC 3 (the “display unit” of the present invention, Equivalent to “determination means”).

  The PC 3 (corresponding to “display means” and “determination means” of the present invention) determines the distance D 2 between the user 5 and the display 4 based on the image information output from the central camera 7 and the end camera 8. It is calculated and it is determined whether or not the distance D2 is a predetermined distance. Then, when it is determined that the distance D2 is a predetermined distance, the image processing described in the first embodiment is started, and when it is determined that the distance D2 is not a predetermined distance, the image processing described in the first embodiment. Not to run.

Hereinafter, a distance calculation process for calculating the distance D2 between the user 5 and the display 4 will be described with reference to FIG.
As shown in FIG. 5, when the distance calculation process is started, first, an assumed distance D1 is set using the PC 3 (step S200). The assumed distance D <b> 1 is a distance necessary for the whole body image of the user 5 standing up against the display 4 to be displayed in an appropriate size on the display 4. The distance D1 may be set in advance, or may be set each time by the user 5 or the administrator. When the distance D1 is set in advance, the distance D1 is preferably in the range of 1 to 6 m. More preferably, the range of 3-4 m is preferable.

  Next, the PC 3 executes the processes of steps S206 to S210 described below in parallel with the processes of steps S201 to S205 described below.

  First, the processing of steps S201 to S205 will be described. As described above, after setting the assumed distance D1 (step S200), the PC 3 obtains the latest image among the images (corresponding to the “display image” of the present invention) captured by the central camera 7 and output to the PC 3. (Step S201). Then, a binarization process for converting the acquired image into two gradations of white and black is performed (step S202).

  Then, a difference image between the image acquired in the previous control cycle and subjected to the binarization process and the image acquired in the current control cycle and subjected to the binarization process is generated (step S203). Here, the difference between the image acquired in the previous control cycle and the image acquired in the current control cycle is recognized as a white pixel.

The difference between the image acquired in the previous control cycle and the image acquired in the current control cycle is due to the movement of something facing the display 4, that is, the movement of the user 5 standing up against the display 4. It is thought to be a thing. Therefore, the PC 3 determines whether or not a white pixel has been detected in the difference image (step S204).
If it is determined that a white pixel has been detected (step S204: YES), the user 5 is located at the position where the white pixel is detected, and the coordinates are stored as (X1, Y1) (step S205). .

  On the other hand, when it is determined that no white pixel is detected (step S204: NO), that is, when it is determined that there is no change between the previously captured image and the current captured image, the processing of steps S201 to S204 is performed. Run again.

  Next, the process of steps S206 to S210 will be described. As described above, after setting the assumed distance D1 (step S200), the PC 3 captures the latest image among the images (corresponding to the “display image” of the present invention) captured by the end camera 8 and output to the PC 3. Obtain (step S206). Then, a binarization process for converting the acquired image into two gradations of white and black is performed (step S207).

  Then, a difference image between the image acquired in the previous control cycle and subjected to the binarization process and the image acquired in the current control cycle and subjected to the binarization process is generated (step S208). Next, the PC 3 determines whether or not a white pixel is detected in the difference image (step S209).

If it is determined that a white pixel has been detected (step S209: YES), the user 5 is located at the position where the white pixel is detected, and the coordinates are stored as (X2, Y2) (step S210). .
On the other hand, when it is determined that no white pixel is detected (step S209: NO), that is, when it is determined that there is no change between the image captured last time and the image captured this time, the processing of step S206 to step S209 is performed. Run again.

  As described above, after executing the processing of steps S201 to S205 and the processing of steps S206 to S210, the coordinates (X1, Y1) and (X2, Y2) stored in the above-described processing, the central camera 7 and the end camera 8 are stored. The distance D2 between the user 5 and the display 4 is calculated on the basis of the distance between (step S211).

Next, it is determined whether or not the distance D2 is larger than the assumed distance D1−α and equal to or smaller than the assumed distance D1 + α (step S212). In addition, (alpha) has the preferable range of 0.5-1 m. More preferably, the range of 0.0-0.1 m is preferable.
When it is determined that the distance D2 is greater than the assumed distance D1-α and not more than the assumed distance D1 + α (step S212: YES), this process is temporarily terminated. In this case, the image processing described in the first embodiment is executed.

  On the other hand, when it is determined that the distance D2 is equal to or less than the assumed distance D1-α (step S212: NO), the processing after step S201 and the processing after step S206 are executed.

  According to the said embodiment, in addition to said (1)-(3), there can exist the effect described below.

  (4) According to the embodiment, even if there is something other than the user 5 in the imaging space including the user 5 facing the display 4, the distance between itself and the display 4 is If it is less than or equal to the assumed distance D1−α, or greater than the assumed distance D1 + α, the image processing is not started. For this reason, even if a moving object other than the user 5 is imaged, it can be prevented that this is mistaken as the user 5.

  (5) In addition, since the distance D2 between the user 5 and the display 4 is calculated using the difference image between the image acquired in the previous control cycle and the evaluation image in the current control cycle, it is simple. The position of the user 5 can be calculated with a simple configuration.

(Third embodiment)
Hereinafter, a third embodiment of the present invention will be described based on FIGS. 6 to 8 with a focus on differences from the first embodiment. In addition, in these FIGS. 6-8, the same code | symbol is shown about the element same as each element demonstrated in 1st embodiment, The overlapping description about these elements is omitted.

  As shown in FIG. 6, the learning support apparatus 1 of the third embodiment includes a central camera 7 (an “imaging” of the present invention) disposed at the upper center of a display 4 (corresponding to a “display screen” of the present invention). In addition to the “means”, an end camera 8 (corresponding to the “imaging means” of the present invention) provided on one upper side of the display 4 is provided.

  The central camera 7 and the end camera 8 image an imaging space including the user 5 facing the display 4. The imaging space of the central camera 7 and the end camera 8 may be in a range in which the whole body of the user 5 standing up against the display 4 can be accommodated. Further, the central camera 7 and the end camera 8 capture images of an imaging space including the user 5 facing the display 4 every elapse of a predetermined time Te, respectively, and the PC 3 (the “display unit” of the present invention, Equivalent to “determination means”).

  The PC 3 uses a point at a predetermined position of the display image (for example, the point at the lower left corner on the display 4) as the origin, the horizontal coordinate is the X axis, the vertical coordinate is the Y axis, and these X axis and Y axis are respectively The coordinates (X, Y, Z) for displaying the learning image 6 are set with the coordinates in the orthogonal direction as the Z axis. In the present embodiment, an image output from the central camera 7 and the end camera 8 to the PC 3 is a display image. Of these display images, an image output from the central camera 7 to the PC 3 is used in the present process as a reference image or an evaluation image. The coordinates for displaying the learning image 6 are set for the image output from the central camera 7.

  In the present embodiment, the Z axis is an axis having a point on the display 4 as an origin and a depth direction as a positive direction. The area of the learning image 6 displayed on the display 4 is made smaller as the value of the Z-axis coordinate is set to a larger value.

  Then, the PC 3 calculates the distance D2 between the user 5 and the display 4 based on the images respectively acquired from the central camera 7 and the end camera 8, and this distance D2 is set as described above. It is determined whether the distance is equivalent to the value of the Z-axis coordinate.

Hereinafter, the processing procedure of the image output to the PC 3 will be described in detail with reference to FIG.
As shown in FIG. 7, when this processing is started, first, the PC 3 starts measuring the learning time Tp (step S300), and then acquires a reference image captured by the central camera 7 (step S300). Step S301). Then, coordinates (X, Y, Z) for displaying the learning image 6 are set (step S302).

  Next, the PC 3 displays the learning image 6 in a region centered on the set coordinates (X, Y) (step S303). At this time, the PC 3 displays the learning image 6 smaller as the value of the Z-axis coordinate is set to a larger value, that is, as the learning image 6 is farther from the user 5. In addition, the measurement of the display time Td of the learning image 6 is started (step S303).

Then, the PC 3 acquires an evaluation image (corresponding to the “display image” of the present invention) and displays it on the display 4 (step S304). In this way, the latest image is used as the evaluation image every time step S304 is executed. For the evaluation image, the same coordinate system is set each time, and an area equivalent to the area B set in step S302 is set as the area B. And
Next, a three-dimensional coordinate calculation process is executed (step S305).

Next, the three-dimensional coordinate calculation process will be described with reference to FIG.
As shown in FIG. 8, when this process is started, the PC 3 first executes the processes of steps S335 to S339 described below in parallel with the processes of steps S330 to S334 described below.

First, the process of steps S330 to S334 will be described.
Of the images captured by the central camera 7 and output to the PC 3, the PC 3 divides the latest image (evaluation image) into a plurality of regions by a method according to the method described in the first embodiment, and the region B ( An image of an area in which the learning image 6 is displayed is acquired (step S330). Then, a binarization process for converting the image into two gradations of white and black is performed (step S331).

Next, a difference image is generated between the image of the region B acquired in the previous control cycle and subjected to the binarization processing and the image of the region B acquired in the current control cycle and subjected to the binarization processing. (Step S332). And it is judged whether the white pixel was detected in this difference image (step S333).
If it is determined that a white pixel has been detected (step S333: YES), the coordinates of the detected white pixel are stored as (X1, Y1) (step S334).

  On the other hand, when it is determined that no white pixel is detected (step S333: NO), that is, when it is determined that there is no change between the previously captured image and the current captured image, the processing of steps S330 to S332 is performed. Run again.

Next, processing in steps S335 to S339 will be described.
The PC 3 sets the same coordinate system as the evaluation image for the latest image among the display images captured by the end camera 8 and output to the PC 3, and divides the image into a plurality of regions. Then, an image of an area corresponding to the area B centered on the coordinates corresponding to the coordinates (X, Y) set in the evaluation image is acquired (step S335). Then, a binarization process is performed on the image in this area to convert it into two gradations of white and black (step S336).

  Next, an image of the area obtained from the end camera 8 and binarized in the previous control cycle, and an area obtained from the end camera 8 and binarized in the current control cycle A difference image from the image is generated (step S337). Next, the PC 3 determines whether or not a white pixel has been detected in the difference image (step S338).

If it is determined that a white pixel has been detected (step S338: YES), the coordinates of the detected white pixel are stored as (X2, Y2) (step S339).
On the other hand, when it is determined that no white pixel is detected (step S338: NO), that is, when it is determined that there is no change between the previously captured image and the current captured image, the processing of steps S335 to S337 is performed. Run again.

  As described above, after executing the processing of steps S330 to S334 and the processing of steps S335 to S339, based on the coordinates (X1, Y1) and (X2, Y2) stored in the above processing, the user 5 and the display are displayed. 4 is calculated (step S340).

  After executing such a three-dimensional coordinate calculation process (step S305), the PC 3 determines whether or not the calculated distance D2 is within a predetermined range (step S306). Next, a method for determining whether or not the distance D2 is within the predetermined range will be described with reference to FIG.

As shown in FIG. 9A, for example, when the Z-axis coordinate value “Z” is set to a small value, the learning image 6 appears to be displayed near the user 5, The learning image 6 is displayed in a large size. At this time, it is determined that the user 5 is within the predetermined range when the user 5 is away from the display 4, that is, the distance D2 between the user 5 and the display 4 is increased.
On the other hand, when the Z-axis coordinate value “Z” is set to a large value, as shown in FIG. 9B, the learning image 6 appears to be displayed far from the user 5, The learning image 6 is displayed in a small size. At this time, it is determined that the user 5 is within the predetermined range when the user 5 approaches the display 4, that is, when the distance D2 between the user 5 and the display 4 is reduced.

Thus, when the Z-axis coordinate value “Z” is set to a large value, the distance D2 decreases, and when the Z-axis coordinate value “Z” is set to a small value, the distance D2 increases. Therefore, it is determined that the user 5 is within the predetermined range.
For this reason, in the present embodiment, when the sum of the Z-axis coordinate value “Z” and the distance D2 becomes the predetermined value D3, it is determined that the distance D2 is within the predetermined range.

  When it is determined that the distance D2 is within the predetermined range (step S306: YES), the gradation value of the area B where the learning image 6 is next displayed in the evaluation image and the gradation value of the area A in the reference image are 2 A dimension correlation coefficient r is calculated (step S307). And it is judged whether this two-dimensional correlation coefficient r is below the reference value C (step S308).

  When it is determined that the two-dimensional correlation coefficient r is equal to or less than the reference value C (step S308; YES), information based on the learning image 6 is provided and the display time Td is cleared (step S310). Specifically, as shown in the first embodiment, an English word corresponding to a dog and a bark of a dog are generated from the voice generation unit 3a, and an animation that causes the displayed dog to walk away is displayed. Alternatively, the image of the dog is changed to an English word “Dog” indicating the dog.

  When it is determined that the two-dimensional correlation coefficient r exceeds the reference value C (step S308: NO), and when it is determined that the distance D2 is not within the predetermined range (step S306: NO), the learning image 6 is displayed. It is determined whether or not the time Td exceeds the longest display time Tdmax (step S309).

  When it is determined that the display time Td is equal to or shorter than the longest display time Tdmax (step S309: NO), the processing after step S304 is executed.

  On the other hand, when it is determined that the display time Td has exceeded the longest display time Tdmax (step S309: YES), the learning image 6 currently displayed on the display 4 is deleted, and the display time Td is cleared (step S311). .

  Next, the PC 3 determines whether or not the learning time Tp has exceeded the predetermined time Tpc (step S312). When it is determined that the learning time Tp has exceeded the predetermined time Tpc (step S312: YES), this process ends. On the other hand, when it is determined that the learning time Tp is equal to or shorter than the predetermined time Tpc (step S312: NO), the processing after step S301 is executed.

  As described in the first embodiment, in recent years, learning efficiency improves as the heart rate during physical exercise increases due to light physical exercise, while learning efficiency decreases when the heart rate increases excessively due to excessive physical exercise. It is becoming clear. Therefore, in the present embodiment, it is preferable to set the longest display time Tdmax and the coordinates (X, Y, Z) so that the exercise intensity falls within a predetermined range.

According to the said embodiment, in addition to said (1)-(3), there can exist the effect described below.
(6) According to the present embodiment, the user 5 moves not only in the X-axis direction and the Y-axis direction but also in the Z-axis direction (front-rear direction). For this reason, the range in which the user 5 moves is increased, and the exercise intensity can be improved more favorably, and as a result, the learning efficiency can be improved.

(Fourth embodiment)
Hereinafter, a fourth embodiment embodying the present invention will be described based on FIG. 10 and FIG. 11 with a focus on differences from the first embodiment. 10 and 11, the same reference numerals are given to the same elements as those described in the first embodiment, and redundant description of these elements is omitted.

  As shown in FIG. 10, the learning support device 1 of the fourth embodiment includes a first camera 10 (“the display screen” of the present invention) disposed at the upper center of the display 4 (corresponding to the “display screen” of the present invention). In addition to the "imaging means" and the second camera 11 (corresponding to the "imaging means" of the present invention), an infrared light 13 for irradiating infrared light is provided.

The first camera 10 and the second camera 11 are arranged so that the imaging range is a range in which the whole body of the user 5 standing up against the display 4 can be accommodated. A filter 12 that transmits only infrared light is attached to the light receiving portion of the first camera 10. For this reason, the 1st camera 10 will image the infrared light image which consists only of infrared light.
The first camera 10 and the second camera 11 take an image of an imaging space including the user 5 facing the display 4 every elapse of a predetermined time Te, respectively, and the PC 3 (the “display unit”, “ Equivalent to “determination means”).

In the present embodiment, an image output from the first camera 10 and the second camera 11 to the PC 3 is a display image. In particular, an image output from the first camera 10 to the PC 3 is displayed on the display 4, and an image output from the second camera 11 to the PC 3 is used as a reference image or an evaluation image in this processing.
Hereinafter, the processing procedure of the image output to the PC 3 will be described in detail with reference to FIG.

  As shown in FIG. 11, when this process is started, first, the PC 3 starts measuring the learning time Tp (step S400), and then acquires a visible light image captured by the second camera 11. And it displays on the display 4 (step S401).

  And the image imaged at the time nearest to the time of the start of this process among the images imaged with the 1st camera 10 and output to PC3 is acquired as a standard image (Step S402). Further, the latest image is acquired as an evaluation image (step S402).

  Next, coordinates for displaying the learning image 6 by the method according to the above-described first embodiment with respect to the latest image (evaluation image) among the images captured by the first camera 10 and output to the PC 3. (X, Y) is set (step S403). Specifically, one area is selected from a plurality of areas formed by dividing the evaluation image, and the center coordinates of the selected area (hereinafter, area B) are set as coordinates (X, Y). .

  Similar to the first embodiment, the same coordinate system is set for the reference image and the evaluation image. The PC 3 sets a region corresponding to the region B in the reference image (a region centered on the set coordinates (X, Y) in the reference image) as the region A.

  Moreover, the same coordinate system as the image output from the first camera 10 to the PC 3 is set for the latest image among the visible light images output from the second camera 11 to the PC 3. Similar to the image output to the PC 3, it is divided into a plurality of areas. Then, the learning image 6 is displayed in a region corresponding to the region B with the coordinate corresponding to the coordinates (X, Y) as the center, and the measurement of the display time Td of the learning image 6 is started (step S404).

  Next, the PC 3 displays a visible light image captured by the second camera 11 on the display 4 (step S405). In this way, the latest visible light image is displayed on the display 4 every time step S405 is executed. For these visible light images, the same coordinate system is set each time and the region set in steps S403 and S404 is set. The learning image 6 is continuously displayed.

And the newest image is acquired as an evaluation image among the infrared-light images imaged with the 1st camera 10 and output to PC3 (step S406). In this way, the latest infrared light image is used as the evaluation image every time step S406 is executed. For these evaluation images, the same coordinate system is set each time, and the region B set in step S403 is set each time. Region B is set at the same position.

Next, the PC 3 calculates a two-dimensional correlation coefficient r between the gradation value of the region B in the evaluation image and the gradation value of the region A in the reference image (step S407). And it is judged whether this two-dimensional correlation coefficient r is below the reference value C (step S408).
When it is determined that the two-dimensional correlation coefficient r is equal to or less than the reference value C (step S408: YES), information based on the learning image 6 is provided and the display time Td is cleared (step S410).

  On the other hand, when it is determined that the two-dimensional correlation coefficient r exceeds the reference value C (step S408: NO), that is, the hand of the user 5 is located in the region B where the learning image 6 is displayed in the evaluation image. If it is determined that it is not, it is next determined whether or not the display time Td of the learning image 6 exceeds the longest display time Tdmax (step S409). When it is determined that the display time Td is equal to or shorter than the longest display time Tdmax (step S409: NO), the processing after step S405 is executed.

  When it is determined that the display time Td exceeds the longest display time Tdmax (step S409: YES), the learning image 6 currently displayed on the display 4 is then erased and the display time Td is cleared (step S411). .

  Next, the PC 3 determines whether or not the learning time Tp has exceeded the predetermined time Tpc (step S412). When it is determined that the learning time Tp has exceeded the predetermined time Tpc (step S412: YES), this process ends. On the other hand, when it is determined that the learning time Tp is equal to or shorter than the predetermined time Tpc (step S412: NO), the processing after step S402 is executed.

According to the said embodiment, in addition to said (1)-(3), there can exist an effect described below.
(7) According to the present embodiment, the position of the user 5 is detected using the infrared light that is irradiated from the infrared light 13, reflected by the user 5, and picked up by the first camera 10. be able to. For this reason, when there is something other than the user 5 in the space including the user 5 facing the display 4, it is possible to suppress misidentification of the thing other than the user 5 as the user 5, and make the user 5 appropriate. Can be detected.

(Other embodiments)
Note that the learning support device 1 according to the present invention is not limited to the configuration exemplified in the above embodiment, and can be implemented as, for example, the following form in which the embodiment is appropriately changed. The following modifications are not applied only to the above-described embodiment, and different modifications can be combined with each other.

  As shown in FIG. 12, a plurality of areas may be selected and different learning images 6a to 6c may be displayed in the respective areas. In this case, after the learning images 6a to 6c are displayed on the display 4, the PC 3 arbitrarily selects one learning image from the learning images 6a to 6c. For example, when the learning image 6b is selected, after the learning images 6a to 6c are displayed on the display 4, the PC 3 gives some instruction so that the user 5 selects the learning image 6b. In the example of FIG. 12, English voice (CAT) voice corresponding to a cat is generated from the voice generator 3a. Thereafter, only when the predetermined part of the user 5 is located in the area where the learning image 6b is displayed, the information based on the learning image 6b is provided, and what changes when the predetermined part is located in an area other than the learning image 6b. Also don't give. According to this modification, the user 5 is given learning information 6b (from the learning images 6a to 6c) in a state where information based on the learning image 6b (in this modification, voice of English words related to cats) is given. The operation of selecting an image relating to a cat) is performed. For this reason, the user 5 can perform the learning which matches the learning image 6b and the information based on the learning image 6b more suitably.

  In the third embodiment, the learning image 6 is displayed in the depth direction of the display 4, but the display method of the learning image 6 of the present invention is not limited to this. For example, you may display in the aspect which jumps out from the display 4 toward the user 5.

  During the image processing, the distance D2 between the user 5 and the display 4 is constantly monitored, and when it is determined that the distance D2 is not the assumed distance D1, the image processing is terminated. Good.

  In the above embodiment, it is determined whether the hand of the user 5 is located in the area where the learning image 6 is displayed using the two-dimensional correlation coefficient r of the gradation value. The method is not limited to this, and for example, the determination may be made using the two-dimensional luminance correlation coefficient r.

  In the above embodiment, the information based on the learning image 6 is the voice, animation, or image of the English word corresponding to the learning image 6 (the dog in the above embodiment). Is not limited to this. As information based on the learning image 6, vibration may be generated.

  In the above embodiment, one or two cameras are arranged, but the number of cameras arranged is not limited to this, and more cameras may be arranged. According to this modification, the position of the user 5 in the Z-axis direction can be detected with high accuracy.

  In the above embodiment, the camera is arranged on the upper part of the display 4, but the arrangement position of the camera in the present invention is not limited to this, and a whole body image of the user 5 can be taken. Any position is acceptable. For example, you may arrange | position in the position which opposes on both sides of the display 4 and the user 5. FIG.

  In the above embodiment, the image is displayed on the display 4, but the image display method of the present invention is not limited to this. For example, a projector may be connected to the PC 3 so that an image is emitted from the projector.

  In the above embodiment, it has been described that the user 5's hand corresponds to the predetermined part, but the present invention is not limited to this, and any part of the user 5 can be the predetermined part.

  In the above embodiment, an example used for English learning has been described, but what can be learned using the present invention is not limited to English. For example, it can be used for learning other foreign languages such as Chinese and French. Moreover, the learning is not limited to language, and can be used for learning in other fields as long as the learning is to promote the establishment of some kind of memory.

1 ... Learning support device 2 ... Camera 3 ... PC
4 ... Display 5 ... User 6 ... Learning image 7 ... Central camera 8 ... End camera 10 ... First camera 11 ... Second camera 12 ... Filter 13 ... Infrared light

Claims (8)

A learning support apparatus comprising: a display screen for displaying an image; and an imaging means for capturing an imaging space including a user facing the display screen as a display image,
Display means for displaying the display image on the display screen and displaying a learning image in at least one of a plurality of regions formed by partitioning the display image;
Determination means for determining whether or not a predetermined part of the user is located in a region where the learning image is displayed, based on the display image;
The learning support apparatus, wherein when the determination unit determines that the predetermined part is located in a region where the learning image is displayed, the display unit provides information based on the learning image. The learning support device according to claim 1,
The determination unit stores a display image output from the imaging unit before display of the learning image as a reference image, stores a display image output from the imaging unit after display of the learning image as an evaluation image, Learning based on a two-dimensional correlation coefficient between a gradation value of the region in the reference image and a gradation value of the region in the evaluation image, to determine whether or not the predetermined part is located in the region Support device. The learning support device according to claim 2,
Dividing the region into a central portion and an outer portion excluding the central portion;
Based on a two-dimensional correlation coefficient between the gradation value of the outer portion of the region in the reference image and the gradation value of the outer portion of the region in the display image, it is determined whether or not the predetermined portion is located in the region. A learning support apparatus characterized by The learning support device according to any one of claims 1 to 3,
The display means displays the learning image on the display screen so that the exercise intensity of the user when the user moves his body according to the learning image displayed on the display screen is within a predetermined range. A learning support device characterized by having The learning support device according to any one of claims 1 to 4,
A plurality of the imaging means are provided,
The determination unit calculates a distance between the predetermined part and the display screen based on each display image captured through each imaging unit,
A learning support apparatus, wherein when a predetermined distance is between the predetermined part and the display screen, it is determined whether or not the predetermined part of the user is located in a region where the learning image is displayed. A learning support apparatus comprising: a display unit that displays an image on a display screen; and a plurality of imaging units that capture an imaging space including a user facing the display screen as a display image,
The display unit displays a display image captured through one of the plurality of imaging units on the display screen;
Partitioning the display image displayed on the display screen into a plurality of regions, selecting at least one of the plurality of regions,
A predetermined range is set in coordinates extending in a direction perpendicular to the display screen;
In the selected area, the learning image is displayed in a size according to the set predetermined range,
Based on each display image captured through the plurality of imaging means, a distance between a predetermined part of the user and the display screen is calculated,
A determination process is performed to determine whether or not the user's predetermined part is located in an area where the learning image is displayed when the predetermined part and the display screen are within the predetermined range. Learning support device. The learning support device according to any one of claims 1 to 6,
The image pickup means picks up infrared light. A learning support method comprising: a display step of displaying an image; and an imaging step of capturing an imaging space including a user facing the display screen as a display image,
A display step of displaying the display image on the display screen and displaying a learning image in at least one of a plurality of regions formed by partitioning the display image;
A determination step of determining whether or not a predetermined part of the user is located in a region where the learning image is displayed, based on the display image,
In the display step, the learning image is displayed in a different area for every elapse of a predetermined time,
When it is determined through the determination step that the predetermined part is located in a region where the learning image is displayed, the display step provides information based on the learning image.

Images