TWI559269B - System, method, and computer program product for simulated reality learning - Google Patents

Description

Quasi-reality teaching system and method thereof, and computer program product

The invention relates to a pseudo-reality teaching system and a method thereof, and a computer program product, in particular to a virtual reality teaching system and a method thereof and a computer program product using the virtual reality technology.

In the traditional scientific experimental computer teaching environment, the presentation of scientific experiments is mostly presented by text, pictures or Flash animation. However, at present, a small number of virtual science experimental teaching environments use Virtual Reality or Augmented Reality technology, and the image or experimental objects on the display screen are still set by pre-setting ideal parameters. Drawn by 3D computer graphics software, there is still a gap in the sense between the virtual objects of the computer and the real objects, so that the learners can not feel the realism.

In summary, how to provide a pseudo-reality teaching system and its method and computer program product that can realistically present the image of the experimental object is the goal that is currently in great need.

The present invention provides a pseudo-reality teaching system and method thereof, and a computer program product, which utilizes a pseudo-realistic technology, that is, through a random access digital media technology with time-lapse photography technology, First record all possible experimental paths and results in the server module, for the learners to actively operate different variables, and present the experimental results and history corresponding to different experimental variables, which can achieve the real-life somatosensory operation experiment and The visual effect of real visual/auditory feelings.

The pseudo-reality teaching system of one embodiment of the present invention is used for an experimental operation program. The virtual reality teaching system comprises a display module, a detection module and a server module. The display module is configured to display an interface to a learner, wherein the interface includes at least one display area and an experimental equipment setting area, and the experimental equipment setting area includes a plurality of experimental equipment physical images for the learner to select. The detecting module is configured to capture and recognize one of the learners' actions or accept an operation instruction generated by the learner using a control unit, and output a corresponding one of the control parameters to update the content of the display area. The server module is electrically connected to the display module and the detection module, and the server module comprises a memory unit and a processing unit. A memory unit for storing a plurality of real-life movies related to a plurality of possible experimental paths of the experimental operating procedure. The processing unit is electrically connected to the memory unit, and calculates an experimental result and an experimental result corresponding to the experimental data according to the control parameter, and selects a corresponding real-life film according to the experimental result, so that the display module updates the display area. Content, and store experimental results and experimental data in a memory unit.

A pseudo-reality teaching method according to another embodiment of the present invention is used in an experimental operation program. The virtual reality teaching method comprises: a display module displaying an interface to a learner, wherein the interface provides at least one display area and an experimental equipment setting area, and the experimental equipment setting area comprises a plurality of experimental equipment entity images for the learner to select; Detecting interaction between the learner and the physical image of the experimental device, and outputting one of the control parameters to a server module; and the server module storing a plurality of realities related to the plurality of possible experimental paths of the experimental operating program The film, receiving the control parameter and calculating an experimental result and an experimental result of the corresponding experimental data, and selecting a corresponding real-life movie according to the experimental result, so that the display module updates the content of the display area.

Another embodiment of the present invention stores a computer program product for use in a virtual reality teaching program, when the computer loads the computer program and executes a virtual reality teaching method for an experimental operation program. Pseudo The real-world teaching method comprises: displaying a interface to a learner by using a display module, wherein the interface provides at least one display area and an experimental equipment setting area, and the experimental equipment setting area comprises a plurality of experimental equipment entity images for the learner to select; Detecting interaction between the learner and the physical image of the experimental device by a detection module, and outputting one of the control parameters to a server module; and storing the plurality of possibilities related to the experimental operation program by using the server module A plurality of real-world movies of the experimental path, receiving control parameters, and calculating an experimental result of the experimental data and the corresponding experimental data, and selecting a corresponding real-life movie according to the experimental result, so that the display module updates the content of the display area.

The purpose, technical contents, features, and effects achieved by the present invention will become more apparent from the detailed description of the appended claims.

A1‧‧‧ action

A2‧‧‧Operating instructions

B‧‧‧Control parameters

C‧‧‧Experimental results

D‧‧‧Experimental data

E‧‧‧real film

1‧‧‧ display module

10‧‧‧ interface

11‧‧‧ display area

12‧‧‧Device setting area

13‧‧‧Scenario setting area

14‧‧‧ parameter setting area

15‧‧‧Chart setting area

2‧‧‧Detection module

21‧‧‧Action sensing components

22‧‧‧Arithmetic components

3‧‧‧Server Module

31‧‧‧ memory unit

32‧‧‧Processing unit

S1~S3‧‧‧ steps

1 is a schematic view showing a pseudo-reality teaching system according to an embodiment of the present invention.

2 is a schematic view showing an operation module according to an embodiment of the present invention.

FIG. 3 is a flow chart showing the steps of the pseudo-reality teaching method according to an embodiment of the present invention.

The embodiments of the present invention will be described in detail below with reference to the drawings. In addition to the detailed description, the present invention may be widely practiced in other embodiments, and any alternatives, modifications, and equivalent variations of the described embodiments are included in the scope of the present invention. quasi. In the description of the specification, numerous specific details are set forth in the description of the invention. In addition, well-known steps or elements are not described in detail to avoid unnecessarily limiting the invention. figure The same or similar elements will be denoted by the same or similar symbols. It is to be noted that the drawings are for illustrative purposes only and do not represent actual dimensions, quantities, or actual relative positions of the components. Some of the details may not be fully drawn in order to facilitate the simplicity of the drawings.

Referring to FIG. 1 , a virtual reality teaching system according to an embodiment of the present invention includes a display module 1 , a detection module 2 , and a server module 3 . The display module 1 is configured to display an interface 10 to a learner. The interface 10 includes at least a display area 11 and a fixture setting area 12. For example, the display module can be a computer screen, a projection display device (for example, a projector with a projection screen) or a holographic image display device, but is not limited thereto. The equipment setting area 12 includes a plurality of equipment entity images (not shown) for the learner to select as required for subsequent experimental operations. For example, in a buoyancy experiment, the fixture setting area 12 will appear different artifacts related to the experiment, such as weights, spring scales, electronic scales, timers, liquids of different densities, different shapes or densities to be treated. In particular, the physical images of these devices are real-time images taken by a camera. In a preferred embodiment, the physical images of the devices are taken at full angles of the camera, so that the learner can select and rotate a solid image of the object to be measured to observe the topography of each angle.

In an embodiment, the interface 10 further includes a context setting area 13 that includes a plurality of experimental scenarios. The experimental scenarios may be physical experiments, chemical experiments or biological experiments. For example, the experimental scenarios may be buoyancy experiments, Hooke's law experiments, fermentation reaction experiments, osmotic experiments or photosynthesis experiments. Then, the display module 1 updates the image of the plurality of device entities corresponding to the experimental situation displayed in the display area according to the specific experimental situation selected by the learner, for example, after selecting the buoyancy test, the display weight is updated in the display area. Spring scales, electronic scales, timers, liquids of different densities, and multiple physical images of objects of different shapes or densities for the learner to select and drag the desired physical image of the equipment to a working area in the display area 11 (not shown) for subsequent experimental combinations and operations.

In another embodiment, the interface 10 further includes a parameter setting area 14 . The detecting module 2 controls the parameter setting area 14 by the learner to output the corresponding control parameter B to the server module 3 . For example In a buoyancy experiment, after the learner selects a weight in the equipment setting area 12, the weight of the weight can be further set to 10 g or other weight in the parameter setting area 14, or the learner can set a test object. Different experimental parameters such as weight, density, volume, etc., for the learner to personally design and control the cause and manipulation of the variable, so as to learn the experimental data C and the experimental result D produced by the combination of different experimental parameters.

In order to enable the learner to organize a plurality of experimental data C in a graphical representation or a table, etc., to observe whether the experimental data C has a specific trend or whether the relationship between the experimental parameters conforms to a specific formula, in yet another embodiment, The interface 10 further includes a chart setting area 15. The chart setting area 15 includes at least one set of stored experimental data C, at least one set of stored experimental results D, an experimental formula, a table or a drawing template for the learner to select the experimental data C or experiment to be drawn. Result D. Therefore, the virtual reality teaching system will automatically record multiple experimental data C and experimental results D and present them in related diagrams or tables.

Following the above description, please continue to refer to FIG. 1. The detection module 2 is used to capture and recognize one of the learners' actions A1 or accept the learner to use one of the manipulation units (not shown) to generate an operation command A2, and The output corresponds to one of the control parameters B to update the content of the display area 11. In one embodiment, the learner's action A1 includes at least one of the learner's eye movements and the learner's gestures. For example, the learner's eye movements include actions such as moving, rotating, or pausing, while the learner's gestures include actions such as moving, rotating, grabbing, or clicking. In one embodiment, the manipulation unit includes at least one of a mouse, a rocker, or a touch screen.

In an embodiment, referring to FIG. 2, the detection module 2 can be an identification unit having a two-dimensional or three-dimensional image sensing function for detecting the above action, for example, it can be a PrimeSenseTM PrimeSensor (for example, : Microsoft Kinect), ZCAMTM from 3DV SYSTEM, EyeToyTM from Sony Corporation, or 3D image Sensor from CANESTATM. The identification unit 2 includes a motion sensing component 21 and an arithmetic component 22. The motion sensing component 21 is configured to capture at least one of the learners' actions A1. An operation component 22 is electrically connected to the motion sensing component 21, and according to at least the motion image, calculates a coordinate change of the motion and recognizes the motion to output a corresponding control parameter B.

Continuing to refer to FIG. 1 , the server module 3 is electrically connected to the display module 1 and the detection module 2 , and the server module 3 includes a memory unit 31 and a processing unit 32 . The memory unit 30 is for storing a plurality of real-life movies. The processing unit 32 is electrically connected to the memory unit 31, and calculates an experimental result D corresponding to the experimental data C and the corresponding experimental data C according to the control parameter B, and selects the corresponding real-life movie E according to the experimental result D, The display module 1 updates the content of the display area 11, and stores the experimental result D and the experimental data C in the memory unit 31. In an embodiment, the server module 3 can be a cloud server module.

It should be noted that the present invention utilizes a pseudo-realistic technology, that is, through random access-based digital media technology with Time-lapse Photography technology, pre-recording all possible experimental paths and results. In the server module, the learner actively operates (Hands-on) different causes. However, the above experimental results and the actual situation film are taken in advance by a camera or a camera and recorded different experimental paths and results, corresponding to different experimental parameters, operational variables and experimental data calculated by the server module operated by the learner. . Therefore, for different operation results, the display module will present real random access digital media results of different experiments in the display area, that is, the above experimental results.

In an embodiment, each of the real-life films is an experimental reality in which the experimental results are presented by time-lapse photography, so that the learner can observe the experimental process with a long reaction time in a short time. The invention utilizes random access digital media technology with time-lapse photography technology to help highlight physical or chemical changes in the experimental process for learners to compare and observe, and improve their learning efficiency, which is a traditional scientific experimental teaching. The learning effect that the system cannot achieve.

The following describes a method for teaching a virtual reality according to an embodiment of the present invention. Referring to FIG. 3, the steps include: first, displaying a interface to a learner by using a display module, wherein the interface provides at least one display area and a device setting area. And the equipment setting area contains multiple equipment entity images for the learner to select (S1). In a preferred embodiment, the step further comprises selecting one of a plurality of experimental scenarios from a context setting area of the interface, wherein the experimental scenarios may be physical experiments, chemical experiments, or biological experiments, for example, the experiments The situation may be a buoyancy experiment, a Hooke's law experiment, a fermentation reaction experiment, an osmotic experiment or a photosynthesis experiment, and the display module updates the device entities corresponding to the experimental situation displayed in the display area according to the experimental context selected by the learner. The image has been described above and will not be described here.

Then, a detection module detects the interaction between the learner and the device entity image, and outputs a corresponding control parameter to a server module (S2). In one embodiment, the step of detecting interaction between the learner and the device entity image is to capture and identify an operation instruction generated by the learner using a manipulation unit, wherein the manipulation unit includes a mouse, a joystick, or At least one of a touch screen. In another embodiment, the step of detecting interaction between the learner and the device entity image is to capture and identify one of the learner actions. In an embodiment, the learner's action A1 includes at least one of the learner's eye movement and the learner's gesture, as described above, and is not described herein again. As described above, the step of capturing and recognizing the action used by the learner includes: capturing, by the action sensing component, at least one motion image of the user; and calculating the motion by the computing component according to at least the motion image The coordinates change and identify the action to output the corresponding control parameters.

In a preferred embodiment, before the step of detecting interaction between the learner and the device entity image, the experiment parameter is set by a parameter setting area of the interface to output the corresponding control parameter to the server module. As mentioned before, it will not be repeated here.

Furthermore, the server module receives the control parameters and calculates an experimental result of generating an experimental data and corresponding experimental data, and selects one of the actual reality videos according to the experimental result, so that the display module updates the content of the display area (S3) ). In one embodiment, each of the reality films is presented by time-lapse photography. The experimental reality of the experimental results can enable the learner to observe the experimental process with a long reaction time in a short time.

In an embodiment, in a step of displaying a interface to a learner in a display module, the interface further provides a chart setting area including at least one set of stored experimental data, at least one set of stored experimental results, an experimental formula, A table or a drawing template for the learner to select the experimental data or experimental results that he wants to draw, as described above, and will not be repeated here.

It can be understood that in the era of developed Internet, the virtual reality teaching method of the present invention is not limited to being stored in a computer multimedia (for example, a disc), and may also be a computer program product for users to use the Internet. The network is downloaded and stored in a vehicle to perform the pseudo-reality teaching method shown in FIG. 3. The vehicle can be a tablet computer, a desktop computer or a server, but is not limited thereto. The steps and related embodiments of the pseudo-realistic teaching method shown in FIG. 3 have been described above, and are not described herein again.

In summary, the present invention relates to a virtual reality teaching system and method thereof, and a computer program product, which utilizes a pseudo-realistic technology, that is, through random access digital media technology with time-lapse photography technology, pre-recording all possible experimental paths. And the result is in the server module, which is used for the learner to actively manipulate different variables and present the experimental results and history corresponding to different experimental variables, so as to achieve the real somatosensory operation experiment and the real visual/auditory feeling teaching. effect. In addition, the experimental reality of the experimental results by time-lapse photography technology enables the learner to observe the experimental process with a long reaction time in a short time. The invention utilizes random access digital media technology with time-lapse photography technology to help highlight physical or chemical changes in the experimental process for learners to compare and observe, and improve their learning efficiency, which is a traditional scientific experimental teaching. The learning effect that the system cannot achieve.

The embodiments described above are merely illustrative of the technical spirit and characteristics of the present invention, and the purpose of the present invention is to enable those skilled in the art to understand the contents of the present invention and to implement the present invention. Equivalent changes or modifications made by the spirit of the present invention should still be included in the scope of the present invention.

A1‧‧‧ action

A2‧‧‧Operating instructions

B‧‧‧Control parameters

C‧‧‧Experimental results

D‧‧‧Experimental data

E‧‧‧real film

1‧‧‧ display module

10‧‧‧ interface

11‧‧‧ display area

12‧‧‧Device setting area

13‧‧‧Scenario setting area

14‧‧‧ parameter setting area

15‧‧‧Chart setting area

2‧‧‧Detection module

3‧‧‧Server Module

31‧‧‧ memory unit

32‧‧‧Processing unit

Claims (24)

A pseudo-reality teaching system for an experimental operation program, the pseudo-reality teaching system comprising: a display module for displaying an interface to a learner, wherein the interface includes at least one display area and an experimental equipment setting a region, and the experimental device setting area includes a plurality of experimental device physical images for the learner to select; a detecting module for capturing and recognizing one of the learners' actions or accepting the learner to use a control unit One of the operation instructions, and outputting one of the control parameters to update the content of the display area; and a server module electrically connected to the display module and the detection module, respectively, comprising: a memory a unit for storing a plurality of real-life movies related to a plurality of possible experimental paths of the experimental operation program; and a processing unit electrically connected to the memory unit, and calculating an experimental data according to the control parameter and Corresponding to the experimental result of the experimental data, and selecting the corresponding real-life film according to the experimental result, so that the display module updates the display area , And the experimental results and the experimental data is stored in the memory unit. The virtual reality teaching system of claim 1, wherein the server module comprises a cloud server module. The virtual reality teaching system of claim 1, wherein the interface further comprises a context setting area, where the plurality of experimental scenarios are included, and the display module is selected according to the learner The experimental situation update is displayed on the display device entity image corresponding to the experimental situation in the display area. The pseudo-reality teaching system as claimed in claim 3, wherein the experimental scenario comprises a physical experiment, a chemical experiment or a biological experiment. The virtual reality teaching system of claim 1, wherein the interface further comprises a parameter setting area, wherein the detecting module controls the parameter setting area by the learner to output the corresponding control parameter to the server mode group. The virtual reality teaching system of claim 1, wherein the interface further comprises a chart setting area, comprising at least one set of stored experimental data, at least one set of stored experimental results, an experimental formula, a form or A drawing template for the learner to select the experimental data or the experimental result that he wants to draw. The virtual reality teaching system of claim 1, wherein the action of the learner comprises at least one of an eye movement of the learner and a gesture of the learner. The virtual reality teaching system of claim 7, wherein the detection module comprises an identification unit, comprising: a motion sensing component for capturing at least one motion image of the learner; and an arithmetic component And electrically connected to the motion sensing component, and according to the motion image, calculate a coordinate change of the motion and recognize the motion to output the corresponding control parameter. The virtual reality teaching system of claim 8, wherein the action comprises at least one of a click, a move, a grab, and a rotation. The virtual reality teaching system of claim 1, wherein the control unit comprises at least one of a mouse, a joystick or a touch screen. The virtual reality teaching system of claim 1, wherein each of the real-life films presents an experimental reality of the experimental result by time-lapse photography. A pseudo-reality teaching method for an experimental operation program, the pseudo-reality teaching system comprising: displaying a interface to a learner by a display module, wherein the interface provides at least one display area and an experimental equipment setting area, And the experimental device setting area includes a plurality of experimental device physical images for the learner to select; detecting, by the detecting module, the interaction between the learner and the physical image of the experimental device, and outputting one of the control parameters to a servo And a plurality of real-life movies storing a plurality of possible experimental paths related to the experimental operating program, receiving the control parameters, and calculating an experimental data and an experimental result corresponding to the experimental data And selecting a corresponding real-life movie according to the result of the experiment, so that the display module updates the content of the display area. The method according to claim 12, wherein the step of detecting the interaction between the learner and the physical image of the experimental device is to capture and recognize one of the actions of the learner. The method as claimed in claim 13, wherein the action of the learner comprises at least one of an eye movement of the learner and a gesture of the learner. The method for teaching a virtual reality according to claim 13, wherein the step of capturing and recognizing the action used by the learner comprises: capturing, by the action sensing component, at least one motion image of the learner; and The computing component calculates a coordinate change of the action according to the motion image and recognizes the motion to output the corresponding control parameter. The method of claim 2, wherein the action comprises at least one of a click, a move, a grab, and a rotation. The method of claim 12, wherein the step of detecting interaction between the learner and the physical image of the experimental device is to capture and identify an operation instruction generated by the learner using a manipulation unit. The method for teaching a virtual reality according to claim 17, wherein the control unit comprises at least one of a mouse, a joystick or a touch screen. The method for teaching a virtual reality according to claim 12, further comprising: selecting one of a plurality of experimental scenarios from a context setting area of the interface, and displaying, by the learner, the experimental context update selected by the learner The physical images of the experimental equipment corresponding to the experimental situation in the display area. The method for teaching a virtual reality as claimed in claim 19, wherein the experimental scenario comprises a physical experiment, a chemical experiment or a biological experiment. The method for teaching a virtual reality according to claim 12, wherein before detecting the interaction between the learner and the physical image of the experimental device, the experiment parameter is further set by a parameter setting region of the interface to output the corresponding control. Parameters to the server module. The method according to claim 12, wherein the interface further provides a chart setting area including at least one set of stored experimental data, at least one set of stored experimental results, an experimental formula, a table or a drawing. a template for the learner to select the experimental data or the experimental result that he wants to draw. The method according to claim 12, wherein each of the real-life films presents an experimental reality of the experimental result by time-lapse photography. A computer program product for storing a real-world teaching program, which can be completed by one of the claims 12 to 23 when the computer is loaded into the computer program and executed.