KR20120074977A - Educational materials and methods using augmented reality the performance of voice recognition and command - Google Patents
Educational materials and methods using augmented reality the performance of voice recognition and command Download PDFInfo
- Publication number
- KR20120074977A KR20120074977A KR1020100136977A KR20100136977A KR20120074977A KR 20120074977 A KR20120074977 A KR 20120074977A KR 1020100136977 A KR1020100136977 A KR 1020100136977A KR 20100136977 A KR20100136977 A KR 20100136977A KR 20120074977 A KR20120074977 A KR 20120074977A
- Authority
- KR
- South Korea
- Prior art keywords
- augmented reality
- image
- information
- display unit
- command
- Prior art date
Links
- 230000003190 augmentative effect Effects 0.000 title claims abstract description 94
- 238000000034 method Methods 0.000 title claims abstract description 41
- 239000000463 material Substances 0.000 title claims abstract description 40
- 230000008569 process Effects 0.000 claims description 25
- 238000009434 installation Methods 0.000 claims description 18
- 230000000007 visual effect Effects 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 10
- 239000003550 marker Substances 0.000 description 16
- 238000012545 processing Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 241000251468 Actinopterygii Species 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 230000002194 synthesizing effect Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 241000276427 Poecilia reticulata Species 0.000 description 1
- 241001441730 Triacanthidae Species 0.000 description 1
- 241000256856 Vespidae Species 0.000 description 1
- 210000001015 abdomen Anatomy 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000007519 figuring Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/06—Electrically-operated educational appliances with both visual and audible presentation of the material to be studied
- G09B5/065—Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/005—Language recognition
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Tourism & Hospitality (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Computational Linguistics (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The present invention provides a wireless unit for receiving a voice command of a user; A control unit which controls to display information of the teaching material photographed by Webchem and related information, and displays the image of the user and the information in combination; ; A display unit (screen) for displaying an image output from the controller, and a web cam connected to the controller to capture an image; The present invention relates to an augmented reality educational material and a method of using the same to store information on a photographed subject and perform voice recognition and commands consisting of a storage medium (USB or CD) and an augmented reality educational webcam storing a program for augmented reality. .
Description
The present invention uses a three-dimensional space to illuminate the teaching material with a camera connected to the control unit, three-dimensional in the display unit to make it easier to understand, and as the companionship displayed on the display by voice is converted to three-dimensional by augmented reality, with the speaker It relates to augmented reality educational materials for performing the voice recognition and commands to be described in and how to use the same.
With the recent development of science and technology, the educational field is also more easily understood by learners using advanced devices such as computers and projects.
Recently, devices using three-dimensional space have been developed to make three-dimensional images easier to understand in flat computers.
Augmented reality is a term derived from a virtual environment and a virtual reality, and refers to a mixture of a real world image and a virtual image by inserting a computer graphic image into the real environment. Real-world information may contain information that the user does not need, and sometimes the user may lack information. But with computer-generated virtual environments, you can simplify or hide information you don't need. In other words, the augmented reality system combines the real world and the virtual world to allow interaction with the user in real time.
The key to implementing augmented reality is to match your webcam with your OpenGL or DirectX camera. Matching cameras can be done in two ways.
First, match the internal parameters (intrinsic parameters) of the actual webcam, such as focal length and center point, and match the posture information (extrinsic parameters) where the webcam is located in the real space.
1. Matching Internal Properties
Basically, the camera calibration process is about figuring out the camera's 5 DOF internal properties, but once you have given constraints to this process to get the focal length, matching the internal properties is fairly easy. This is because the focal length is converted into fov using glPerspective () function in OpenGL. However, matching the internal properties of the 5 DOF is not a trivial task, so you should be familiar with Computer Graphics' Graphics Pipeline and be able to derive the calculations. This process converts the internal properties of the 5 DOF to a 4x4 Projection Matrix and glMatrixMode (GL_PROJECTION); glLoadMatrixd (4x4 matrix); Matching can be done through functions. If this is a burden, you can refer to the function that converts to ARToolkit.
2. Matching posture information
Posture information is a 6 DOF attitude (3 DOF for rotation, 3 DOF for positioning) of the target to be tracked, and there are two ways to match the attitude information.
Move the virtual camera directly using the glLookAt () function or glMatrixMode (GL_MODELVIEW); glLoadMatrixd (4x4 matrix); Using functions, the camera is fixed at the origin and instead moves a virtual 3D object. Again, ARToolkit is easy to understand.
If you match the two properties as above, you can see the augmented image.
In addition, the mobile augmented reality service can be classified into Location-based AR, Image Recognition-based AR, and Image Tracking-based AR.
1. Location-based AR: This is the type of mobile AR that is currently being used the most. It simply recognizes the location using GPS or Compass information and provides the appropriate service. My favorite favorites are Layar, Wikitude, Nearest Tube, Sekai Camera, and I need Coffee. Most augmented reality services are actually available only with location information, so this category is the most active.
2. Image Recognition-based AR: It's familiar to us through Google's Goggles or OlaWorks' ScanSearch. First, the target is taken and then the picture is sent to the server. The server recognizes the picture with Image Retrieval technology and sends the result back to the smartphone. This category includes Goggles, Point & Find, Kooaba, SREngine, ScanSearch, and other barcode or QR code searchers.
3. Image Tracking-based AR: These two categories are because virtual visual information does not blend seamlessly when augmenting information, and three-dimensional tracking of objects of interest is not achieved. It's not much active right now, but this category includes D'Fusion from Total Immersion, juneio from Metaio, and Qualcomm's augmented reality engine coming soon. At present, mobile tracking has not been activated much because the technical approach is not made in many places. However, the technical difficulties are almost solved by the results of the research of TU Graz's Mobile AR research team. It seems to be activated much in the next one or two years.
The simplest form of augmented reality is applied in the fields of entertainment and news. If you're forecasting on TV, you'll see the weather map in front of the weather caster changing naturally. In fact, the weather cast is in front of a blue-screen, and the computer-generated virtual footage creates a virtual studio environment to create augmented reality. It is done. In addition, three-dimensional data of the patient is acquired in real time through indirect tests such as magnetic resonance image, computed tomography image, and ultrasound image, and the acquired data is rendered in real time and overlaid with the patient. By doing so, the surgeon may perform a patient's operation smoothly, and a virtual image may be displayed on the visor of the pilot helmet or the windshield of the cockpit to inform the pilot of a lot of information about the manipulation.
Augmented reality used in such a learning system and the like will be described. In other words, virtual reality is an information activity that makes it possible to indirectly experience a situation that cannot be experienced directly in the real world due to spatial and physical constraints through interaction with the human sensory system in a virtual space constructed using a computer. One of the new paradigms in the field.
In recent years, the technology related to additional reality used in the learning system has been continuously developed.
For example, Korean Patent Publication No. 1009113760000 includes (a) recognizing a current location of a user; (b) outputting the recognized current location information of the user to a server storing image data according to a location, and receiving the image data output from the server corresponding to the current location information; (c) recognizing a direction that the user views; And (d) outputting the partial image data to be displayed to the user among the received image data on a screen composed of transparent electronic elements through which light input from the outside is transmitted based on the recognized direction information. The augmented reality implementation apparatus that does not require a complex optical structure is described using,
Publication No. 10-0868732 discloses context information through a business card, assigns it to a business card, and sends it to a business card information providing server. When a business card is viewed through a portable terminal, the relevant information of the business card party is provided. By requesting and downloading to a mobile terminal, you can plant dynamic information using the existing business card as it is so that the user can always update the information with the latest contents, and change or replace the existing media using various media. Multimedia business card information providing system and a method of providing the same using augmented reality that can easily grasp the information of the business card party without having described
Korean Patent Publication No. 100777215 provides a plurality of marker images indicating location information, and includes a user input unit for selecting at least one marker image from a user, and is portable for displaying the selected marker image. terminal; An image acquisition unit for acquiring a real world image including the portable terminal; A memory unit for storing a plurality of 3D graphic images; A marker image recognition unit for recognizing a marker image displayed on the portable terminal from the real-world image to obtain the marker image related information; An image processor configured to form an augmented reality image by synthesizing the 3D graphic image read from the memory unit on the real world image based on the marker image; And a display unit for displaying the augmented reality image received from the image processing unit.
Publication No. 100844238. Various routines may be provided and implemented via computer network 12, such as the Internet, to allow a large number of learners 44 to access and use the routines. The feedback functions take into account that the Internet is a medium in which it is difficult to directly observe the learner's behavior during the course of the various lessons provided by the routines. Routines can be implemented as an ordering of episodes, with each episode being systematically built on specific features learned in the preceding episode. Some of these routines are disclosed techniques, including setup routines, authentication routines, augmentation routines, synthesis routines, sequential synthesis routines, mastery routines, motivational routines, application routines and the ordering of these routines,
Korean Patent Publication No. 10-0911376 includes the steps of: (a) recognizing a current location of a user; (b) outputting the recognized current location information of the user to a server storing image data according to a location, and receiving the image data output from the server corresponding to the current location information; (c) recognizing a direction that the user views; And (d) outputting the partial image data to be displayed to the user among the received image data on a screen composed of transparent electronic elements through which light input from the outside is transmitted based on the recognized direction information. The augmented reality implementation apparatus that does not require a complex optical structure is described using,
In Korean Laid-Open Patent Publication No. 1020100073186, a posture estimation is performed on a marker printed on a textbook using a posture transformation matrix to maximize a marker recognition rate even in partial occlusion, shadows, and sudden changes in lighting. A recognition device and method thereof are disclosed,
The publication No. 1020090087622 includes (a) transmitting a billiard table photo image taken through a photographing unit in a mobile communication terminal to a billiard class server, (b) receiving a billiard table photo image from the billiard class server, the billiard table Analyzing the location and color of the billiard ball with reference to the photographic image; (c) obtaining class information corresponding to the billiard table photographic image with reference to previously stored class reference information in the billiard class server; and (d) Disclosed is a method for teaching billiards using augmented reality, comprising providing a mobile communication terminal with a lesson photographic image obtained by synthesizing the obtained lesson information with the received billiard table image using augmented reality in a billiard class server. It is
Publication No. 1020060065159 discloses a video data collection device for collecting live image data to be used as a virtual object, and extracts a marker ID by recognizing and tracking a marker to be used as a virtual object from the collected video data. An input image processing apparatus for generating an image, an output image processing apparatus and an output image processing apparatus for processing and outputting virtual object information generated through the image data collecting apparatus and the input image processing apparatus using augmented reality technology An apparatus for providing augmented reality and composite reality using a 3D marker recognition method including a display device for receiving and displaying image data is disclosed.
In Publication No. 10-2010-0118943, a user's image with a marker is taken to output image information, and the user's image excluding the screen background is extracted from the captured image information. The marker is recognized from the image information, and the position information and the prop information of the marker are extracted. Subsequently, it can be seen that an augmented reality foreign language experience learning system for selecting a background from a user and combining and displaying the extracted user's image and the prop information by referring to the location information of the marker on the background selected by the user is disclosed. have.
Conventional technologies have disadvantages that are difficult to recognize properly by various devices, lighting environments, and the like. Learners feel unnatural about printed markers, change the size of printed markers, and increase the recognition rate of markers placed in textbooks. It is a problem to be solved by the present invention to improve the inconvenience to be commanded in the keyboard and mouse.
The present invention, in order to solve the above problems, the present patent application No. 10-2010-117930 filed by the applicant, the name of the invention; It is an improvement of educational materials and methods of using augmented reality,
Connecting and installing a webcam and a storage medium to the control unit, installing a program, and installing the apparatus so that the teaching material and the web cam are directed to the 45 ° position; Setting up a folder by executing setup.exe inputted to the control unit, clicking the install button, and then installing a program of DirectX 9 by installing visual C ++ 2008 Redistributable; Clicking on a keyboard-shaped augmented reality icon on the upper portion of the textbook displayed on the display unit or controlling content with a keyboard; In accordance with the content controlled in the control step, the step of broadcasting a description in the speaker: An object to provide an educational material and a method using augmented reality consisting of.
The present invention by augmenting the image reflected by the web cam in the teaching material by the voice instruction, by controlling the content displayed on the display to appear augmented image, to realize the educational material, to reduce the cost of the educational material, the educational material to infinite At the same time, the contents of the screen are broadcast from the speaker, which has the advantage of maximizing the effectiveness of education.
1 is a view illustrating a storage medium connection of the present invention
USB terminal connection (Fig. 1A), CD installation connection (1B), webchem setting (1C), screen schematic (1D)
Figure 2 CD installation connection state diagram of the storage medium of the present invention
Initial screen (Fig. 2A), folder designation (Fig. 2B), installation button execution (Fig. 2C, Fig. 2D),
visual C ++ 2008 Redistributable Description (2E),
DirectX 9 installation (Figure 2F, Figure 2G).
3 is a detailed view of the keyboard for augmented reality voice recognition of the present invention
4 is an enhanced 3D content list of elementary science textbooks of the present invention
5 to 8 augmented reality use state diagram of science textbooks
9 is a flow chart using educational materials using augmented reality of the present invention
10 is a process chart using educational materials using augmented reality of the present invention
The present invention provides a wireless unit for receiving a voice command of a user; A control unit which controls to display information of the teaching material photographed by Webchem and related information, and displays the image of the user and the information in combination; ; A display unit (screen) for displaying an image output from the controller, and a webcam connected to the controller to capture an image; The present invention relates to an augmented reality educational material and a method of using the same to store information on a photographed subject and perform voice recognition and commands consisting of a storage medium (USB or CD) and an augmented reality educational webcam storing a program for augmented reality. .
Method of use of the present invention comprises the steps of connecting the webcam to the control unit with a USB terminal (or insert a CD instead of the USB terminal), the teaching material and the web cam toward the 45 ° position;
Specifying a folder to be installed by executing setup.exe set in the control unit, clicking the install button, and then installing DirectX 9 by installing visual C ++ 2008 Redistributable;
Receiving an augmented reality voice command of the image displayed on the display unit, the same image being captured by the web cam on the display unit;
Transmitting a signal received from the wireless unit to the controller to control the content to augment the information about the selected content (augmented reality icon) in conjunction with a USB terminal (or CD);
The present invention relates to an educational material and a method of using augmented reality, comprising: sending a signal to a display unit to control the content in the control step so that the display unit is displayed on the display unit.
According to the present invention, when one or two items are inputted in advance of a USB terminal or a CD for a video or audio corresponding to a corresponding page, and Webchem photographs the corresponding page, the control unit selects from the contents input to the USB terminal or CD. When the input page appears on the display, if you want to augmented reality by voice command from the displayed page (screen), the wireless unit transmits to the control unit, the control unit recognizes the voice, and inputs the data (USB terminal or CD input) Recognize the picture or page number, etc., and select the corresponding material (video and audio). Then, the selected video is augmented and displayed on the display. Use,
The next page is about how to use the teaching materials using augmented reality by instructor handing over the pages manually, webcam photographing the pages and repeating the above operation by voice command.
The device used in the present invention is a CPU of the control unit (computer): Pentium IV 2.4GHz or more, the memory is 250MB or more, the HDD is 10GB or more, and the soft program used is OS: MS Windows XP or more, and uses a webcam.
In addition, since the sopot program used in the present invention may use a general program used in the field of augmented reality, the present invention is not a core technology, and thus the content thereof will not be described in detail.
WebChem can be a common webcam used in personal computers. It may be implemented as a general-purpose computer or as a device having a separate display module. In other words, when webchem is implemented as a personal computer, a ubiquitous education system can be implemented at no additional cost.
The 3D augmented reality image and description data stored in the storage medium display an image by augmented reality on the display unit through a control unit. For example, all data such as information encoded in the photographed picture and data input by the learner may be augmented through a storage medium (USB, CD, etc.).
The wireless unit recognizes the voice and transmits it to the control unit and is installed on one side of the control unit.
The control unit installs a speech recognition processing unit and analyzes the voice according to the analyzed signal,
Select the data (video and audio) corresponding to the data on the USB terminal or CD inputted by the picture or page number, etc. shown on the page, and then augment the selected video so that it appears on the display. It sends a signal to the speaker to be broadcast so that it can be used as an educational textbook.
As shown in Figure 3, during the voice command, the speaking, recognition, retelling, command execution button to be lit, to show the current progress.
4 is a 3D content that can be experimented and observed in view of a science textbook of the first semester of the third grade elementary school through a webcam.
FIG. 5 is a screen showing page 23 of the first semester science textbook of FIG. 4, and when the augmented reality icon is installed on the upper right side of the screen and clicks the augmented reality icon, the 3D image of the augmented reality icon is explained. It comes out of the speaker.
The present invention will be described in detail with reference to the principle of operation of augmented 3D content that can be experimented and observed in view of the elementary science textbook through a webcam in detail as follows.
Example 1
First process (installation)
After connecting the webcam and the storage medium (USB terminal (Fig. 1A), CD installation (1B)) to the control unit, installing the program, and installing the device (Fig. 1C) to direct the textbook and webchem to the 45 ° position,
Run the setup.exe set on the initial screen (Figure 2A) on the control panel, specify the folder (Figure 2B), click the Install button (Figure 2C, Figure 2D), install Visual C ++ 2008 Redistributable and install DirectX 9 After installing the program CD (Fig. 2F, Fig. 2G),
Second Process (Content Control)
The control unit is operated to appear on the display unit in the same manner as the one captured by Webchem. Then, when the display unit shows a signal in the "Speak" signal in the image of "Talking, Recognizing, Performing a Command, Say Again," When the command of "Dear Belly Button" is displayed, a signal is displayed on the display unit in "Recognizing" in the process of selecting the corresponding image (augmented reality) and audio among the contents input to the USB terminal or CD by the control unit, and then the "Command" command on the display unit. When the "Operation" is displayed, the 3D shape of the daughter-in-law's augmented belly is enlarged in the center of the display part as shown in Fig. 5, and at the same time, "Look at the appearance and meaning of the daughter's navel." The training materials using augmented reality were used.
After manually turning over the next page, it was taken with WEBCEM and used educational materials using augmented reality in the same manner as the second process.
Zoom In, Zoom Out, Rotate, Listen, Pause
Listen to the information
Example 2
First process (installation)
After connecting the webcam and the storage medium (USB terminal (Fig. 1A), CD installation (1B)) to the control unit, installing the program, and installing the device (Fig. 1C) to direct the textbook and webchem to the 45 ° position,
Run the setup.exe set on the initial screen (Figure 2A) on the control panel, specify the folder (Figure 2B), click the Install button (Figure 2C, Figure 2D), install Visual C ++ 2008 Redistributable and install DirectX 9 After installing the program CD (Fig. 2F, Fig. 2G),
Second Process (Content Control)
The control unit is operated to appear on the display unit in the same manner as the one captured by Webchem. Then, when the display unit shows a signal in the "Speak" signal in the image of "Talking, Recognizing, Performing a Command, Say Again," When the "Gomari" command is issued, the control unit displays a signal in "Recognizing" in the process of selecting a video (augmented reality) and sound from the contents input to the USB terminal or CD, and then executes the "command" on the display unit. "Appears, the 3D shape of the Gomari augmented reality appears enlarged in the center of the display portion, and at the same time" a grateful plant to clean and clear the water of the river. " The training materials using augmented reality were used.
After manually turning over the next page, it was taken with WEBCEM and used educational materials using augmented reality in the same manner as the second process.
Zoom In, Zoom Out, Rotate, Listen, Pause
Listen to the information
Example 3
First process (installation)
After connecting the webcam and the storage medium (USB terminal (Fig. 1A), CD installation (1B)) to the control unit, installing the program, and installing the device (Fig. 1C) to direct the textbook and webchem to the 45 ° position,
Run the setup.exe set on the initial screen (Figure 2A) on the control panel, specify the folder (Figure 2B), click the Install button (Figure 2C, Figure 2D), install Visual C ++ 2008 Redistributable and install DirectX 9 After installing the program CD (Fig. 2F, Fig. 2G),
Second Process (Content Control)
The control unit is operated to appear on the display unit in the same manner as the one captured by Webchem. Then, when the display unit shows a signal in the "Speak" signal in the image of "Talking, Recognizing, Performing a Command, Say Again," When the "Early Fish" command is given to the display, a signal is displayed on the display unit in "Recognizing" in the process of selecting a corresponding image (augmented reality) and sound from the contents input to the USB terminal or CD, and then the "Command" command on the display unit. When the "Performance" is displayed, the 3D shape of the augmented fish augmented reality appears enlarged as shown in Figure 7 in the center of the display, and at the same time "observing the movement of the sighted fish." The training materials using augmented reality were used.
After manually turning over the next page, it was taken with WEBCEM and used educational materials using augmented reality in the same manner as the second process.
Zoom In, Zoom Out, Rotate, Listen, Pause
Listen to the information
Example 4
First process (installation)
After connecting the webcam and the storage medium (USB terminal (Fig. 1A), CD installation (1B)) to the control unit, installing the program, and installing the device (Fig. 1C) to direct the textbook and webchem to the 45 ° position,
Run the setup.exe set on the initial screen (Figure 2A) on the control panel, specify the folder (Figure 2B), click the Install button (Figure 2C, Figure 2D), install Visual C ++ 2008 Redistributable and install DirectX 9 After installing the program CD (Fig. 2F, Fig. 2G),
Second Process (Content Control)
The control unit is operated to appear on the display unit in the same manner as the one captured by Webchem. Then, when the display unit shows a signal in the "Speak" signal in the image of "Talking, Recognizing, Performing a Command, Say Again," When the "long wasp" command is issued, a signal is displayed on the display unit in "Recognizing" in the process of selecting the corresponding image (augmented reality) and audio from the contents input to the USB terminal or CD, and then the "command" command on the display unit. When the "Performance" is displayed, the 3D shape of the augmented reality wasp is magnified and appears in the center of the display as shown in Fig. 8, and at the same time, "observe the movement of the wasp." The training materials using augmented reality were used.
After manually turning over the next page, it was taken with WEBCEM and used educational materials using augmented reality in the same manner as the second process.
Zoom In, Zoom Out, Rotate, Listen, Pause
Listen to the information
Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.
Figure 1 is a storage medium connection state of the present invention, USB terminal connection (Fig. 1A), CD installation connection (1B), WebChem setting (1C), screen schematic (1D), Figure 2 CD installation connection state of the storage medium of the present invention Figure, initial screen (Figure 2A), folder designation (Figure 2B), installation button execution (Figure 2C, Figure 2D), visual C ++ 2008 Redistributable Description (2E), DirectX 9 installation (Figure 2F, Figure 2G), Figure 3 Voice recognition augmented reality keyboard detail of the present invention, Figure 4 augmented 3D content list of the elementary science textbook of the present invention, Figures 5 to 8 science textbook augmented reality use state diagram, Figure 9 education using augmented reality of the present invention Material use flow chart, Figure 10 can be seen that showing the educational material use process using the augmented reality of the present invention.
The apparatus of the present invention includes a wireless unit for receiving a voice command of a user; A control unit which controls to display information of the teaching material photographed by Webchem and related information, and displays the image of the user and the information in combination; ; A display unit (screen) for displaying an image output from the controller, and a web cam connected to the controller to capture an image; It can be seen that the augmented reality training material for performing the voice recognition and command consisting of a storage medium (USB or CD) and the augmented reality education webcam is stored in the webcam to store information about the subject and the augmented reality program.
Claims (3)
A wireless unit for receiving a voice command of a user; A control unit which controls to display information of the teaching material photographed by Webchem and related information, and displays the image of the user and the information in combination;
A display unit (screen) for displaying an image output from the controller, and a web cam connected to the controller to capture an image;
Augmented reality education to perform the voice recognition and command characterized in that the web cam is configured to store information about the subject to shoot and a storage medium (USB or CD) and augmented reality education webcam is stored stored program material.
Connecting the webcam to the control unit with a USB terminal (or inserting a CD instead of the USB terminal), and installing the device so that the teaching material and webcam are directed to the about 45 ° position;
Specifying a folder to be installed by executing setup.exe set in the control unit, clicking the install button, and then installing DirectX 9 by installing visual C ++ 2008 Redistributable;
Displaying an image identical to the shape captured by the webcam on the display unit and receiving an augmented reality voice command of the image displayed on the display unit by the wireless unit;
Transmitting a signal received from the wireless unit to the controller to control the content to augment the information about the selected content (augmented reality icon) in association with the USB terminal (or CD);
In this step, the content controlled in the control step is signaled to the display unit to appear on the display unit and at the same time the description is broadcast from the speaker:
Method of using educational materials using augmented reality, characterized in that used.
First process (installation)
After connecting the webcam and the storage medium (USB terminal, CD installation) to the control unit, installing the program, and installing the device to direct the textbook and webcam to about 45˚ position,
In the control panel, execute setup.exe set in the initial screen of the display panel, specify the folder, click the Install button, install the program CD for installing DirectX 9 by installing visual C ++ 2008 Redistributable,
Second Process (Content Control)
The control unit is operated to appear on the display unit in the same manner as the one captured by Webchem. Then, when the display unit shows a signal in the "Speak" signal in the image of "Talking, Recognizing, Performing a Command, Say Again," When the "long wasp" command is issued, a signal is displayed on the display unit in "Recognizing" in the process of selecting the corresponding image (augmented reality) and audio from the contents input to the USB terminal or CD, and then the "command" command on the display unit. When the "Performance" is displayed, the 3D shape of the augmented reality wasp is magnified and appears in the center of the display as shown in Fig. 8, and at the same time, "observe the movement of the wasp." To be broadcast,
How to use the teaching materials using augmented reality, characterized in that after manually turning the next page, photographed with webcam and used in the same manner as the second process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020100136977A KR20120074977A (en) | 2010-12-28 | 2010-12-28 | Educational materials and methods using augmented reality the performance of voice recognition and command |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020100136977A KR20120074977A (en) | 2010-12-28 | 2010-12-28 | Educational materials and methods using augmented reality the performance of voice recognition and command |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| KR20120074977A true KR20120074977A (en) | 2012-07-06 |
Family
ID=46709029
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020100136977A KR20120074977A (en) | 2010-12-28 | 2010-12-28 | Educational materials and methods using augmented reality the performance of voice recognition and command |
Country Status (1)
| Country | Link |
|---|---|
| KR (1) | KR20120074977A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015182908A1 (en) * | 2014-05-27 | 2015-12-03 | 주식회사 버츄얼스톰 | Smart learning system using transparent display and method thereof |
| CN108681389A (en) * | 2018-05-11 | 2018-10-19 | 亮风台（上海）信息科技有限公司 | A kind of method and apparatus read by arrangement for reading |
| CN108735014A (en) * | 2018-06-04 | 2018-11-02 | 聆课教育科技(上海)有限公司 | A kind of multimedia teaching facility, system and the method that multimedia content is presented |
| KR20200022576A (en) * | 2018-08-23 | 2020-03-04 | 제이에스씨(주) | Apparatus for presenting interactive education contents based on augmented reality and artificial intelligence of voice recognition |
| WO2020256403A1 (en) * | 2019-06-19 | 2020-12-24 | (주) 애니펜 | Method and system for creating content on basis of vehicle interior image, and non-temporary computer-readable recording medium |
-
2010
- 2010-12-28 KR KR1020100136977A patent/KR20120074977A/en not_active Application Discontinuation
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015182908A1 (en) * | 2014-05-27 | 2015-12-03 | 주식회사 버츄얼스톰 | Smart learning system using transparent display and method thereof |
| CN108681389A (en) * | 2018-05-11 | 2018-10-19 | 亮风台（上海）信息科技有限公司 | A kind of method and apparatus read by arrangement for reading |
| CN108681389B (en) * | 2018-05-11 | 2021-12-14 | 亮风台（上海）信息科技有限公司 | Method and device for reading through reading device |
| CN108735014A (en) * | 2018-06-04 | 2018-11-02 | 聆课教育科技(上海)有限公司 | A kind of multimedia teaching facility, system and the method that multimedia content is presented |
| KR20200022576A (en) * | 2018-08-23 | 2020-03-04 | 제이에스씨(주) | Apparatus for presenting interactive education contents based on augmented reality and artificial intelligence of voice recognition |
| WO2020256403A1 (en) * | 2019-06-19 | 2020-12-24 | (주) 애니펜 | Method and system for creating content on basis of vehicle interior image, and non-temporary computer-readable recording medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103905809B (en) | Message processing device and recording medium | |
| Reyna | The potential of 360-degree videos for teaching, learning and research | |
| US8963916B2 (en) | Coherent presentation of multiple reality and interaction models | |
| CN111009158B (en) | Virtual learning environment multi-channel fusion display method for field practice teaching | |
| US20090204880A1 (en) | Method of story telling presentation and manufacturing multimedia file using computer, and computer input device and computer system for the same | |
| JP6683864B1 (en) | Content control system, content control method, and content control program | |
| KR20120074977A (en) | Educational materials and methods using augmented reality the performance of voice recognition and command | |
| CN111862711A (en) | Entertainment and leisure learning device based on 5G internet of things virtual reality | |
| US20020028002A1 (en) | System and method for storage, retrieval and display of information relating to specimens in marine environments | |
| US11758217B2 (en) | Integrating overlaid digital content into displayed data via graphics processing circuitry | |
| CN112947756A (en) | Content navigation method, device, system, computer equipment and storage medium | |
| Kostaras et al. | Assessing the usability of augmented reality systems | |
| CN114846808B (en) | Content distribution system, content distribution method, and storage medium | |
| CN114327083A (en) | House property, object display method and device, readable storage medium and electronic equipment | |
| Kuchelmeister et al. | The Amnesia Atlas. An immersive SenseCam interface as memory-prosthesis | |
| JP2011060130A (en) | Aquarium real-time pictorial book image display system and method | |
| JP2021009351A (en) | Content control system, content control method, and content control program | |
| CN104349065A (en) | Picture shooting method, picture shooting device and intelligent terminal | |
| US20220343783A1 (en) | Content control system, content control method, and content control program | |
| Shamsuddin et al. | An overview of augmented reality of underwater habitat | |
| US20230129708A1 (en) | Procedure guidance and training apparatus, methods and systems | |
| DeHart | Directing audience attention: cinematic composition in 360 natural history films | |
| Scheinerman | Exploring Augmented Reality | |
| Ucchesu | A Mixed Reality application to support TV Studio Production | |
| Umbelino | street ar t Development of an Augmented Reality Application in the Context of Street Art |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A201 | Request for examination | ||
| E601 | Decision to refuse application |