CN102411854B - Classroom teaching mixing technology application system based on enhanced reality and method thereof - Google Patents

Classroom teaching mixing technology application system based on enhanced reality and method thereof Download PDF

Info

Publication number
CN102411854B
CN102411854B CN201110255428.7A CN201110255428A CN102411854B CN 102411854 B CN102411854 B CN 102411854B CN 201110255428 A CN201110255428 A CN 201110255428A CN 102411854 B CN102411854 B CN 102411854B
Authority
CN
China
Prior art keywords
image
mark
augmented reality
technology application
answer
Prior art date
Application number
CN201110255428.7A
Other languages
Chinese (zh)
Other versions
CN102411854A (en
Inventor
周志颖
李超伦
谢毓斌
林士程
温舒蕾
徐小枫
郭巍
Original Assignee
苏州梦想人软件科技有限公司
明博教育科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州梦想人软件科技有限公司, 明博教育科技有限公司 filed Critical 苏州梦想人软件科技有限公司
Priority to CN201110255428.7A priority Critical patent/CN102411854B/en
Publication of CN102411854A publication Critical patent/CN102411854A/en
Application granted granted Critical
Publication of CN102411854B publication Critical patent/CN102411854B/en

Links

Abstract

The invention relates to a classroom teaching mixing technology application system based on enhanced reality and a method thereof. The system comprises that: an object printed with a marked surface; an image acquisition device used for acquiring an image of the object in a classroom environment; an image processing module used for processing and analyzing the acquired image; an image positioning module used for analyzing and calculating a coordinate of an identified mark in the acquired image; an image generation module used for placing a virtual object corresponding to the identified mark at a coordinate position of the mark in the image; a data statistics and analysis module used for counting and analyzing answer content; a display device used for displaying an image which is generated finally or a data statistics analysis result. According to the system and the method in the invention, a plurality of interaction modes in teaching can be realized, compared with a present classroom interaction feedback system, realization cost and needed expense of the system in the invention are obviously reduced, and flexibility of usage is substantially raised.

Description

Classroom instruction hybrid technology application system and method based on augmented reality

Technical field

The invention belongs to computing machine augmented reality and interaction technique in the mixing application of classroom teaching, be specifically related to a kind of classroom instruction hybrid technology application system and method based on augmented reality.

Background technology

Traditional classroom Story teaching normally carries out with the form of books and video-tape or lantern slide, and the means of education are more single.Although use video-tape or lantern slide, with respect to books teaching, the standardization level in classroom has obtained certain raising, after having no interactive broadcasting, still take teacher's instruction as main, still rests on the one-way transmission stage of information.The functipnal capability of this teaching to student, the culture effect of the aspects such as social quality, emotion will is very micro-.Because traditional form of teaching is interactive very poor, unilaterally by teacher, lectured, make a lot of students lose learning interest.Knowledge can not be applied in a flexible way, and also just cannot change ability into.Because its one-side teaching character, causes highly relying on teacher in teaching process, make quality of instruction stable not.

Existing classroom interaction feedback system is generally by electronic carrier (as PDA, mobile phone or telepilot), allows all students in classroom can interactive feed back to a kind of teaching application system of teacher.By teacher's telepilot, student's telepilot (everyone), receiver and supporting software etc., formed.It can test classroom instruction, classroom, vote, ludic activity, unexpectedly guess that learning teaching assessment etc. organically combines, on the basis of existing most of classroom " a computer Yi Bu projector ", both the irrealizable numerical digit interactive teaching of Traditional Classroom can be realized, and various forms etc. can be automatically corrected again, add up and generate.For example, the interactive feedback system of ChinaIRS (China Interactive Response System, be called for short: ChinaIRS), or other like products.

Although existing classroom interaction feedback system has changed traditional " force-fed duck " formula teaching pattern, highlight the Class characteristic of interaction flexibly, but, the input cost of its hardware facility is relatively high, teacher and each student need to distribute an electronic romote-controller, receive and treating apparatus the function singleness of these hardware facilities but also corresponding signal will be set, form is fixed, and cannot bring lesson application effect more flexibly.

Summary of the invention

The object of the invention is to the defect for traditional teaching way and existing classroom interaction feedback system, a kind of classroom instruction hybrid technology application system and method based on augmented reality is provided, improve the interactive of classroom instruction, make student in being rich in the teaching environment of temperament and interest, strengthen the understanding of knowledge and interest.

Technical scheme of the present invention is as follows: a kind of classroom instruction hybrid technology application system based on augmented reality, comprising:

The object that is printed on index face, is placed in classroom environment, print index face image can effectively be gathered and be identified;

Image modalities is printed on the object of index face at the image of classroom environment described in gathering, and by the image transmission collecting to computing machine;

Image processing module, for the image collecting is processed and analyzed, obtains the mark that matches with template in image, and identifies the implication of this mark representative;

Image locating module, the coordinate that is marked at the image collecting of identifying for analytical calculation;

Video generation module, for virtual objects corresponding to identified mark being placed in to the coordinate position of this mark of image, and merges mutually with the true environment in image;

Data statistics and analysis module, the student's who image processing module is identified for the question and answer link in teaching answer content is carried out statistics and analysis;

Display device, image or data statistic analysis result that the virtual objects finally generating for show image generation module and true environment combine together.

Further,, wherein, between the mark in the described object that is printed on index face and background color, there is high-contrast in the classroom instruction hybrid technology application system based on augmented reality as above, preferred version is artwork master code labeling, and artwork master code labeling is positioned at identified areas; Described identified areas is a border figure with reservation shape, Yi Bian preferred scheme is for being with square boundary frame jaggy.

Further, the implication of the representative of the mark in the described object that is printed on index face also comprises following information: pupilage, answer content.

Further, on the described object that is printed on index face, also comprise the option content of question and answer and/or the direct performance information of pupilage.

Further, the described object that is printed on index face is stationary state or motion state.

Further, the classroom instruction hybrid technology application system based on augmented reality as above, wherein, described image modalities is camera or other camera heads; Computing machine is provided with the interface module suitable with image modalities.

Further, in the image of described image modalities collection, comprise one or more objects that are printed on index face.

Further, classroom instruction hybrid technology application system based on augmented reality as above, wherein, described image processing module carries out data processing for every frame image of image modalities input, utilize the identified areas that is printed on the object of index face in image outline search technique identification image, image in identified areas is analyzed, obtained the picture material that the pixel of pixel and template matches, and determine the information of this picture material representative.

Further, the classroom instruction hybrid technology application system based on augmented reality as above, wherein, described image locating module carries out the calculating of three-dimensional coordinate for the mark in every frame image; And calculate transition matrix according to the unique point of mark.

Further, classroom instruction hybrid technology application system based on augmented reality as above, wherein, the three-dimensional coordinate of the mark that described video generation module is calculated according to image locating module, is put into dummy object in the three-dimensional system of coordinate at mark place by transition matrix.

Further, classroom instruction hybrid technology application system based on augmented reality as above, wherein, the pupilage that described data statistics and analysis module identify according to image processing module and answer content are added up answer result, accuracy to each student and every problem is analyzed, generate answer honor roll, flow to display device and show.

Further, the classroom instruction hybrid technology application system based on augmented reality as above, wherein, described display device is various electronic consoles or screen, or multimedia overhead projector equipment, computing machine is provided with the interface module suitable with display device.

Further, described display device is the Minitype digital projector integrated with camera, by the image projecting finally generating on the curtain or body of wall at camera rear.

A classroom instruction hybrid technology application process for augmented reality, comprising:

By image modalities, gather the image in classroom environment, in the image gathering, comprise the object that at least one is printed on index face;

By image processing module, the image gathering is processed, obtained being printed on the marking image in the object of index face, and marking image and template are matched, identify the implication of this mark representative;

Calculate the coordinate in the image collecting that is marked at of identifying;

Virtual objects corresponding to identified mark is placed in to the coordinate position of this mark of image, and merges mutually with the true environment in image;

If the information that the implication of mark representative comprises pupilage, answer content, the student's that image processing module is identified answer content is carried out statistics and analysis;

By display device, show the image that the final virtual objects generating and true environment combine together.

Further, classroom instruction hybrid technology application process based on augmented reality as above, wherein, the information acquisition card that the described object that is printed on index face is held for student, student is by the information acquisition card in mobile hand, the mark of the information acquisition card in image that image modalities gathers is subjected to displacement, thereby the coordinate position of the virtual objects that mark is corresponding is changed, form the effect that virtual objects moves.

Further, classroom instruction hybrid technology application process based on augmented reality as above, wherein, the information acquisition card that the described object that is printed on index face is held for student, in information acquisition card, also comprise the option content of question and answer and/or the direct performance information of pupilage, student shows information acquisition card corresponding to selected answer according to the content of question-and-answer problem, and by the marking image on image modalities Information Monitoring capture card.

Beneficial effect of the present invention is as follows: what the present invention adopted on image tracking technique is the tracking mode based on artificial target's thing, is applicable to being very much applied to teaching pattern, and student does not need to dress HMD, can be freely movable, freely mutual.Aspect hardware realizes, only a camera need to be installed before computer, student only need hold the card of simple material, this card is the information acquisition card of passive and wireless, by ripe image recognition technology, the marking image on card is processed and analyzed, just can realize the various interaction modes in teaching, it is embodied as this and compares obvious reduction with funds in need with existing classroom interaction feedback system, but the dirigibility of using significantly improves.In the middle of teaching practice, the present invention has embodied following significant advantage:

(1), using augmented reality as teaching aid, can make learner carry out glibly interaction in virtual and real environment;

(2) using augmented reality as teaching aid, by producing a kind of new teaching and learning strategy, under this pattern, even if student, without any computer experience, also can carry out interactive learning;

(3) learner can be immersed in learning content, has changed learner and has only faced traditional mode of learning of static text.

Accompanying drawing explanation

Fig. 1 is a kind of embodiment schematic diagram of the classroom instruction hybrid technology application system based on augmented reality;

Fig. 2 is the another kind of embodiment schematic diagram of the classroom instruction hybrid technology application system based on augmented reality;

Fig. 3 is a kind of pattern schematic diagram of information acquisition card in embodiment;

Fig. 4 is the another kind of pattern schematic diagram of information acquisition card in embodiment;

Fig. 5 is the schematic diagram of an instantiation of image processing, location and generative process;

Fig. 6 is that sign, the angle point of figure in embodiment extracts the method flow diagram that quadrilateral detects;

Fig. 7 is a kind of teaching scene schematic diagram of the specific embodiment of the invention;

Fig. 8 is the another kind teaching scene schematic diagram of the specific embodiment of the invention.

Embodiment

Below in conjunction with drawings and Examples, the present invention will be described in detail.

The invention discloses a kind of classroom instruction hybrid technology application based on augmented reality, what adopt is the presentation mode based on display device, industrial camera is used for to augmented reality environment, and by human-computer interaction technology Optimal performance and efficiency, createed a new teaching pattern that merges the augmented reality environment of real world and virtual reality.

As shown in Figure 1, the classroom instruction hybrid technology application system based on augmented reality, comprising:

The object 1 that is printed on index face, is placed in classroom environment, print index face image can effectively be gathered and be identified;

Image modalities 2 is printed on the object of index face at the image of classroom environment described in gathering, and by the image transmission collecting to computing machine 3;

Image processing module 31, for the image collecting is processed and analyzed, obtains the mark that matches with template in image, and identifies the implication of this mark representative;

Image locating module 32, the coordinate that is marked at the image collecting of identifying for analytical calculation;

Video generation module 33, for virtual objects corresponding to identified mark being placed in to the coordinate position of this mark of image, and merges mutually with the true environment in image;

Data statistics and analysis module 34, the student's who image processing module is identified for the question and answer link in teaching answer content is carried out statistics and analysis;

Display device 4, image or data statistic analysis result that the virtual objects finally generating for show image generation module and true environment combine together.

In the present embodiment, be printed on index face to as if a kind of information acquisition card of passive and wireless, can be papery or other simple material, with low cost, can when needing at any time to use, by printing equipment, print generation.In information acquisition card, key is to include obvious identifiable marker.Information acquisition card is held by student, in classroom instruction process, according to need to the using at any time of curricula interaction, and guarantees successfully to be gathered in entering classrooms image by image modalities.On the other hand, the object that is printed on index face is not limited to the card of being held by student, also can be arranged on some fixture in classroom environment, image modalities collects after its mark, equally can be by processing and calculating, on the position of image corresponding to this mark, produce the dummy object of any needs, thereby build more rich and varied image environment and show by display device.The object that is printed on index face can be stationary state, can be also motion state; In same classroom instruction scene, can be both one and use separately, can be also a plurality of uses simultaneously, there is very strong application flexibility.

Between mark in the described object that is printed on index face and background color, should there is high-contrast, so that the processing of image.Preferred version is artwork master code labeling, is preferably labeled as black, and its background is white, and artwork master code labeling is positioned at identified areas.Described identified areas is a border figure with reservation shape, so that the position at the good identified region of software place, preferred scheme is for being with square boundary frame jaggy on one side, be preferably square framework, on a limit of square framework, be provided with breach, this breach has destroyed the symmetry on border, thereby increases tagged diversity.

The form that the object that is printed on index face in present embodiment is information acquisition card, on card, the mark of identified areas is a kind of artwork master code labeling of two dimension.In general the implication of mark comprises following one or more information: virtual objects, pupilage, answer content.These corresponding information are all set in advance in computer software, and, in the image processing module of computing machine, also comprise markd standard form, so that carry out the coupling of image in image processing.

A kind of pattern design of information acquisition card as shown in Figure 3, only comprises the artwork master code labeling that is positioned at identified areas.This information acquisition card is generally used for classroom teaching role's the use of playing the part of, student holds an information acquisition card, after on card, the image of mark is effectively gathered, treated and calculating generates a virtual objects that this mark is corresponding on the relative position of image, as a kind of cartoon character, student changes the position of mark in image by mobile message capture card, thereby the coordinate of virtual objects is changed in time, produces the visual effect such as cartoon character motion on output display unit.When information acquisition card shifts out after the scope of image collection, show that the virtual objects on image just disappears automatically.That is to say, student can move flexibly by the simple manually cartoon character that card is controlled in image frame, thereby makes environment and the atmosphere of teaching become more active, interactive stronger.

The another kind of pattern design of information acquisition card as shown in Figure 4, comprise the artwork master code labeling that is positioned at identified areas, meanwhile, also comprise the option content (A in diagram) of question and answer, and the direct performance information of pupilage (student number, class, name).The scope of application of this information acquisition card is relatively wide, and the as above the same use of playing the part of as classroom teaching role of example on the one hand, can also be used as the answer sheet of classroom question and answer on the other hand.The mark of this information acquisition card, except can corresponding a kind of virtual objects, also must comprise following information: pupilage, answer content.In the question and answer link of classroom instruction, each student has the information acquisition card corresponding with the number of options of problem, for example 4, represent respectively A, B, C, D, the implication of the mark representative in each information acquisition card is set in advance in the software of computing machine.Option for a problem, student lifts own selected answer card, by image modalities, gathered, at image processing module, obtain after concrete image, by with the mating of template, the implication characterizing according to mark determines that whether each student's answer choice is correct, then, can carry out statistics and analysis to student's answer content, generate statistics, for example form answer honor roll, and show.Meanwhile, can also generate corresponding virtual objects according to the result of answer, be presented at the position of mark in the information acquisition card in student's hand of real image, to strengthen the interest of teaching question and answer.

The present invention does not have too strict requirement for image modalities, supports camera or other camera heads on most of market; Computing machine is provided with the interface module suitable with image modalities.Image modalities need to gather classroom instruction real scene image clearly, preferably has automatic focusing function flexibly.

The present invention is completed by computer software the processing of image and relevant computing, and this software also has the required operation pages of teaching simultaneously, shows various option of operation and function.For back-end data processing section, software comprises image processing module, image locating module, video generation module, data statistics and analysis module.

Image processing module carries out data processing to every frame image of image modalities input, and the processing of image should say very ripe prior art.For example, first image is carried out to binary conversion treatment, because the mark of identified areas on card is a kind of artwork master code labeling of two dimension, therefore, after binary conversion treatment, be conducive to search and the identification of mark.Next, the region that in image, all marks may exist is searched for, can be utilized the identified areas at present general image outline search technique recognition image.In the present embodiment, if search the square that meets identified region, and there are corresponding 8 unique points, so just think that this is the candidate of mark.Obtain after the candidate of mark, extract the content of mark, utilize ripe two dimensional image analysis technology to analyze the marking image in identified areas, if the pixel of pixel and template matches, proof has found the black and white mark needing, the mark corresponding informance setting in advance by computer-internal, just can determine the concrete meaning information of this marking image content representative.In teaching scene, the corresponding information of mark is generally the virtual objects relevant to the content of courses, or the answer of classroom question and answer, and student's identity.These information are all confirmed, and are used in order to the location generation of image or the statistical study of data in image processing module.

As shown in Figure 5, when image processing module completes image after the coupling of processing and mark, image locating module carries out the calculating of three-dimensional coordinate for the mark in every frame image, and calculate transition matrix according to the unique point of mark, the value of this transition matrix is for rendering to dummy object the three-dimensional system of coordinate at black and white mark place.

As a rule, there are two kinds of methods the position of Computer Analysis objects in images three-dimensional coordinate, and a kind of is as telltale mark by black and white sign (Marker), the algorithm calculated amount that this identification needs is fewer, advanced algorithm is comparative maturity also, but need to have sign, could locate.Another kind is as common picture, and as telltale mark, a large amount of calculated amount of this algorithm requirements computing machine, also exists the larger error of calculation simultaneously, so algorithm is not very ripe, is also the technique direction of studying present stage.Therefore, the present invention still uses black and white sign, as the basic algorithm of augmented reality.The idiographic flow of algorithm can be with reference to the last listed list of references [1] " calibration for a video-based augmented reality conferencing system. " of this paper.Simply be described below below:

1. sign, angle point extract quadrilateral and detect

As shown in Figure 6, first adopt the method for fixed threshold, image is carried out to binary segmentation.The application of considering tracker is mainly that light environment is more stable indoor, adopts dynamic threshold, or the separation mode of multiple threshold value.Next, bianry image is carried out to connected domain extraction, the connected domain less to area, directly gives up, and has so just improved efficiency of algorithm.

2. Marker Identity

After obtaining quadrilateral, enter into the cognitive phase of mark.First, the quadrilateral on screen need to be carried out to inverse transformation, obtain normalization square.Corresponding with four groups of points of the plane of delineation according to volume coordinate, can be in the hope of homography matrix H.First do not consider the direction of mark, utilizing tetragonal upper left point on the image that marker detection partly obtains is the first point, corresponding, clockwise successively with the left upper apex of the template image of hypothesis, by 4 correspondences respectively, can try to achieve homography matrix H.By H, carry out screen quadrilateral to the foursquare correction of standardization, to the square of proofreading and correct, get it on, under, a left side, four possible code areas, the right side, are a side of 1111, are the principal direction of mark, meanwhile, and obtain the code that this mark is corresponding.After having determined the code that mark is corresponding, the template image in the image at mark middle part and mark storehouse with same code is carried out to template matches, determine mark.

3. the tracking of mark

After marker is determined, can determine with four angle points and follow the trail of, obtain world coordinate system and sign coordinate system and camera coordinate system and sign coordinate system.When directly obtaining square mark failure from image, system will enter the tracking mode of mark, MXRToolkit realizes the tracking to mark by the corners Matching of image, this algorithm obtains the initial position of angle point by homography matrix corresponding to previous frame image, in present image, search for arest neighbors angle point, upgrade current corner location, and then by the method for RANSAC list of references [2] " a paradigm for model fitting with applications to image analysis and automated cartography. ", reject the point of matching error, obtain new homography matrix.

The three-dimensional coordinate of the mark that video generation module is calculated according to image locating module, is put into dummy object in the three-dimensional system of coordinate at mark place by transition matrix.As shown in Figure 5, according to the three-dimensional coordinate of mark, by virtual objects--teapot is placed into the coordinate position of mark.In the present embodiment, select the game render engine of torque3D, virtual object is placed among real image.Because dummy object has Static and dynamic, so can have an X-rayed in real world, rotation, controls dummy object, and angled image by natural mutual mode, embody, this is all the function that current torque3D game render engine can be realized completely.Add that particle, audio-visual texts can allow user leave deep impression.

In addition, operation for classroom teaching question and answer link, image processing module is transferred to data statistics and analysis module by the pupilage comprising in identified mark and answer content information, the pupilage that data statistics and analysis module identify according to image processing module and answer content are added up answer result, accuracy to each student and every problem is analyzed, generate answer honor roll, flow to display device and show.These functions are worked out in advance in software, in conjunction with corresponding interface operation, can be low-cost, and the convenient, flexible classroom question and answer interactive teaching of realizing, compare the existing classroom interaction feedback system that need to equip electronic romote-controller and there is extremely strong superiority.

Display device of the present invention can be various electronic consoles or screen, or multimedia overhead projector equipment, these facilities can be selected arbitrarily according to the actual conditions of classroom environment completely, as long as computing machine is provided with the interface module suitable with display device, just can be convenient to use.In order better to embody portable feature of the present invention, and superiority flexible and convenient to use, the present invention provides a kind of structure more reasonable in instantiation, the mentality of designing of the display device that adaptability is stronger, is conducive to, condition poor area simple and crude at any facility and realizes interacti ve teaching mode of the present invention.As shown in Figure 2, the display device providing in this embodiment is a kind of Minitype digital projector 5 integrated with camera (image modalities 2), and computing machine 3 adopts portable notebook computer, and whole teaching software is all arranged in notebook computer.The integration mode of this hardware just makes classroom instruction hybrid technology application system of the present invention in any environment, to be used arbitrarily.Teacher only need to distribute the information acquisition card (being printed on the object 1 of index face) of papery to student, this card can be printed at any time and be convenient to and carry.Before teaching starts, the preliminary work that need to do is exactly in the software of computer, necessary information to be set, and classmates hold the information acquisition card that also can have been identified by computer predefined.Classroom teaching, in needs, open camera 2 and micro projector 5, by image projecting on the curtain 6 or body of wall at camera rear, software in operation computer, image processing module 31, image locating module 32, video generation module 33, data statistics and analysis module 34 will be processed and computing automatically, thereby realize the scene interaction of various teaching processes.Visible, the present invention need to not arrange any specialized equipment in specific implementation in classroom, all hardware units only need teacher to carry, what be distributed to student is also card papery, cheap, do not worry in use can being damaged, do not need student to carry out any prior training, be highly suitable for young student and use yet.

Teaching classroom scene below in conjunction with concrete, conducts further description the application of technical solution of the present invention.

Example 1-reads aloud part in classroom

The present invention has designed the interface of reading aloud part in the operation interface of software, and teacher selects to operate reading aloud part, in interface menu, can eject the button of reading aloud paragraph, for teacher, selects which paragraph to read aloud.After having selected concrete paragraph, the output interface of display device can enter into and the three-dimensional model scene of reading aloud paragraph and mating, as shown in Figure 7.Now, participation is read aloud interactive student and is held information acquisition card, camera collection is after the image in classroom, by a series of processing and the calculating of the image processing module in computer software, image locating module, video generation module, what each classmate was played the part of reads aloud the position that cartoon character corresponding to role rendered to the mark place of its information acquisition card in image, like this, virtual objects just merges with actual teaching environment.In Fig. 6, three classmates' information acquisition card is just corresponding three cartoon characters on screen, in the time of each classmate's mobile message capture card, image processing module is by the processing to each frame image, make image locating module can determine at any time the three-dimensional coordinate position of mark and throw in virtual image on continually varying position, the corresponding three-dimensional cartoon image of mark will be moved simultaneously.Be accompanied by reading aloud of role, student utilizes three-dimensional cartoon role and the scene interaction of the information acquisition card operational correspondence in hand.Three-dimensional cartoon image shows as on the ground that stands in scene, and by information acquisition card black and white, be marked in screen and move up and down, thus the motion all around in scene of operation cartoon character.Meanwhile, in software interface, also provide recording function, as the upper right corner of Fig. 7 is provided with, recorded button, the moving whole process of student's operation sheet current flow angle gamut and dubbing of reading aloud can have been recorded, in order to follow-up, play at any time.As need return, can select the return push-button in the software interface upper left corner, make program turn back to previous menu interface.

Example 2-classroom question and answer part

The present invention has designed the interface of question and answer parts in the operation interface of software, teacher selects question and answer partly to operate, in interface menu, can eject the button of question and answer paragraph, teacher arbitrarily clicks wherein a section of question and answer interface, and interface will enter into the current scene of mating with paragraph.As shown in Figure 8, now, in student's hand, generally hold four information acquisition card, the design form of information acquisition card can be as shown in Figure 4, comprise an artwork master code labeling, meanwhile, also comprise the option content (A in diagram) of question and answer, and the direct performance information of pupilage (student number, class, name), four cards are respectively the option of A, B, C, D.Corresponding to each information acquisition card, black and white mark above it all at least will comprise the implication of this two aspect of answer content and pupilage, these implication information arrange in advance in computer software, to guarantee the accuracy of image information coupling.According to system problem, students utilizes the mark of the information acquisition card in hand and question-and-answer problem to carry out interaction.Due to, the mark that can have A, B, C, tetra-options of D in each child's hand, according to determined answer, as long as lift corresponding answer mark in the answer term of validity, can, by camera collection, by image processing module, the implication of mark representative be identified.Result after identification can be transferred into data statistics and analysis module, and the answer situation of each student or per pass problem is carried out to statistics and analysis.Teacher can click interface with mouse, carries out the question and answer of next topic, and the question and answer link with all exercise questions of the same manner cycling above finally can eject new interface---answer honor roll.In honor roll, can check the result statistics of per pass exercise question, comprise correct, answer mistake and invalid.Screenshot capture when can also check statistics.As wanted, return, click the return push-button in interface.In addition, in the question and answer link of classroom, image locating module in native system software and video generation module can generate corresponding virtual objects and render to the mark position on screen according to the design of prior program equally, as there is according to the correctness of student's answer the image of " √ " or " X " in the mark position of information acquisition card in image in Fig. 8, and can coordinate other effect, make the Teaching Atmosphere of classroom question and answer more active.

Obviously, those skilled in the art can carry out various changes and modification and not depart from the spirit and scope of the present invention the present invention.Like this, if of the present invention these are revised and within modification belongs to the scope of the claims in the present invention and equivalent technology thereof, the present invention is also intended to comprise these changes and modification interior.

List of references:

[1]Kato?H.and?BillinghurstM..Marker?tracking?and?hmd?calibration?for?a?video-based?augmented?reality?conferencing?system.In?IWAR’99:Proceedings?of?the?2nd?IEEE?and?ACM?International?Workshop?on?Augmented?Reality,page?85,Washington,DC,USA,1999.IEEE?Computer?Society.

[2]M.A.Fischler?and?R.C.Bolles.Random?sample?consensus:a?paradigm?for?model?fitting?with?applications?to?image?analysis?and?automated?cartography.Commun.ACM,24(6):381-395,1981.

Claims (16)

1. the classroom instruction hybrid technology application system based on augmented reality, comprising:
The object that is printed on index face, is placed in classroom environment, print index face image can effectively be gathered and be identified; The implication of the mark representative of the described object that is printed on index face comprises following information: pupilage, answer content;
Image modalities is printed on the object of index face at the image of classroom environment described in gathering, and by the image transmission collecting to computing machine;
Image processing module, for the image collecting is processed and analyzed, obtains the mark that matches with template in image, and identifies the implication of this mark representative;
Image locating module, the coordinate that is marked at the image collecting of identifying for analytical calculation;
Video generation module, for virtual objects corresponding to identified mark being placed in to the coordinate position of this mark of image, and merges mutually with the true environment in image;
Data statistics and analysis module, the student's who image processing module is identified for the question and answer link in teaching answer content is carried out statistics and analysis, be specially: the pupilage identifying according to image processing module and answer content are added up answer result, accuracy to each student and every problem is analyzed, generate answer honor roll, flow to display device and show;
Display device, image or data statistic analysis result that the virtual objects finally generating for show image generation module and true environment combine together.
2. the classroom instruction hybrid technology application system based on augmented reality according to claim 1, is characterized in that: between the mark in the described object that is printed on index face and background color, have high-contrast.
3. the classroom instruction hybrid technology application system based on augmented reality according to claim 2, is characterized in that: in the described object that is printed on index face, be labeled as artwork master code labeling, artwork master code labeling is positioned at identified areas.
4. the classroom instruction hybrid technology application system based on augmented reality according to claim 3, is characterized in that: described identified areas is a border figure with reservation shape.
5. the classroom instruction hybrid technology application system based on augmented reality according to claim 4, is characterized in that: Yi Bian described identified areas is for being with square boundary frame jaggy.
6. according to the classroom instruction hybrid technology application system based on augmented reality described in claim 2 or 3, it is characterized in that: on the described object that is printed on index face, also comprise the option content of question and answer and/or the direct performance information of pupilage.
7. the classroom instruction hybrid technology application system based on augmented reality according to claim 1, is characterized in that: the described object that is printed on index face is stationary state or motion state.
8. the classroom instruction hybrid technology application system based on augmented reality according to claim 1, is characterized in that: described image modalities is camera or other camera heads; Computing machine is provided with the interface module suitable with image modalities.
9. the classroom instruction hybrid technology application system based on augmented reality according to claim 8, is characterized in that: in the image of described image modalities collection, comprise one or more objects that are printed on index face.
10. the classroom instruction hybrid technology application system based on augmented reality according to claim 1, it is characterized in that: described image processing module carries out data processing for every frame image of image modalities input, utilize the identified areas that is printed on the object of index face in image outline search technique identification image, image in identified areas is analyzed, obtain the picture material that the pixel of pixel and template matches, and determine the information of this picture material representative.
The 11. classroom instruction hybrid technology application systems based on augmented reality according to claim 1, is characterized in that: described image locating module carries out the calculating of three-dimensional coordinate for the mark in every frame image; And calculate transition matrix according to the unique point of mark.
The 12. classroom instruction hybrid technology application systems based on augmented reality according to claim 11, it is characterized in that: the three-dimensional coordinate of the mark that described video generation module is calculated according to image locating module, is put into dummy object in the three-dimensional system of coordinate at mark place by transition matrix.
The 13. classroom instruction hybrid technology application systems based on augmented reality according to claim 1, it is characterized in that: described display device is various electronic consoles or screen, or multimedia overhead projector equipment, computing machine is provided with the interface module suitable with display device.
The 14. classroom instruction hybrid technology application systems based on augmented reality according to claim 13, it is characterized in that: described display device is the Minitype digital projector integrated with camera, by the image projecting finally generating on the curtain or body of wall at camera rear.
15. 1 kinds of classroom instruction hybrid technology application processes based on augmented reality, comprising:
By image modalities, gather the image in classroom environment, in the image gathering, comprise the object that at least one is printed on index face; The information acquisition card that the described object that is printed on index face is held for student, in information acquisition card, also comprise the option content of question and answer and/or the direct performance information of pupilage, student shows information acquisition card corresponding to selected answer according to the content of question-and-answer problem, and by the marking image on image modalities Information Monitoring capture card;
By image processing module, the image gathering is processed, obtained being printed on the marking image in the object of index face, and marking image and template are matched, identify the implication of this mark representative;
Calculate the coordinate in the image collecting that is marked at of identifying;
Virtual objects corresponding to identified mark is placed in to the coordinate position of this mark of image, and merges mutually with the true environment in image;
If the information that the implication of mark representative comprises pupilage, answer content, the student's that image processing module is identified answer content is carried out statistics and analysis;
By display device, show the image that the final virtual objects generating and true environment combine together, or data statistic analysis result.
The 16. classroom instruction hybrid technology application processes based on augmented reality according to claim 15, it is characterized in that: student is by the information acquisition card in mobile hand, the mark of the information acquisition card in image that image modalities gathers is subjected to displacement, thereby the coordinate position of the virtual objects that mark is corresponding is changed, form the effect that virtual objects moves.
CN201110255428.7A 2011-09-01 2011-09-01 Classroom teaching mixing technology application system based on enhanced reality and method thereof CN102411854B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201110255428.7A CN102411854B (en) 2011-09-01 2011-09-01 Classroom teaching mixing technology application system based on enhanced reality and method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201110255428.7A CN102411854B (en) 2011-09-01 2011-09-01 Classroom teaching mixing technology application system based on enhanced reality and method thereof

Publications (2)

Publication Number Publication Date
CN102411854A CN102411854A (en) 2012-04-11
CN102411854B true CN102411854B (en) 2014-01-29

Family

ID=45913914

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201110255428.7A CN102411854B (en) 2011-09-01 2011-09-01 Classroom teaching mixing technology application system based on enhanced reality and method thereof

Country Status (1)

Country Link
CN (1) CN102411854B (en)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2501925B (en) * 2012-05-11 2015-04-29 Sony Comp Entertainment Europe Method and system for augmented reality
CN103513753B (en) * 2012-06-18 2017-06-27 联想(北京)有限公司 Information processing method and electronic equipment
CN102821323B (en) * 2012-08-01 2014-12-17 成都理想境界科技有限公司 Video playing method, video playing system and mobile terminal based on augmented reality technique
CN102902710B (en) * 2012-08-08 2015-08-26 成都理想境界科技有限公司 Based on the augmented reality method of bar code, system and mobile terminal
CN103035135B (en) * 2012-11-27 2014-12-10 北京航空航天大学 Children cognitive system based on augment reality technology and cognitive method
CN103366610B (en) * 2013-07-03 2015-07-22 央数文化(上海)股份有限公司 Augmented-reality-based three-dimensional interactive learning system and method
CN103606310B (en) * 2013-10-29 2016-01-13 中兴通讯股份有限公司 Teaching method and system
US9704295B2 (en) * 2013-11-05 2017-07-11 Microsoft Technology Licensing, Llc Construction of synthetic augmented reality environment
CN104269079A (en) * 2014-09-30 2015-01-07 张启熙 Mobile terminal preschool education system and method based on mirror reflection and augmented reality technology
CN104732826A (en) * 2015-03-27 2015-06-24 宋晓宇 Augmented reality (AR) interactive teaching material and teaching resource cloud platform
CN105430311A (en) * 2015-11-16 2016-03-23 上海尚镜信息科技有限公司 Infrared tracking projecting system and method
CN106056109A (en) * 2016-07-30 2016-10-26 深圳市寒武纪智能科技有限公司 Storytelling robot based on computer vision
CN106128196A (en) * 2016-08-11 2016-11-16 四川华迪信息技术有限公司 E-Learning system based on augmented reality and virtual reality and its implementation
CN106780680A (en) * 2016-11-28 2017-05-31 幻想现实(北京)科技有限公司 Three-dimensional animation generation method, terminal and system based on augmented reality
CN106530858A (en) * 2016-12-30 2017-03-22 武汉市马里欧网络有限公司 AR-based Children's English learning system and method
CN107341991A (en) * 2017-05-17 2017-11-10 广州视源电子科技股份有限公司 Method for indicating position, operation device and display system for position instruction
CN107392961B (en) * 2017-06-16 2019-12-06 华勤通讯技术有限公司 space positioning method and device based on augmented reality
CN107705645A (en) * 2017-09-01 2018-02-16 合肥迅大信息技术有限公司 Teaching auxiliary system based on augmented reality
CN110119194A (en) * 2018-02-06 2019-08-13 广东虚拟现实科技有限公司 Virtual scene processing method, device, interactive system, head-wearing display device, visual interactive device and computer-readable medium
CN110119190A (en) * 2018-02-06 2019-08-13 广东虚拟现实科技有限公司 Localization method, device, recognition and tracking system and computer-readable medium
CN110119195A (en) * 2018-02-06 2019-08-13 广东虚拟现实科技有限公司 Visual interactive device and marker
CN108492632A (en) * 2018-03-23 2018-09-04 四川科华天府科技有限公司 A kind of Teaching System based on situated teaching

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101923791A (en) * 2010-05-31 2010-12-22 华中师范大学 Method for learning Chinese by combining reality enhancing technique and plane reading material
CN101976463A (en) * 2010-11-03 2011-02-16 北京师范大学 Manufacturing method of virtual reality interactive stereoscopic book
CN102110379A (en) * 2011-02-22 2011-06-29 黄振强 Multimedia reading matter giving readers enhanced feeling of reality
TWM407458U (en) * 2010-11-25 2011-07-11 Univ Kun Shan Interactive question-and-answer teaching materials which utilize augmented reality technology

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05100617A (en) * 1991-10-09 1993-04-23 Nec Corp Cai system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101923791A (en) * 2010-05-31 2010-12-22 华中师范大学 Method for learning Chinese by combining reality enhancing technique and plane reading material
CN101976463A (en) * 2010-11-03 2011-02-16 北京师范大学 Manufacturing method of virtual reality interactive stereoscopic book
TWM407458U (en) * 2010-11-25 2011-07-11 Univ Kun Shan Interactive question-and-answer teaching materials which utilize augmented reality technology
CN102110379A (en) * 2011-02-22 2011-06-29 黄振强 Multimedia reading matter giving readers enhanced feeling of reality

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
JP特开平5-100617A 1993.04.23

Also Published As

Publication number Publication date
CN102411854A (en) 2012-04-11

Similar Documents

Publication Publication Date Title
Seo et al. What is developmentally appropriate in early childhood mathematics education? Lessons from new research
Robertson Twelve tips for using a computerised interactive audience response system
Vazquez et al. Virtual and real world adaptation for pedestrian detection
Scheiter et al. The effects of realism in learning with dynamic visualizations
KR101383982B1 (en) Apparatus and method for augmented reality
Kliper-Gross et al. Motion interchange patterns for action recognition in unconstrained videos
Singhal et al. Augmented chemistry: Interactive education system
US20040121298A1 (en) System and method of capturing and processing hand-written responses in the administration of assessments
Martín-Gutiérrez et al. Design and validation of an augmented book for spatial abilities development in engineering students
Zhang et al. The development and evaluation of an augmented reality-based armillary sphere for astronomical observation instruction
US9584766B2 (en) Integrated interactive space
Radkowski et al. Augmented reality-based manual assembly support with visual features for different degrees of difficulty
WO2016051366A2 (en) Switching between the real world and virtual reality
CN104575142B (en) Seamless across the Media open teaching experiment room of experience type digitlization multi-screen
Santos et al. Augmented reality as multimedia: the case for situated vocabulary learning
CN103366610A (en) Augmented-reality-based three-dimensional interactive learning system and method
Bebis et al. Review of computer vision education
US20120322037A1 (en) Anomaly Detection Educational Process
CN102880360B (en) Infrared type multi-point interaction electric whiteboard system and blank Projection surveying method
CN104656890A (en) Virtual realistic intelligent projection gesture interaction all-in-one machine
Ciptadi et al. Movement pattern histogram for action recognition and retrieval
CN104395856B (en) For recognizing the computer implemented method and system of dumb show
Tarng et al. Development of a virtual butterfly ecological system based on augmented reality and mobile learning technologies
CN102521859B (en) Reality augmenting method and device on basis of artificial targets
Huang et al. Piano ar: A markerless augmented reality based piano teaching system

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
C41 Transfer of patent application or patent right or utility model
TA01 Transfer of patent application right

Effective date of registration: 20131218

Address after: Linquan street Suzhou City Industrial Park, 215000 Jiangsu province No. 399 Southeast University Wenchang hospital room 407

Applicant after: Suzhou Dream Man Software Technology Co.,Ltd.

Applicant after: Mainbo Education Technology Co., Ltd.

Address before: Linquan street Suzhou City Industrial Park, 215000 Jiangsu province No. 399 Southeast University Wenchang hospital room 407

Applicant before: Suzhou Dream Man Software Technology Co.,Ltd.

Applicant before: Beijing Transmission New Media Technology Co.,Ltd.

ASS Succession or assignment of patent right

Owner name: MAINBO EDUCATION TECHNOLOGY CO., LTD.

Free format text: FORMER OWNER: BEIJING TRANSMISSION NEW MEDIA TECHNOLOGY CO., LTD.

Effective date: 20131218

C14 Grant of patent or utility model
GR01 Patent grant
C56 Change in the name or address of the patentee
CP01 Change in the name or title of a patent holder

Address after: Linquan street Suzhou City Industrial Park, 215000 Jiangsu province No. 399 Southeast University Wenchang hospital room 407

Patentee after: Suzhou Dream Man Software Technology Co.,Ltd.

Patentee after: MAINBO EDUCATION TECHNOLOGY CO., LTD.

Address before: Linquan street Suzhou City Industrial Park, 215000 Jiangsu province No. 399 Southeast University Wenchang hospital room 407

Patentee before: Suzhou Dream Man Software Technology Co.,Ltd.

Patentee before: Mainbo Education Technology Co., Ltd.