CN103093654B - Double video camera interactive intelligent tracking teaching system - Google Patents
Double video camera interactive intelligent tracking teaching system Download PDFInfo
- Publication number
- CN103093654B CN103093654B CN201310033116.0A CN201310033116A CN103093654B CN 103093654 B CN103093654 B CN 103093654B CN 201310033116 A CN201310033116 A CN 201310033116A CN 103093654 B CN103093654 B CN 103093654B
- Authority
- CN
- China
- Prior art keywords
- teacher
- student
- module
- video camera
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Closed-Circuit Television Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a double video camera interactive intelligent tracking teaching system. The double video camera interactive intelligent tracking teaching system comprises an image collecting module, a video camera control module, a user interface interaction module, a thread interaction module, a teacher tracking module, a student testing module and a student tracking module. During the teaching process, a control main engine obtains images which are photographed by teachers and students in real time through a dual image acquisition card. A tracing algorithm and a detecting algorithm based on an image processing technique are used for calculating the results and choosing appropriate control strategies. Mutual change among signals are reasonably arranged, functions of the live video camera including swaying, pitching, zooming and focusing can be rapidly, flexibly and accurately achieved. Teachers are capable of locating a proper position in the shooting picture, and functions like teachers' automatic tracing and positioning, students' video mapping and offering a close-up view while questioning testing and writing on the blackboard are achieved. In the end, the automatic intelligent tracing teaching system of tracing the external disturbance, no specific behavior limits and shooting without specially-assigned person on guard is achieved.
Description
Technical field
The interactive intelligent tracing teaching system that the present invention relates to the technology such as a kind of combining image collection, moving object detection and tracking and camera control, belongs to computer vision instructional technology field.
Background technology
The agriculture engineering far away that the portion of receiving an education widelys popularize inspires, rely on the high speed development of modern internet technology, comprise unprecedented development and the application of change, E-learning technology of mode of learning, the Web-based instruction day by day universal etc., intelligent-tracking tutoring system puts it at each universities, middle and primary schools of China classroom teaching gradually in recent years, becomes one of principal mode of teaching skill training.Along with the fast development of target following technical research in image processing in recent years and theory on computer vision, and computing machine and related hardware equipment technology innovation speed with rapid changepl. never-ending changes and improvements and the reduction of production cost, intelligent-tracking tutoring system can not only allow each common teachers take real-time follow-up image being under no restraint in giving lessons, and have all kinds of interface are provided, the advantage of convenient operation, has obtained demand and application more widely in each teaching, training field.
For example, in the daily teaching of vast middle and primary schools, teacher can give lessons according to teaching custom at ordinary times, system will be carried out analytical calculation to its motion feature automatically, ensure the appropriate area of teacher's moment in shooting picture by the tracking strategy of efficient stable, then be made into multimedia file (courseware by teachers' instruction and with the process of student's interaction, teaching resource etc.), by means of the teaching platform such as the Web-based instruction and remote on-demand, the teaching resource of high-quality can be provided for backcountry student, and this system can also be government's meeting, corporate training, academic exchange, the industries such as basic education provide the solution of multimedia messages.
Common tutoring system scheme at present, as the indoor locating system of Guangzhou Ao Wei subelectron Science and Technology Ltd., by using the infrared reception bar of multiple infrared remote receiver compositions, utilize the principle of mutual send and receive signal between sensor to obtain target location coordinate information, but it does not relate to the identification to teacher and student action, and function ratio is more single; The multi-item panorama infrared locking and tracking camera shot system of Hangzhou Kai Rui Technology Co., Ltd., its main camera is equipped with infrared camera and therewith rotates, infrared target that can any angle of locking tracking, but while giving close-up shot to teacher's writing on the blackboard action, must detect by sensor special, and whole system comprises two tracking cameras, two infrared cameras, complex structure, teacher also must wear infrared transmitter during giving lessons; Beijing Tongbu Technology Co., Ltd. comprises a classroom feature video camera, an active infrared scan video camera and student's panoramic camera, realize the tracking to teacher by infrared detection technique, but only realize the function of photographing panoramic picture for student's video camera, system lacks interactivity; The framing system of Beijing Han Boer Information Technology Co., Ltd comprises teacher's positioning shooting machine, teacher's tracking camera, two orientation of student video cameras, a student trace video camera, there is good tracking and detect effect, but its whole system body complex structure, cost costliness, is unfavorable for installing and safeguarding;
Summary of the invention
The problem to be solved in the present invention is, in at present common system scheme, the tracking to teacher easily occurring is subject to that surrounding environment influence, teachers' instruction moving region are limited, interchange between teacher and student is less, need special messenger's compounding practice, and the defects such as system configuration complexity (be greater than two video cameras or need other sensor special part), have designed a kind of twin camera interactive intelligence tracing teaching system for imparting knowledge to students.
For realizing the object of present design, the invention provides a kind of twin camera interactive intelligent tracing teaching system.Site-plan as shown in Figure 1, comprising:
Image capture module: use two-way image pick-up card can be in real time and independently respectively to teacher's video camera and student's camera acquisition image, then the image collecting is stored in Installed System Memory frame by frame, the image wherein teacher's camera acquisition being arrived is as the input picture of teacher's tracking module or student trace module, and the image that student's video camera is collected is as the input picture of student's detection module; When camera control module receives when teacher or student's video camera are produced to the instruction of motion control, the image that image capture module can collect by renewal shows in real time in the display window of user interface interaction module;
Camera control module: under manual tracing mode, the region that operator observes for needs, only need be in the picture disply window of interface alternation module, use left mouse button to click this region, just can be to teacher's video camera sending controling instruction, make this region that goes to of its fast and stable, mouse beacon pulley is during to front slide, this region is amplified and checked, while sliding backward, can expand observed regional extent; Under automatic tracing mode, teacher's tracking module, student trace module and the student's detection module comprising in thread interactive mode can be coordinated switch and calculate according to corresponding determination strategy between teacher's tracking, student's detection and student trace flow process, the result obtaining according to it realizes the independent function of controlling to teacher's video camera and student's video camera, comprise teacher follow the tracks of during swing to teacher's video camera and convergent-divergent control, student stand up and while answering a question, student given the actions such as close-up shot;
User interface interaction module: for operation and the operation of user to system provides interface to support, comprise the selection of initiation parameter setting unit, tracking mode and the real-time demonstration to teacher's video camera in figure acquisition module and student's shot by camera image; Wherein, initial parameter arranges interface and comprises the height information to teacher, and classroom is long, the typing of high dimensional parameters, and concrete installation parameter is calculated and returned to the geometric relationship of installing according to system.On the other hand, can carry out Lookup protocol by the initial position when the motion tracking to teacher according to teacher's height; Then the image each frame being collected by image capture module shows in real time in the picture disply window of specifying, so that the result that user can detect teacher's tracking, student trace and student is observed the most intuitively; Tracking mode provides two kinds of selections: manually tracking mode and autotrack mode, in the time selecting manual tracking mode, for application request, can be in the shooting display window of teacher's video camera, according to corresponding rule, simple operations mouse just can completing user and system between interactive function, reach the object that classroom arbitrary region convergent-divergent is checked; In the time selecting autotrack mode, in initial parameter after setting completed, click after " bringing into operation " button, the operation of system and the operation for thread interactive module, camera control module and image capture module are all automatically to carry out, without extra manual operation, convenient succinct;
Thread interactive module: when system is selected automatic tracing mode, and click after " bringing into operation " button, to pass to teacher's tracking module for the coloured image collecting from teacher's video camera, through characteristic area identification, template extraction, set up mixed Gauss model, set up color of object histogram, maximum likelihood is estimated (EM), after the steps such as Kalman prediction search target, can obtain the concrete movable information of teacher and location parameter, and realize teacher's video camera is swung according to designed teacher's tracking strategy, the control that convergent-divergent and focusing etc. moves, for the coloured image collecting from student's video camera, passed to student's detection module, after color space conversion, smoothing processing, thresholding processing, Gauss's dynamic background are processed, obtain the testing result of student movement, and realize accordingly the function of the relevant action control to student's video camera, in the time that student's detection module detects that Students on platform writing on the blackboard is answered a question, by sending message to thread interactive module, switch teacher's tracking module to student trace module, and make teacher's tracking module temporary suspension, student trace module brings into operation, from teacher's camera acquisition to image pass through image gray-scale transformation, thresholding processing, background is wiped out after processing, the student's location parameter calculating, if meet certain threshold condition, control teacher's video camera and give close-up shot demonstration to student's writing on the blackboard, in the time detecting that under student, seat is gone back at dais, again carry out the switch operating of module thread, recover the function of teacher's tracking module, and student trace module is hung up,
Teacher's tracking module: the image gathering for teacher's video camera of Real-time Obtaining from image capture module, first in the first two field picture to whether existing teacher's characteristic area to judge, if exist and its characteristic area carried out to template modeling, point out mistake if do not exist, restart to detect, then mixed Gauss model is set up in teacher's feature templates region, and according to the distribution situation of its color histogram, the probability position coordinates of characteristic area being occurred by maximum likelihood estimation algorithm (EM) carries out likelihood to be estimated to calculate, and obtains the estimated position of teacher's characteristic area at current frame image, finally, teacher's characteristic area position coordinates in previous frame and current frame image is carried out to data recursive operation by Kalman filter, obtain the accurate estimated position parameter of teacher's characteristic area in present frame, can obtain thus positional information and characteristic area parameter during teacher movement, when the position coordinates serving as a teacher is moved, can control size and the direction that teacher's video camera moves difference according to its coordinate swings or pitching accordingly, if in computation process, detect that teacher position is not moved at the appointed time, and the detected value of teacher's characteristic area is less than certain threshold value, think that teacher is now carrying on the back body book, give feature enlarging lens by controlling teacher's video camera to its writing on the blackboard, to can show clearly teacher's writing on the blackboard in the display window of interface alternation module,
Student's detection module: the image gathering for student's video camera of Real-time Obtaining from image capture module, first carry out initialisation image pre-service, by image from color space conversion to gray space, then the residing classroom of student area image is set up to background model, obtained background model after filtering after device smoothing processing, is realized to the detection to student movement by this background model of real-time update; In the time having student to stand up or to produce other violate-action, will cause the variation of background model, by can be calculated the location parameter and the eigenwert size that produce moving region, again to accessed parameter information process thresholding processing, the correct operation that student stands up is just thought in the action only producing in the situation that meeting certain threshold condition, and control student's video camera and give close-up shot to the student that stands up, otherwise think the violate-action to detecting, ignore; Then the eigenwert of the moving region that produces change of background is calculated, determine whether the correct operation of answering a question for Students on platform by condition judgment, if really for Students on platform is answered a question, realize the switching of teacher's tracking module to student trace module by sending message to thread interactive module, and after cushioning by certain hour, again upgrade the background model in region, classroom and continue student movement to detect;
Student trace module: in the time that thread interactive module is obtained the testing result of the Students on platform writing on the blackboard answer being detected by student's detection module, realize the switching between teacher's tracking module and student trace module by message processing facility, hang up teacher's tracking module, and the student trace module that brings into operation; Now by the coloured image of teacher's camera acquisition, first pass through image binaryzation processing, by pixel is carried out to opening and closing operation, remove testing result is produced to the noise disturbing, then obtain the movable information during Students on platform by the interframe background method of wiping out, in the time detecting that student does not move beginning writing on the blackboard in region, dais, teacher's video camera is controlled to realize giving close-up shot to student's writing on the blackboard, and clear demonstration student writing on the blackboard in the display window of interface alternation module; In the time detecting that student's moving characteristic value exceedes certain threshold value, think that now student goes down dais and get back to seat, send message to thread interactive module, realize the switching again with teacher's tracking module, teacher's tracking module is brought into operation, and student trace module is hung up.
The present invention's advantage is compared with prior art:
(1) teacher's track algorithm of the present invention not only can detect tracking to teacher in the motion in blackboard and region, dais in real time, and can also judge the situation whether teacher carries out writing on the blackboard in blackboard region, if just in writing on the blackboard, control teacher's video camera its writing on the blackboard is provided to close-up shot.Teacher's tracking module of the present invention is during teachers' instruction, and the comprehensive scope trace that teacher is carried out comprises that it walks about at dais and go down the situation of dais while entering in the middle of student.
(2) this cover system that the tracking based on machine vision and pattern-recognition and detection algorithm are realized, restriction and constraint to teachers' instruction region and action are liberated to a great extent in actual use, compare with infra-red tracing system with active supersonic sounding technology, teacher is without wearing transmitter locating device, only need in the time that system starts, carry out the extraction of characteristic area template, and system is moved and also can not produced any health effect to teacher for a long time; Compared with wearing transmitter location technology, the present invention does not claim to the action during teachers' instruction and moving range, and teacher's pursuit movement region is more extensive; With compared with Multichannel fusion technology, the present invention has certain antijamming capability to the impact of shade and illumination, better to the light adaptation in classroom, and the tracking stability of algorithm is stronger.
(3) student's detection module of the present invention is not only detecting that student rises immediately, give and the student's close-up shot of standing up, and can be in the time detecting that Students on platform is write writing on the blackboard and answered a question, send message to main frame, send interactive type communication with teacher's track thread, thereby realize tracking to upper dais student and writing on the blackboard is given to the function of close-up shot.
(4) conventional teaching tracker, for operations such as the controls of camera lens, tends to need many people's cooperation, and after recording, also needs to carry out artificial processing processing.And in the present invention, can select according to the actual requirements tracking mode, when in autotrack mode lower time, comprising thread interactive module, camera control module and following the tracks of picture disply etc. is all automatically to complete, and carries out operation bidirectional without special messenger.
(5) the present invention only uses two video cameras, by a main frame, they are carried out to corresponding control, not only reduced system cost, and the initial parameter setting unit that user interface interaction module comprises can provide directly perceived easy help for system installation and initial parameter setting.System also provides proprietary extensions mouth for network direct broadcasting and teaching recorded broadcast, can, under the prerequisite of the system of guarantee favorable compatibility, expand its range of application and applicable scene.
Brief description of the drawings
Fig. 1 is the overall design drawing of a kind of twin camera interactive intelligent of the present invention tracing teaching system;
Fig. 2 is the design drawing of user interface interaction module in the present invention;
Fig. 3 is the design drawing of initial parameter setting unit in the present invention;
Fig. 4 is the process flow diagram of teacher's tracking module in the present invention;
Fig. 5 is the process flow diagram of middle school student's detection module of the present invention;
Fig. 6 is the process flow diagram of middle school student's tracking module of the present invention;
Fig. 7,8 is the scheme of installation in the present invention.
Embodiment
As shown in Figure 1, a kind of twin camera interactive intelligent of the present invention tracing teaching system is by 8 module compositions: image capture module, camera control module, user interface interaction module, thread interactive module, teacher's tracking module, student's detection module, student trace module.
Entirety implementation procedure is as follows:
(1) system initialization parameter set unit.In the process using in actual installation, for the size in different classrooms, the installation site of teacher's video camera and student's video camera all can be different, arrange by system initialization parameter, ceiling height and angle of inclination can obtain video camera installation time, better to follow the tracks of and detect effect in order to obtain in the later stage, the initial presetting bit of teacher's video camera need to be set according to actual conditions toward contact; For teacher's height information of input, the angle of inclination while also installation for calculating student video camera provides foundation, and after calculating, initiation parameter setting unit is amendment systematic parameter automatically, with the variation conforming.
(2) realization of camera control and image collecting function.According to the guidance of system initialization parameter, install after video camera, image capture module respectively by Color Image Acquisition card from teacher's video camera and student's camera acquisition image, be stored in frame by frame in the internal memory of computing machine, in needs, read successively, then in the teacher's video camera display window in User Interface and student's video camera display window, show in real time, for user is by visual and clear shooting picture.According to system, operation needs satisfied tracking condition, sends instruction by 485 buses to camera control module, through overregulating the steps such as viewing area size, focal length of camera, reaches desirable display effect.For the collection of image during camera motion, need to be by the movement velocity of video camera and the collection display speed of image capture module, the stability of guarantee picture disply and real-time be set.
(3) realization of teacher's tracking and student's detection, following function.
Manually under tracing mode, user only need by mouse just can be directly and in user interface the shooting picture in teacher's video camera display window carry out alternately, click left mouse button in the region of wanting to observe, just can send corresponding steering order to camera control module, if click fringe region, video camera can move to appointed place with speed faster, if click from compare Jin region, centre position, video camera can move to this place with slower speed.If need to obtain the picture rich in detail of writing on the blackboard on blackboard, only need the intermediate roller of sliding mouse forward, in the time sliding backward mouse roller, can expand the scope of observation, to observe from the overall situation process of giving lessons.
Automatically under tracing mode, first user need to arrange the height information that teacher is provided in module at initiation parameter, user interface interaction module can be according to teacher's height, and the initial position that automatically regulates teacher to follow the tracks of, to ensure that teacher is in the apparent centre position of teacher's video camera.Click after the button of " bringing into operation ", teacher's tracking module and student's detection module move simultaneously, wherein teacher's tracking module carries out detection and tracking in strict accordance with teacher's tracking strategy to teacher's motion, if teacher's transverse shifting, send instruction to teacher's video camera, make it swing to ensure the appropriate location of teacher's moment in picture disply according to certain speed, if teacher's writing on the blackboard gives close-up shot to its writing on the blackboard.If student's detection module detects that student stands up to answer a question and sends instruction to student's video camera, to standing up, student gives close-up shot, if detect, student goes on dais writing on the blackboard answer, send message to thread interactive module, by it, teacher's tracking module and student trace module are switched, student trace module brings into operation.If student trace module detects student's writing on the blackboard, give close-up shot to its writing on the blackboard, if detect, student goes back to seat, again sends message to thread interactive module, student trace module is switched to teacher's tracking module, recovered the function of teacher's tracking module.
(4) user interface output.On the main interface of user interface interaction module, be provided with teacher's video camera display window and student's video camera display window, the presetting bit of teacher's video camera arranges button, and teacher's video camera velocity of rotation button is set, can allow user according to the demand of oneself and wish to camera control module, image capture module and thread interactive module directly operate, then from follow the tracks of testing result display window, obtain corresponding shooting picture, thereby directly from the display window of teacher's video camera and student's video camera, teacher is followed the tracks of and student's detection, the process of following the tracks of is observed.
The specific implementation process of above-mentioned each module is as follows:
1. user interface interaction module
The implementation procedure of this module is as shown in Figure 2,3:
(1) after working procedure, there is main interface as shown in Figure 2, the left window is teacher's video camera display window, right side window is student's video camera display window, in functional area below main interface, left side comprises that tracking mode choice box, initiation parameter arrange button, button is set camera motion speed and teacher's preset positions of camera arranges button, middle for image capture module is released the button, system is moved button and exit button, right side is action button and student's testing result display box of camera control module; In tracking mode choice box, complete the selection to system keeps track mode by clicking different options;
(2) click initiation parameter window is set, can ejection system parameter interface be set as shown in Figure 3, in teacher's height edit box, insert teacher's height data, for ensureing tracking effect, acquiescence allows teacher's height between 150cm ~ 200cm, clicks teacher's height confirmation is set.In the size edit box of classroom, insert successively classroom length, height and the dais distance to blackboard, click after installation parameter calculating, system can be calculated installation results automatically, and by parameter display in the edit box of ceiling height and mounting inclination angle, after setting completed, click determined;
(3) set after system operational parameters, click " opening camera " button, start the image capture module to teacher's video camera and student's video camera, then click " bringing into operation " button, corresponding function in execution thread interactive module, what the result that detects, follows the tracks of with student for teacher's tracking will be real-time is presented in corresponding window interface;
(4) if the use of completion system only need be clicked the button that quits a program while exiting.
2. teacher's tracking module
Implementation procedure corresponding to this module be as shown in Figure 4:
(1) for from teacher's camera acquisition to image, first use sorter (Adaboost) in region, dais, teacher's characteristic area to be searched for, if find teacher's feature templates, it being set up to mixed Gauss model processes, and according to the distribution situation of its color histogram, the probability position coordinates of characteristic area being occurred by maximum likelihood estimation algorithm (EM) carries out likelihood to be estimated to calculate, and obtains the estimated position of teacher in image; If do not find, Resurvey image continuation search is searched;
(2) the estimated position parameter obtaining is merged by Kalman filter and current frame image parameter, obtain accurate location coordinate and the eigenwert size of teacher's characteristic area in next frame image, by the changes in coordinates of teacher's characteristic area being calculated to judge whether teacher moves, if coordinate figure is non-vanishing, think and produced and moved left and right during teachers' instruction, now send message to teacher's video camera, control moving left and right of teacher's video camera, if coordinate figure is zero, carry out next step detection;
(3) do not carry out, under the prerequisite of transverse shifting, the eigenwert of its characteristic area being calculated teacher, think that now teacher is carrying on the back body book if be less than threshold value, send message to teacher's video camera and give writing on the blackboard close-up shot; Return to detection continuation calculated characteristics area coordinate if be greater than this threshold value;
(4) teacher's the coordinate that vertically moves is detected, if be greater than dais position threshold, think that now teacher goes down dais, control teacher's video camera and be rocked to the presetting bit of region, classroom, if there is no this threshold value, continue to detect;
(5) serve as a teacher while movement in region, classroom, repeat equally above detecting step, if when teacher detected through dais threshold value, think that now teacher gets back to dais, again its characteristic area is detected and followed the tracks of, if there is no process, continue to follow the tracks of;
(6) whole process shows in the control action of teacher's video camera all being followed the tracks of the real-time teacher in user interface interaction module of to display window during teacher follows the tracks of.
3. student's detection module
The implementation procedure of this module is as shown in Figure 5:
(1) from student's camera acquisition to image, carry out initialisation image pre-service, by image from color space conversion to gray space, and the residing classroom of student feature regional images is set up to background model, obtained background model after filtering after smoothing processing, is realized to the detection to student movement by this background model of real-time update;
(2) in the time having student to stand up or to produce other disturbed motion, by calculating the location parameter and the eigenwert size that produce moving region, and to the parameter information process thresholding processing calculating, the correct operation that student stands up is just thought in the action only producing in the situation that meeting certain threshold condition, and control student's video camera and give close-up shot to the student that stands up, otherwise think the violate-action to detecting, ignore;
(3) eigenwert of the moving region that produces change of background is calculated, determine whether the correct operation of answering a question for Students on platform by condition judgment, if really for Students on platform is answered a question, realize the switching between teacher's tracking module and student trace module by sending message to thread interactive module;
(4) after certain surge time, student's detection module restarts to upgrade background, continues student's athletic performance to detect;
(5) whole process to the detection in student region with give the action such as close-up shot all by real-time showing in real time in student's detection window of user interface interaction module.
4. student trace module
The implementation procedure of this module is as shown in Figure 6:
(1) when bring into operation student trace module under the switching of thread interactive module time, by the coloured image by teacher's camera acquisition, first pass through image binaryzation processing, by pixel is carried out to opening and closing operation, can produce the noise disturbing to testing result to remove;
(2) obtain movable information and the position coordinates during Students on platform by the interframe background method of wiping out, judge whether student carries out writing on the blackboard action, in the time detecting that student does not move beginning writing on the blackboard in region, dais, teacher's video camera is sent to message and give close-up shot to realize to student's writing on the blackboard, otherwise continue to detect;
(3) in the time detecting that student's moving characteristic value exceedes certain threshold value, think that now student goes down dais and get back to seat, send message to thread interactive module, realize the switching again with teacher's tracking module, teacher's tracking module is resumed operation, and student trace module is hung up;
(4) whole process to student trace with writing on the blackboard is given to the actions such as close-up shot and all will in the display window of the real-time teacher's video camera shooting picture in user interface interaction module, show.
5. system is installed and is calculated
As shown in Figure 7,8, be the scheme of installation of twin camera interactive intelligent tracing teaching system, concrete computation process is as follows:
Camera is to the setting height(from bottom) of ceiling: H
0cm; Camera is to the height on ground: H
1cm; Outside, dais is to the distance of blackboard: S
0cm; The height in classroom: H cm; The length in classroom: S cm; Teacher's height is: L cm; The angle of pitch of camera is: θ °;
Can be calculated by simple geometry:
The minimum value of the camera angle of pitch is:
the maximal value of the camera angle of pitch is:
The setting height(from bottom) of camera is: H
0=H-H
1; The span that the angle of pitch is installed is:
Non-elaborated part of the present invention belongs to those skilled in the art's known technology; In addition the above; it is only the embodiment in the present invention; but protection scope of the present invention is not limited to this; any people who is familiar with this technology is in the disclosed technical scope of the present invention; can understand conversion or the replacement expected; all should be encompassed in of the present invention comprise scope within, therefore, protection scope of the present invention should be as the criterion with the protection domain of claims.
Claims (1)
1. a twin camera interactive intelligent tracing teaching system, is characterized in that comprising: image capture module, camera control module, user interface interaction module, thread interactive module, teacher's tracking module, student's detection module, student trace module;
Image capture module: use two-way image pick-up card can be in real time and independently respectively to teacher's video camera and student's camera acquisition image, then the image collecting is stored in Installed System Memory frame by frame, the image wherein teacher's camera acquisition being arrived is as the input picture of teacher's tracking module or student trace module, and the image that student's video camera is collected is as the input picture of student's detection module; When camera control module receives when teacher or student's video camera are produced to the instruction of motion control, the image that image capture module can collect by renewal shows in real time in the display window of user interface interaction module;
Camera control module: under manual tracing mode, the region that operator observes for needs, only need be in the picture disply window of interface alternation module, use left mouse button to click this region, just can be to teacher's video camera sending controling instruction, make this region that goes to of its fast and stable, mouse beacon pulley is during to front slide, this region is amplified and checked, while sliding backward, can expand observed regional extent; Under automatic tracing mode, teacher's tracking module, student trace module and the student's detection module comprising in thread interactive mode can be coordinated switch and calculate according to corresponding determination strategy between teacher's tracking, student's detection and student trace flow process, the result obtaining according to it realizes the independent function of controlling to teacher's video camera and student's video camera, comprise teacher follow the tracks of during swing to teacher's video camera and convergent-divergent control, student stand up and while answering a question, student given the actions such as close-up shot;
User interface interaction module: for operation and the operation of user to system provides interface to support, comprise the selection of initiation parameter setting unit, tracking mode and the real-time demonstration to teacher's video camera in figure acquisition module and student's shot by camera image; Wherein, initial parameter arranges interface and comprises the height information to teacher, and classroom is long, the typing of high dimensional parameters, and concrete installation parameter is calculated and returned to the geometric relationship of installing according to system; On the other hand, can carry out Lookup protocol by the initial position when the motion tracking to teacher according to teacher's height; Then the image each frame being collected by image capture module shows in real time in the picture disply window of specifying, so that the result that user can detect teacher's tracking, student trace and student is observed the most intuitively; Tracking mode provides two kinds of selections: manually tracking mode and autotrack mode, in the time selecting manual tracking mode, for application request, can be in the shooting display window of teacher's video camera, according to corresponding rule, simple operations mouse just can completing user and system between interactive function, reach the object that classroom arbitrary region convergent-divergent is checked; In the time selecting autotrack mode, in initial parameter after setting completed, click after " bringing into operation " button, the operation of system and the operation for thread interactive module, camera control module and image capture module are all automatically to carry out, without extra manual operation, convenient succinct;
Thread interactive module: when system is selected automatic tracing mode, and click after " bringing into operation " button, to pass to teacher's tracking module for the coloured image collecting from teacher's video camera, through characteristic area identification, template extraction, set up mixed Gauss model, set up color of object histogram, maximum likelihood is estimated (EM), after the steps such as Kalman prediction search target, can obtain the concrete movable information of teacher and location parameter, and realize teacher's video camera is swung according to designed teacher's tracking strategy, the control that convergent-divergent and focusing etc. moves, for the coloured image collecting from student's video camera, passed to student's detection module, after color space conversion, smoothing processing, thresholding processing, Gauss's dynamic background are processed, obtain the testing result of student movement, and realize accordingly the function of the relevant action control to student's video camera, in the time that student's detection module detects that Students on platform writing on the blackboard is answered a question, by sending message to thread interactive module, switch teacher's tracking module to student trace module, and make teacher's tracking module temporary suspension, student trace module brings into operation, from teacher's camera acquisition to image pass through image gray-scale transformation, thresholding processing, background is wiped out after processing, the student's location parameter calculating, if meet certain threshold condition, control teacher's video camera and give close-up shot demonstration to student's writing on the blackboard, in the time detecting that under student, seat is gone back at dais, again carry out the switch operating of module thread, recover the function of teacher's tracking module, and student trace module is hung up,
Teacher's tracking module: the image gathering for teacher's video camera of Real-time Obtaining from image capture module, first in the first two field picture to whether existing teacher's characteristic area to judge, if exist and its characteristic area carried out to template modeling, point out mistake if do not exist, restart to detect, then mixed Gauss model is set up in teacher's feature templates region, and according to the distribution situation of its color histogram, the probability position coordinates of characteristic area being occurred by maximum likelihood estimation algorithm (EM) carries out likelihood to be estimated to calculate, and obtains the estimated position of teacher's characteristic area at current frame image, finally, teacher's characteristic area position coordinates in previous frame and current frame image is carried out to data recursive operation by Kalman filter, obtain the accurate estimated position parameter of teacher's characteristic area in present frame, can obtain thus positional information and characteristic area parameter during teacher movement, when the position coordinates serving as a teacher is moved, can control size and the direction that teacher's video camera moves difference according to its coordinate swings or pitching accordingly, if in computation process, detect that teacher position is not moved at the appointed time, and the detected value of teacher's characteristic area is less than certain threshold value, think that teacher is now carrying on the back body book, give feature enlarging lens by controlling teacher's video camera to its writing on the blackboard, to can show clearly teacher's writing on the blackboard in the display window of interface alternation module,
Student's detection module: the image gathering for student's video camera of Real-time Obtaining from image capture module, first carry out initialisation image pre-service, by image from color space conversion to gray space, then the residing classroom of student area image is set up to background model, obtained background model after filtering after device smoothing processing, is realized to the detection to student movement by this background model of real-time update; In the time having student to stand up or to produce other violate-action, will cause the variation of background model, by can be calculated the location parameter and the eigenwert size that produce moving region, again to accessed parameter information process thresholding processing, the correct operation that student stands up is just thought in the action only producing in the situation that meeting certain threshold condition, and control student's video camera and give close-up shot to the student that stands up, otherwise think the violate-action to detecting, ignore; Then the eigenwert of the moving region that produces change of background is calculated, determine whether the correct operation of answering a question for Students on platform by condition judgment, if really for Students on platform is answered a question, realize the switching of teacher's tracking module to student trace module by sending message to thread interactive module, and after cushioning by certain hour, again upgrade the background model in region, classroom and continue student movement to detect;
Student trace module: in the time that thread interactive module is obtained the testing result of the Students on platform writing on the blackboard answer being detected by student's detection module, realize the switching between teacher's tracking module and student trace module by message processing facility, hang up teacher's tracking module, and the student trace module that brings into operation; Now by the coloured image of teacher's camera acquisition, first pass through image binaryzation processing, by pixel is carried out to opening and closing operation, remove testing result is produced to the noise disturbing, then obtain the movable information during Students on platform by the interframe background method of wiping out, in the time detecting that student does not move beginning writing on the blackboard in region, dais, teacher's video camera is controlled to realize giving close-up shot to student's writing on the blackboard, and clear demonstration student writing on the blackboard in the display window of interface alternation module; In the time detecting that student's moving characteristic value exceedes certain threshold value, think that now student goes down dais and get back to seat, send message to thread interactive module, realize the switching again with teacher's tracking module, teacher's tracking module is brought into operation, and student trace module is hung up.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310033116.0A CN103093654B (en) | 2013-01-28 | 2013-01-28 | Double video camera interactive intelligent tracking teaching system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310033116.0A CN103093654B (en) | 2013-01-28 | 2013-01-28 | Double video camera interactive intelligent tracking teaching system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103093654A CN103093654A (en) | 2013-05-08 |
| CN103093654B true CN103093654B (en) | 2014-12-03 |
Family
ID=48206171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310033116.0A Active CN103093654B (en) | 2013-01-28 | 2013-01-28 | Double video camera interactive intelligent tracking teaching system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103093654B (en) |
Families Citing this family (43)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103327250A (en) * | 2013-06-24 | 2013-09-25 | 深圳锐取信息技术股份有限公司 | Method for controlling camera lens based on pattern recognition |
| CN103905734A (en) * | 2014-04-17 | 2014-07-02 | 苏州科达科技股份有限公司 | Method and device for intelligent tracking and photographing |
| CN104065924A (en) * | 2014-06-23 | 2014-09-24 | 苏州阔地网络科技有限公司 | On-line synchronization classroom intelligent tracking method and system |
| CN104065923B (en) * | 2014-06-23 | 2017-05-17 | 阔地教育科技有限公司 | On-line synchronization classroom tracking control method and system |
| CN104038717B (en) | 2014-06-26 | 2017-11-24 | 北京小鱼在家科技有限公司 | A kind of intelligent recording system |
| CN104202529B (en) * | 2014-09-18 | 2018-07-24 | 深圳市中智科创机器人有限公司 | A kind of camera status control method and its device |
| CN104268887B (en) * | 2014-09-30 | 2017-07-11 | 中智科创机器人有限公司 | A kind of camera attitude scaling method and device suitable for the switching of many scenes |
| CN104574359B (en) * | 2014-11-03 | 2017-09-29 | 南京邮电大学 | A kind of student trace localization method based on principal and subordinate's video camera |
| CN104394363B (en) * | 2014-11-21 | 2018-03-23 | 阔地教育科技有限公司 | A kind of Online class director method and system |
| CN104869365B (en) * | 2015-06-02 | 2018-12-18 | 阔地教育科技有限公司 | A kind of mouse tracking method and device based on straight recording and broadcasting system |
| CN105578044A (en) * | 2015-12-22 | 2016-05-11 | 杭州凡龙科技有限公司 | Panoramic view adaptive teacher image analysis method |
| CN105611237A (en) * | 2015-12-22 | 2016-05-25 | 杭州凡龙科技有限公司 | Method for simulating five cameras by utilizing double cameras for education recording and broadcasting |
| CN106161930A (en) * | 2016-06-27 | 2016-11-23 | 乐视控股（北京）有限公司 | Camera control method and device |
| CN106454200A (en) * | 2016-08-10 | 2017-02-22 | 惠州紫旭科技有限公司 | Video interaction method and video interaction system based on scene switching |
| CN106657893B (en) * | 2016-11-10 | 2019-09-03 | 浙江蓝鸽科技有限公司 | Recorded broadcast method and system with intelligent handoff functionality |
| CN106780565B (en) * | 2016-11-15 | 2020-03-31 | 天津大学 | Multi-student sitting-up detection method based on optical flow and k-means clustering |
| CN106878759A (en) * | 2017-02-13 | 2017-06-20 | 广东小天才科技有限公司 | Multi-camera lens switching method and system in teaching live broadcast |
| CN106941580B (en) * | 2017-03-22 | 2019-12-03 | 北京昊翔信达科技有限公司 | The method and system that teacher student automatically tracks is realized based on single detective camera lens |
| CN107146499B (en) * | 2017-07-03 | 2021-01-05 | 江苏赛诚极云网络科技有限公司 | Teaching auxiliary device |
| CN107424169A (en) * | 2017-08-02 | 2017-12-01 | 北京翰博尔信息技术股份有限公司 | Orientation of student method and device |
| CN107452018B (en) * | 2017-08-02 | 2020-04-07 | 北京翰博尔信息技术股份有限公司 | Speaker tracking method and system |
| CN107809563B (en) * | 2017-11-30 | 2019-12-24 | 浙江大华技术股份有限公司 | Blackboard writing detection system, method and device |
| JP7218728B2 (en) * | 2017-12-21 | 2023-02-07 | ソニーグループ株式会社 | Control device and control method |
| CN108230781B (en) * | 2018-03-12 | 2021-01-01 | 淄博职业学院 | Multimedia education platform capable of realizing network synchronization |
| CN108881765A (en) * | 2018-05-25 | 2018-11-23 | 讯飞幻境(北京)科技有限公司 | Light weight recorded broadcast method, apparatus and system |
| CN108665737A (en) * | 2018-06-26 | 2018-10-16 | 吉林省瑞中科技有限公司 | Novel classroom projection lamp |
| CN108766067A (en) * | 2018-08-03 | 2018-11-06 | 青岛华师京城网络科技有限公司 | A kind of implementation method in contact classroom |
| CN109522846B (en) * | 2018-11-19 | 2020-08-14 | 深圳博为教育科技有限公司 | Standing monitoring method, device, server and standing monitoring system |
| CN109672836B (en) * | 2018-12-28 | 2020-11-24 | 黑龙江工业学院 | English learning computer video recording device |
| CN109635787B (en) * | 2019-01-22 | 2022-09-30 | 合肥市云联鸿达信息技术有限公司 | Tracking camera teaching system |
| CN110021033B (en) * | 2019-02-22 | 2023-05-30 | 新疆交投科技有限责任公司 | Target tracking method based on pyramid twin network |
| CN112446241A (en) * | 2019-08-29 | 2021-03-05 | 阿里巴巴集团控股有限公司 | Method and device for obtaining characteristic information of target object and electronic equipment |
| CN110933316A (en) * | 2019-12-12 | 2020-03-27 | 苏州杰胜通信息技术有限公司 | Teacher tracking teaching system based on double-camera interactive mode |
| CN111652045B (en) * | 2020-04-17 | 2022-10-28 | 西北工业大学太仓长三角研究院 | Classroom teaching quality assessment method and system |
| CN113763760A (en) * | 2020-06-02 | 2021-12-07 | 重庆艾艺荷马教育科技股份有限公司 | Multimedia fine arts teaching equipment based on internet |
| CN111835969A (en) * | 2020-07-06 | 2020-10-27 | 海信视像科技股份有限公司 | Interactive method for controlling angle of camera and display equipment |
| CN111754830A (en) * | 2020-07-07 | 2020-10-09 | 广西师范大学漓江学院 | Interactive teaching instrument for vocal music teaching |
| CN112804487B (en) * | 2020-12-30 | 2022-10-28 | 济南和普威视光电技术有限公司 | Double-path video target tracking method and system |
| CN112911150A (en) * | 2021-01-29 | 2021-06-04 | 广州合易威视信息科技有限公司 | Automatic snapshot method for high-definition human face in target area |
| CN113055604A (en) * | 2021-04-01 | 2021-06-29 | 四川新视创伟超高清科技有限公司 | Optimal visual angle video processing system and method based on 8K video signal and AI technology |
| CN113572965B (en) * | 2021-08-06 | 2023-04-07 | 广州市奥威亚电子科技有限公司 | Double-blackboard-writing tracking shooting method and device and shooting system |
| CN113910815B (en) * | 2021-11-29 | 2022-09-23 | 深圳市卓翼科技股份有限公司 | Integrated form blackboard |
| CN114245146A (en) * | 2021-12-02 | 2022-03-25 | 北京鸿合爱学教育科技有限公司 | Display picture switching method, device, equipment and computer readable storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020192631A1 (en) * | 2001-05-23 | 2002-12-19 | Chase Weir | Method and system for interactive teaching |
| JP2006163692A (en) * | 2004-12-06 | 2006-06-22 | Canon Inc | Print control apparatus and print control method |
| CN100499808C (en) * | 2007-02-07 | 2009-06-10 | 北京航空航天大学 | Method for controlling automatically video camera moving according image position of objects |
| CN201233664Y (en) * | 2008-06-27 | 2009-05-06 | 深圳市和强迪讯科技发展有限公司 | Apparatus for implementing interactive teaching by using network |
-
2013
- 2013-01-28 CN CN201310033116.0A patent/CN103093654B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN103093654A (en) | 2013-05-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103093654B (en) | Double video camera interactive intelligent tracking teaching system | |
| CN106598226B (en) | A kind of unmanned plane man-machine interaction method based on binocular vision and deep learning | |
| CN110853073B (en) | Method, device, equipment, system and information processing method for determining attention point | |
| CN103716595B (en) | Panoramic mosaic video camera and ball machine inter-linked controlling method and device | |
| CN107392120B (en) | Attention intelligent supervision method based on sight line estimation | |
| CN105049764A (en) | Image tracking method and system for teaching based on multiple positioning cameras | |
| CN103905734A (en) | Method and device for intelligent tracking and photographing | |
| CN102291569B (en) | Double-camera automatic coordination multi-target eagle eye observation system and observation method thereof | |
| CN105830426B (en) | A kind of video generation method and device of video generating system | |
| CN106791485A (en) | The changing method and device of video | |
| CN201845345U (en) | Facial expression identifying data collecting system based on active vision | |
| CN107452018B (en) | Speaker tracking method and system | |
| CN111242962A (en) | Method, device and equipment for generating remote training video and storage medium | |
| CN105578044A (en) | Panoramic view adaptive teacher image analysis method | |
| CN203608274U (en) | Intelligent broadcasting tracking system based on television language | |
| CN105611237A (en) | Method for simulating five cameras by utilizing double cameras for education recording and broadcasting | |
| CN112346798A (en) | Information acquisition method and device | |
| CN101895685A (en) | Video capture control device and method | |
| CN114387679B (en) | System and method for realizing sight estimation and attention analysis based on recurrent convolutional neural network | |
| CN106941580B (en) | The method and system that teacher student automatically tracks is realized based on single detective camera lens | |
| Huang et al. | One-shot imitation drone filming of human motion videos | |
| CN107909598A (en) | A kind of moving object detection and tracking method based on interprocess communication | |
| CN112668487B (en) | Teacher tracking method based on fusion of body fitness and human similarity | |
| CN108876824A (en) | Method for tracking target, device, system and ball machine | |
| CN103546679A (en) | Device and method for reducing separation of shot object from shooting window |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |