CN1459720A - Method of computer numeral teaching - Google Patents

Abstract

A computerized digital education method using a computer includes creating a data stream consisting of a multidata stream and a preamble data stream for the sound, stroke, image and text data, editing the data stream for inserting new data in it and deleting old data from it, and transmitting the data stream for dumping up it or down loading it in order to use them by user.

Description

The method of computer numeral teaching

(1) technical field

The technology of the relevant a kind of multiplex data stream of the present invention, relevant specifically a kind of computer numeral teaching method.

(2) background technology

The life that the internet has changed people is a undisputable fact.Internet such as same clearway link together everyone life.By the internet, people can be easily and are obtained easily any time and the information in any place.In commerce, research, education, and on the various fields, oneself is quite universal in the internet.Ecommerce is also in the ascendant.The study (E-study) of electronization simultaneously is also popular, and this also is the main problem of the present invention.

In traditional distance learning, school can dispatch the student that teaching materials such as textbook and handbook are given various places.Because the teaching material that textbook and handbook are just static, and lack interchange between the teachers and students, thereby effect can not show a candle to class and comes effectively when last teaching with interactive.Though it is sometimes comparatively effective to teach in the classroom, but be and inconvenience to remote student.

For this reason, the people of association area there's no one who doesn't or isn't makes great efforts research, to break through various technical bottlenecks, long-distance teaching can more be raised the efficiency in the hope of solution.Yet a lot of learning systems are arranged, such as: sealing cable system (CCS), it is provided with but quite expensive.Fortunately the internet is universal, a lot of traditional distance learning associations, such as: phoenix university (Phoenix University) has begun to provide the Online Learning program the nineties.Online Learning has progress open (Schedule-free), and the benefit of place open (Location-free) and repetitive learning is quite popular in computer teaching.

Utilize tool software, such as: the FrontPAGe of Microsoft, MacromediaFlash, or Click2LearnAssistant, user also can set up and carry out various dissimilar Online Learnings.Author (wright of teaching material) can select based on the following reasons:

1. validity-how to carry out effectively deciding according to the type of teaching material.It is better that some presents a series of webpage picture, and some is then better with the pattern of video tape.

2. cost-comprise works (authoring), maintenance and course allocated costs.

3. the use of instrument-some instrument needs special skilled.

Does 4. how effectiveness-user obtain course? is it an independent platform (platform)? the explorer of the Microsoft browser that needs which version? or the like.

5. if frequency range-from the family line, the author must consider the restriction of low frequency range.Most image data stream needs bigger frequency range, and transmission quality just can be accepted.

Other.

Although the form of Online Learning is not quite similar, we still can be divided into three kinds roughly.First kind, be the Online Learning of normal use, it is a succession of webpage picture of being made up of hyperlink.We claim that this is the webpage Online Learning.The author can use the FrontPage of Microsoft to set up webpage or use the PowerPoint of Microsoft to set up transparency (can be considered a kind of special webpage).Webpage can comprise different objects, such as: literal, image, voice or the like.

Second kind Online Learning is the form of stream multimedia.We claim that this is the Online Learning data stream, data stream is based on image, can be by such as the NetShow of Microsoft, or the instruments such as RealPlayer of RealNetworks set up, and animation then can be set up by MacromediaFlash or other various stream multimedia softwares.

The Online Learning of the third type is to mix the first two type earlier.We are called the form (web-pAGe-stream-based) of web data stream.For example, the author can use Microsoft's Producer beta version to set up the transparency of data flow diagram picture, as shown in Figure 1.The user can be in the film district 11 inserts suitable film, is used for explaining transparency content 13, and menu 12 then can be used to click and switches to an other transparency.

With delivering of general webpage, webpage is archives, can be shown by browser (browser).Wouldn't talk its actual utility, the advantage that this kind delivered is:

1. deliver be easy to set up and change-author can use such as: instruments such as FrontPAGe and PowerPoint are set up simple webpage.For simple webpage, this class instrument is suitable.Yet webpage needs compiling especially instrument if comprise the multimedia element, has also therefore increased the degree of difficulty of works (authoring).For example the author can use MacromediaFlash to set up the animation archives.At this moment, the author then needs study how to use this cover instrument, and may the expensive time make good animation archives.

Low-dimensional accomplish this-for simple webpage, the author may only need website (HTTP) servomechanism and suitable storage to get final product, and does not need to install special function to the servomechanism of website.

3. the operating system of platform independence-servomechanism can be form NT (form NT), Solaris, Linux or any other can provide HTTP and FTP function person.The user can use any browser, such as: explorer of Microsoft or Netscape are as the HTTP user side of any PC.

4. the size of the archives of low frequency range demand-most of simple and easy webpages only comprises the writings and image element, so and not very big.Therefore, the delay of download (handing down) archives is still acceptable.Otherwise if webpage has film or animation elements, then postponing may be very serious.

Why is the shortcoming that webpage is delivered? for general classroom, the author can transmit his knowledge via giving lessons.Teacher's content of teaching mainly comprises graphic extension literal on sound and the blackboard etc.The student must carefully pay attention to the class, and duty takes notes, and its form that receives information is to continue sound and the diagram having no to interrupt, in the classroom, the student does not need " initiatively (activate) ", can obtain explanation, certainly the student has problem to ask a question, can be directly and the teacher carry out interaction.For the delivering of webpage, webpage is delivered as the computer that coexists and is read the textbook of a hard-cover and hyperlink.Though webpage has sound, image or animation elements, it needs the user to touch usually.Under the most situation, it can't produce interaction with the author, says nothing of the explanation or the guidance that obtain the author.Webpage is delivered and is generally static or nonautomatic.In other words, in the time of most, webpage can't automatic coiling or jumping page or leaf.For a literal is main webpage, and its possibly can't clearly demonstrate relevant theme, and to want the user to stare at tiny word at screen for a long time be difficult.From the angle of watching, but the shortcoming explanatory memorandum that webpage is delivered is as follows:

1. passive and boring-as, to be similar on the line and to illustrate that it is passive and boring that general user only can feel because there is not the sound explanation to instruct.

2. rhembasmus-in the process of study, the user needs to rotate, by touch the menu knob obtain they want to see or tin thing.Even when low frequency range and network congestion, the user also must stand owing to download (toward uploading) webpage, sound and image component and produce the interruption that intermittently postpones.

Efficient poor-with the graphic extension in classroom more as can be known, the information that obtains from the literal webpage often can't make the people grasp emphasis.

Because the time and interrupt or discontinuous, as and if imperfection utilize simple webpage to come impart knowledge or education activities.Thereby, develop one and be similar to the data stream of teaching between the classroom, can constantly transmit information, its urgency is really arranged.

Because the progress of voice, image and other compress technique, much companies and school bring into use the introduction of data stream pattern or web data flow pattern formula (web-page-stream-based) to deliver to be used as online teaching.

Generally speaking, the user clicks hyperlink (ActiveX or insertion (the Plug-In)) element on the webpage, can be connected to the medium servomechanism of number of site.ActiveX or insertion element can produce a new form or open the content that a new application is showed data stream usually in the browser the inside.After receiving the request of user side, the medium servomechanism promptly begins the content of data stream is presented on the display window.The content of data stream can be pure sound, pure image, sound and image or animation or the like.As for deliver (presentation) of web data flow pattern formula (web-pAGe-stream-based), except aforesaid data stream pattern, the medium servomechanism can transmit the word string order and give user side.For example, web data flow pattern formula (web-pAGe-stream-based) introduction is delivered the NetShow medium servomechanism that is to use Microsoft and is pushed (push) sound/image, or explanation sound (sound and image) is to user side.The content of NetShow is to be saved as an active data stream format (ASF, Active Streaming Format).ASF is the transmission shelves form of a low burden (Low-Overhead), can be as embedded literal (for example, URL) the same transmitting multimedia data (image, sound and image).ASF can make the synchronization of data streams of dissimilar contents.Can use embedded literal as order or indication, on still image (transparency), draw lines, circular and other simple geometric figures.

Clearly, still can't be based on the data stream of sound as Online Learning, because visual information is obviously quite important for teaching.Animation data stream is set up by special edit tool usually, as MacromediaFlash, and need take a moment and learns.It is used as brief delivering occasionally.Hence one can see that, and it is not appropriate utilizing sound-image-demand (Audio-Video-On-Demand) (be called for short and claim the image demand) to be used as the only data stream format of Online Learning, and certainly, its extension then comprises the transparency that is used to illustrate.

Image demand (ON-DEMAND) often is to be used for information broadcast (broadcasting) on the line, and it can be considered film on the line.The home education of using video tape to illustrate as a supplement is effective.Up to the present, the online teaching of similar video tape, image demand (ON-DEMAND) is more received than being easier at last, at least, and delivers (presentation) on the line of webpage pattern (WEB-PAGE-BASED) and compares, really so.In addition, image demand (ON-DEMAND) is also made and is recorded than being easier to.In fact, the video tape digital processing can be created as deliver (presentation) of image demand (ON-DEMAND).Yet use the online teaching of image demand (ON-DEMAND) that a lot of difficulties is still arranged, at least, with present network quality, still difficult the realization.It can ascribe following factors to:

1. frequency range-digital picture occupies a large amount of spaces usually.That is transmitted image demand (ON-DEMAND) data stream need be high transfer rate than general internet frequency range.Such as: MPEG and H.263 even present most compression of images algorithm, be sent to the acceptable image quality, and its minimum frequency range per second still must surpass tens of or hundreds of K bits.Though, to play for information, distinct image is not that youngest is important sometimes.Concerning the user who uses slow the Internet because network congestion and buffering, generally all can arrive meet with intermittent suspend and offending intermittently and the problem of the view data frame (frame) of low-res.

2. resolution-as mentioned above, image demand (ON-DEMAND) directly carry out digital processing by the simulation video tape usually to be finished, and the resolution of most simulation band shadow bands is limited, such as: the 320*240 of TV.Therefore, be difficult to be clear that the picture that is stored in image demand (ON-DEMAND) data stream.

3. cost-digitized image demand (ON-DEMAND) needs good equipment and big storage area.It may also need more special functions to realize, such as: Microsoft's medium servomechanism must be installed on the world wide web servomechanism, is used for the data stream of distribution diagram as demand (ON-DEMAND).

Interaction function limited-image demand (ON-DEMAND) user terminal system of most of commerce, such as: RealPlayer is designed primarily to information transmission and music (broadcasting).Therefore, the limited ability of user's interaction is in most systems.For example, commercial system there is no provides any position that can allow the user rotate back into data stream, row row content, the storage data line of wandering about as a refugee to browse, or sends the function of the response or the like of sound or image immediately back to.

In traditional classroom environment, teacher's teaching sound and the graphic extension on the blackboard are the effective forms of lecturing.But because the resolution problem, the picture of recording image is usually clear inadequately.Situation is with even worse when it is digitized into image demand (ON-DEMAND) and delivers (presentation).So the new method that can know record diagram sound is with desirable.

As mentioned above, the sound demand (ON-DEMAND) of graphic extension (transparency displaying) can be set up by tool software, and as the NetShowASF of Microsoft, this may be near a system that we expected.Yet in this system, can only store simple drawing command, for the still not enough elasticity of random drawing to the ASF data stream.

(3) summary of the invention

The objective of the invention is to provide a kind of method of computer numeral teaching for the shortcoming that online teaching is commonly used in improvement, but internet usage and computer are simulated traditional teaching in classroom, teacher's the sound and the diagram note of computer blank (whiteboard) can be made a synchronous multiplex data stream.Data stream and webpage can be delivered on the website, download and play this data stream for the student, and the user can set up sound diagram data stream course, plays smoothly on line, even also can carry out in slow line speed.So-called simulation,

Among the present invention, author's energy recorded voice and arbitrary graphic are on computer blank (whiteboard).The size of blank (whiteboard) is changeable.The sound that the compressible mixing of system is recorded with draw into data stream, use spreading on the internet.Except this basic purposes, the present invention still has much more extremely functions, and for example, when recording, the author can dynamically insert image and literal to blank (whiteboard), and deletes them afterwards.The user can change drawing (drawing), image and literal to webpage.In addition, not only the author can go up drawing at blank (whiteboard); He also can directly draw on any webpage.Advantageously, the user can set up above the drawing note and the sound of more than one file, picture or HTML (Hypertext Markup Language) (html) webpage and tell about.It should be noted that also and can be simply the file layout of a lot of Microsofts to be saved as webpage.At this moment, blank (whiteboard) is regarded as blank webpage.About the foundation of data stream, the present invention be called sound diagram data stream (Audio-Graph Stream, AGS).The figure that the user sets up can be via drawing with sound and inserting image and the literal element is finished.The present invention defines form and the grammer of AGS, provides a subsystem of recording to set up AGS and encodes and play AGS with playing subsystem, and the website that is positioned at this locality or far-end all can be carried out.The present invention just one is not used for setting up on the line data stream and delivers (presentation) tool software, and its also is to set up the tool software of CD-ROM teaching material and the tool software of an exchange of technology simultaneously.

For further specifying above-mentioned purpose of the present invention, design feature and effect, the present invention is described in detail below with reference to accompanying drawing.

(4) description of drawings

Fig. 1 illustrates the example that the conventional projection sheet is showed teaching.

Fig. 2 (a) is the blank synoptic diagram of a preferred embodiment of the present invention.

Fig. 2 (b) is that the webpage of a preferred embodiment of the present invention adds the transparent window synoptic diagram.

Fig. 3 is the ejection sketch plate synoptic diagram of a preferred embodiment of the present invention.

Fig. 4 is the control panel synoptic diagram of a preferred embodiment of the present invention.

Fig. 5 (a) is sound diagram data stream (AGS) synoptic diagram of a preferred embodiment of the present invention.

Fig. 5 (b) is that the AGS of a preferred embodiment of the present invention is segmented into several SDS synoptic diagram.

Fig. 6 is the advance data flow diagram of a preferred embodiment of the present invention.

Fig. 7 (a) is the linked web pages synoptic diagram of a preferred embodiment of the present invention.

Fig. 7 (b) is the embedded webpage synoptic diagram of a preferred embodiment of the present invention.

Fig. 8 (a) is the AG course synoptic diagram of a preferred embodiment of the present invention.

Fig. 8 (b) is the embedded AG course of the C synoptic diagram of a preferred embodiment of the present invention.

Fig. 8 (c) is the C link AG course synoptic diagram of a preferred embodiment of the present invention.

Fig. 9 is the data structure of a preferred embodiment of the present invention.

Figure 10 is the user computer framework of a preferred embodiment of the present invention.

Figure 11 is the mode switch synoptic diagram of a preferred embodiment of the present invention.

Figure 12 (a) is the synoptic diagram that the stroke time-frame of a preferred embodiment of the present invention imports data stream.

Figure 12 (b) is the synoptic diagram of the stroke time-frame content of a preferred embodiment of the present invention.

Figure 13 is the data stream and the incident array synoptic diagram of a preferred embodiment of the present invention.

Figure 14 is idle (idle) state figure use surface chart down of a preferred embodiment of the present invention.

Figure 15 is the synoptic diagram of the properties dialog of a preferred embodiment of the present invention.

Figure 16 is that the figure of recording subsystem of a preferred embodiment of the present invention uses the interface synoptic diagram.

Figure 17 is that the figure of the broadcast subsystem of a preferred embodiment of the present invention uses surface chart.

(5) embodiment

At first, the present invention allows the user to record and play a sound figure note on a blank (whiteboard) or a webpage, shown in Fig. 2 (a) and Fig. 2 (b).Control panel 22 can be selected various tool (for example Bi color size) or skip.Option row 21 can be selected different mode of operations, for example leave unused, suspend, play, record etc., and blank 23 can be drawn various figures in the above, for example write one section word or draw figure or the like.232 of webpage+transparent windows can import a webpage, transparent window then can directly add note or some comments at the relative position of webpage for the people, can point out problem points with input pen at literal on the webpage or figure statement into question, and cooperate recording of sound, make AG group item.Whether in addition, 24 of status Bars can be pointed out the various states that system is present, and for example whether microphone connects, or are not recording, perhaps suspend or the like.

The definition of one webpage is any archives that can use browser (browser) to show, can be literal shelves, image shelves, as: jpeg (.jpg), or a hypertext (html document).Blank (whiteboard) is actual to be a big or small arbitrarily blank webpage, so in ensuing explanation, it can be considered a special webpage.

That system operates in usually is idle, record and play mode between.When recording, the author can select a webpage to add the explanation of sound and figure.We claim this to add the webpage of explaining is a sound figure (AG) group item.The present invention can allow the author record his/her sound with the time, and can directly draw any figure with input pen simultaneously on webpage, in other words, the sound of explanation and the figure that is painted on the blank can be recorded simultaneously.Unlike a lot of document softwares, such as: the Words of Microsoft is based on the copy editor, the sound that present invention focuses on to record simultaneously the drawing (utilizing ruling pen) on the blank and illustrate.And when recording mode, the user still can insert the writings and image element to webpage, carries out the editor of writings and image.

Native system uses a transparent form on webpage.The user is plotted in (shown in Fig. 2 (b)) on transparent (transparent) form and the webpage.Because be transparent, so drawing can directly appear on the webpage time.Though when recording, insert the technical feasibility of figure to webpage,, this method is at usefulness (performance), unsatisfactory on hide (Latency) and the quality (appearance aftermentioned).The user also can set up figure to transparent window via inserting image and textBox except that setting up the figure via drawing.After the user leaves recording mode, (that is, transparent form is removed and system enters idle mode), can require system by using script language (JavaScript), dynamic hyperlink (DHTML) and concatenation type draft (Cascade Style Sheet, CSS) skill is inserted figure, image and literal veritably to webpage.Like this, figure will become the some of webpage.If the user wants printing curve and webpage on the identical paper, it is necessary doing like this.

As shown in Figure 4, except recorded voice, drawing, writings and image, system also records some dynamically instant incidents.These incidents comprise the scrolling webpage, eject (popping up) and (pushingdown) sketch plate 233 (sketch whiteboard) (seeing also Fig. 3) form of leaving behind and draw, and perhaps change pattern, size and color of ruling pen or the like.These functions can be set by the control of the spool on the control panel 41, ejection control 42 or pen type control 43.

The segments of sounds of recording becomes the sound time-frame of fixed cycle.The sound time-frame is through overcompression, can be undertaken by the compress speech algorithm that system provides.The user can and play the choice of talking about between the quality (tradeoff) according to compression speed, selects the compress speech algorithm.As user during at Web page rendering, system stores into a series of " stroke (strokes) " incident with user's drawing.One stroke (stroke) is the sequence of an X-Y scheme point, and it is produced by following the trail of ruling pen, mouse or any input equipment (DEVICE), is depressed into relieving formed data of this section period (for example drawing a line with mouse) from device and promptly is.System is cut into a lot of little parts with stroke (stroke) incident, is called " stroke (stroke) data frame (frame) ", with easily with voice data frame (frame) synchronously.When the user inserted literal or image, system only set up writings and image data frame (Frame), with as literal and image file.Each data frame discerns whether the data frame type of seeing is sound, stroke (stroke) or other incidents or the like by flag (realizing with a bit group) guiding.

Because the Fixed Time Interval that each sound time-frame (being data frame) occupies is determined by employed compression algorithm, for example, in ITUG.723 30 milliseconds, system's determination data flows out the existing time, can with the preceding sound time-frame of this time point of counting be simply.System is with sound, stroke (stroke), the writings and image time-frame is in the mode of time-multiplex, converge into a single data stream, as an AGS (Audio-Graph Stream), as Fig. 5 (a), wherein having comprised each time-frame has a flag F, then has sound time-frame AF, stroke time-frame SF, image time-frame IF or literal time-frame TF or the like in the AGS.

Except AGS, system also sets up a segmentation (separate) simultaneously, and short usually data stream is called advance data stream, shown in Fig. 6 (a).The information that advance data stream comprises is information (IM) or event information (EM).One information is formed to several fields (field) by one.The attribute that the user can add (properties) has title (title), summary (summary), author's (author) name and the network address that is linked to AG group item.Some attribute can be used and search this AG group item as the key word of Search engine.IM is the attribute of narration AG group item, and an EM is in order to record the variation of demonstration (form) when recording.An EM must have a time field, when will change and can take place in order to indication.For example, EM perhaps be used to refer to incident that when the scrolling webpage takes place and why produce new scrolling displacement (offset).Perhaps, EM can be used to indication, when image component be inserted into and the position of image and size why or the like.

When the user stored the data stream of AG group item, advance data stream and data stream were to be stored in an OLE compound file.The webpage of being explained only is linked in AG group item usually, that is has only the URL address of webpage to be stored in advance data stream (header stream) the inside, but not webpage itself.We claim the AG of this moment to organize item for " link AG group item ", shown in Fig. 7 (a).Yet the user can select storage or embedded web page to the AG compound file.We claim that AG group item at this moment is " an embedded AG group item ", shown in Fig. 7 (b).When the user wants to set up oneself CD-ROM (oros) course, or when wanting can off-line to play (not connecting the internet) AG group, it is necessary doing like this.In some cases, system will be automatically embedded web page to the AG compound file.Have a kind of situation to be, when the user wants to send the Email (Email) of AG data stream, the webpage of note then must be placed on the local device, not on the website.

The user organizes into groups relevant AG group Xiang Chengyi group, is called the AG course.The AG course is similar to the transparency of a sequence and comprises AG group item.These AG group item wherein one be called root AG group, remaining is called sub-AG group, shown in Fig. 8 (a).The AG course can be one " C-line (Compound LINKED) " or " C-embedded (Compound Embedded) ", how to store on the AG course and decides.If the compound file of whole sub-AG group items is stored in the compound file of root AG group item, claims that then this AG course is that C-is embedded, as Fig. 8 (b).Otherwise, be called C-link, as Fig. 8 (c), the composition file of dotted line circle (not having embedded) sub-AG group of representing some to be linked wherein.Notice that the root of AG course or sub-AG group can be decided on its above-mentioned note webpage address by embedded or linked.It is embedded identical with AG group item causing an embedded reason of AG course C-, that is, can post (emailing) AG course, and provide off-line to play the possibility of AG course.

System is to use tree structure to represent AG group item and course.Each AG organizes the relative node of Xiang Youyi on tree structure, as Fig. 9.Tree structure (tree) is stratum's form.For an AG course, the node that root AG organizes item is the root node (patent node) of all sub-AG group items, and the node that any AG organizes item also can be to be the root node that a what is called " is responded AG and organized item " simultaneously.When spectators play AG group item with the form of sound and figure, the AG group item of response was set up by system.Spectators or student can respond AG by mailing and organize a response of sending him back to the author.Respond the information that AG group item comprises some original AG groups, as the sound of spectators' response problem and drawing or the like.

Tree-shaped node has a data structure.The member of some data structure corresponds to the IM and the EM of an AG group advance data stream, and some then is used for describing the parameter when recording and playing (playback).When recording, the user can edit existing AG group item.As MP3 or mpeg data stream, the AG data stream is a time data stream, the number of implicit voice data frame (frame).Except promptly scanning at the beginning and counting the sound time-frame, flow out existing time point but there is no the alternate manner determination data, since stroke, literal or image time-frame fixing size useless (bytes of unit).This is meaning, the operation of data stream or archives (segmentation (segmentation) and amplification (appending)) but the two all consumption calculations resource (computingrecource) and time distribution (time consuming).So system uses multistage incident array in EM data structure the inside, and one group of temporary transient data stream reduces the operating frequency of data stream.

Because the AG data stream is very big usually very long, even the compression back is still the same, therefore may spend many times from website download complex archives.To download and play compound file and can carry out simultaneously in order to make, so system is about to the AG data stream and is divided into many less data stream (being called SDS), as Fig. 5 b when delivering the website.Segmentation is to finish by time (sound time-frame) and the size of restriction one SDS.The information of segmentation number is stored in advance data stream (header stream).The name of SDS is also defined by system simultaneously, such as: data_0001, data_0002 or the like.When AG group item is delivered, system will upload advance data stream and SDSs to user's appointed positions (data folder (folder)).

System provides an ActiveX control (or insertion (plug-in)) to be placed on the webpage of delivering, whenever spectators or student by touch the operating key that (click) take system out of (its be can the webpage hyperlink form be) or simple related attachment name, system can obtain the position of preposition shelves and the SDS data stream that is passed by ActiveX control, and the system that takes out of begins to download short advance data stream commonly used.In case advance data stream is received, SDS and the position of explaining the URL of webpage are promptly known by system, if any.It begins to download a SDS and webpage.In case webpage and first SDS are downloaded, system begins to play first SDS, and simultaneity factor continues to download ensuing SDS.Before broadcast (playback), spectators or student need only wait the download of explaining webpage (annotated web pAGe) and first SDS.If the size of SDS is limited in about 200Kbytes, with 56k modem speed, about 15 to 20 seconds of the time of waiting, though all the time of AG group be several hours too.Thus, the segmentation of AG data stream has reduced the buffer delay of playing AG group item.Use other benefits of segmentation still to have: at first, dedicated servo device sign indicating number need be installed in the website can play (playback) data stream.Secondly, the website is a real platform independently, that is, might be the network server on a form NT IIS or any Unix basis.

System provides the method for interactive and dynamic editing.For example, when playing (playback), no matter broadcast group item is local (E-MAIL) or (website) of far-end, the user can respond broadcast group item with the form of sound-figure, as above-mentioned response AG group item.If broadcast group item is local, the user can dynamically switch to and record, edits or even the data stream of cutout.

System installs

Hsrdware requirements

Hsrdware requirements of the present invention as shown in figure 10.Hardware system comprises a CPU (central processing unit) 100, one internal storages 101, one storage facilitiess, has microphone as a hard disk 102, and imports adlib 103, one keyboards, 104, one mouses 105 or an input pen of exporting with loudspeaker, as a plate (pen tablet).If the user wants to play AG data stream of the present invention from the website, also needs a networking card 106 or a modem.

Be noted that, use input pen to be easier to draw than mouse.Reason is that input pen (Pen InputDevice) has higher sampling rate (sampling rate) than mouse, and therefore, it produces more smooth-going and few zigzag lines.Furtherly, a lot of input pens are pressure sensitive types.Pressure can be used to determine the thickness of stroke among the present invention (stroke).

Above-mentioned hardware system is the broadcast that is used for works (authoring), editor and AG data stream.In the present invention, we claim its user terminal system.

If the AG data stream is placed on the website and downloads and play for other people, then needs a server system.Servomechanism can be a computer and the operating system of HTTP and FTP function is provided, and wherein, HTTP is used for data stream and downloads, and FTP is used to data stream and uploads.Need special data stream medium function to be installed on the servomechanism unlike a lot of other data flow techniques, and only operate in special operating system, the present invention does not need special service (service) to be installed on the servomechanism, and platform is for independently.The user can get at a web host company usually and obtain this service (service).Servomechanism and unnecessary is installed at home.

Below discuss based on user terminal system.

Software requirement

The present invention is software systems, and its operating system need possess basic figure, sound and some necessary resources.Because the present invention need download and upload data stream from the website, operating system need provide the function of HTTP user side, FTP user's end and Email (SMTP and POP) user side.The present invention also needs to embed simultaneously web page browsing control (at this moment, operating system needs the web browser of dynamic html (DHTML) ability), and perhaps the present invention need provide the Web browser of oneself to show as the webpage of explaining.

For example, if we use Microsoft's Window to be web browser as operating system and explorer (Explorer), the present invention can realize with " carrier (container) " application mode under ActiveX or OLE grammer.Embodiments of the invention are a container (container), can control the ActiveX control of Explorer web browser.

The present invention is mainly a software application.Following system means the software element of above-mentioned user terminal system.

System introduction

Operator scheme: leave unused (idle), record and play

At first, system operation modes has three: leave unused (idle), record and play (playback).The transformation of three-mode as shown in figure 11.System is divided into three subsystems.Each subsystem corresponds to each pattern, and has its oneself pattern to use interface (GUI).

System normally begins with idle mode, unless can take (invoke) out of by the ActiveX control of delivering on the webpage.The back is play data stream on the line with illustrative system as play mode how.

When the user presses recording key, system can enter recording mode from idle mode, from idle mode enter play mode also with.System can directly enter recording mode from play mode.This will betide, when the user will edit or insert new data and flows to existing data stream, or when setting up a new data stream when playing and responding original data stream, as shown in figure 11.

AG organizes item, sound figure (data) data stream (AGS), advance data stream

The project of explaining is called AG group item, because data owner will be made of sound and drawing, (certainly, also can comprise writings and image).An AG group item, system will set up two data stream at least.Usually very big data stream claims sound figure (data) data stream (AGS), and it comprises sound, stroke, writings and image time-frame, shown in Fig. 5 (a).Each time-frame has a flag, is used for discerning its type.Another data stream claims advance data stream, is used for the data of description stream and the attribute of incident when recording, as shown in Figure 6.

The AG archives, course, tree (tree) and data structure

Two data stream and note webpage (perhaps) are saved as a composition file, are called the AG shelves of AG group item, as Fig. 7 (a) and (b).The user can be with a group AG group AG course of composition.In the AG course, Xiang Youyi root AG group of AG group item, remaining is called sub-AG group item, shown in Fig. 8 (a).The AG archives of root AG group item can comprise the AG archives of following its sub-AG group item, shown in Fig. 8 (b), (c).

System uses a tree structure to represent AG group item.The node of tree corresponds to an AG group item.Each node has a data structure, and wherein, the state of some member's representatives comprises and records and play (playback), and other represent the AG group item of attribute and incident.

The term definition

AG organizes item: target web or note blank (whiteboard).

The AG group item that embeds: the note webpage of AG group item and other data stream (preposition shelves and data) are stored in the compound file the inside.

The AG group item of link: the note webpage of AG group item is not stored in the compound file.Yet URL or the address of explaining webpage are stored in advance data stream the inside.

The formation of AG archives: an AG group compound file comprises: data stream (AGS), advance data stream (headerstream), with possible webpage (if its group is embedded person) be accompanied by a sub-AG group possible compound file, if the words of root AG group item that should group Xiang Weiyi AG course.

AGS sound diagram data stream: a multiplex data stream has sound, stroke (stroke), writings and image data frame (frame).

AG advance data stream (header stream): the data stream of an AG group item has comprised the attribute information of AG group item, and the event information of incident takes place when recording.

Information (IM): one of two kinds of advance data stream types, comprise the attribute that AG organizes item, as: the title of AG group item and author or the like.

Event information (EM): in the advance data stream, one of two kinds of information types, the incident that comprises has: the scrolling of webpage when recording.

The collection of AG course: AG group item may be to play (playback), dispensing (distributed), and mailing (emailed) is as same single unit (unit).Xiang Youyi root AG group of these AG groups item, remaining is called sub-AG group item.

Root AG organizes item: the special AG group of AG course item.About the information of AG course, such as: the order of its composition and whole AG group items then is stored in the AG archives that root AG organizes item.

Sub-AG organizes item: the AG group item of AG course, but not root AG group item.That is it does not contain the information about the AG course, and as same normal AG group item.

C-embeds the AG course: an AG course, the compound file of its root AG group item comprise the compound file of the whole AG group item of its sub-AG course of accompanying.

C-link AG course: an AG course, non-C-embeds the AG course.

Response AG group item: when the user will respond an in progress AG group item to the author, it can set up a response AG group item from the original figure that original data stream obtains, and some attributes of response AG group item are the AG group items that hold from original.The node of responding AG group item is a child group item from original AG group item.

Segmentation AG data stream (SDS): for the playout-delay that can make high amount of traffic in the website still less, in fact AGS is divided into a lot of less AGS.These less AGS are called SDS.

The composition of AG shelves

The AG archives of each AG group Xiang Youyi correspondence; It may be temporary shelves, if the user does not store the words of these archives clearly.AG archives are compound archives, and grammer is OLE.One compound archives can be considered " shelves middle-grade (file in file) " system.The archives that OLE grammer (terminology) definable is stored in compound file are a data stream, and the data folder that is stored in compound file is called storage (storage), as shown in Figure 7.Notice that the data stream in the compound file may be a compound file itself.

The composition of AG archives is to decide according to the type of AG group item.At first, the situation with AG group Xiang Feigen AG group item has two kinds of possibilities.If the AG archives only comprise data stream (AGS) and advance data stream (headerstream), it is called " link ", and AG organizes item, as Fig. 7 (a).Another kind may be, if the AG archives not only comprise this two data stream, also has webpage and links possible webpage element, then is called " embedded " AG and organizes item, shown in Fig. 7 (b).

Secondly, if AG group item is the root AG group item of an AG course.It can be embedded person, maybe can link person, as organizing item by webpage AG that the position determines with other, in addition, an AG group item can further be divided into: the compound file of sub-AG group item also is stored in the compound file the inside of root AG group item simultaneously if all are accompanied, then this root AG group item is called one " C-is embedded " root AG group item, as Fig. 8 (b).Otherwise it is called one " C-link ", and root AG organizes item, shown in Fig. 8 (b).

The composition of AG data stream (AGS)

The data that produce by recording are called AG data stream (AGS), as shown in Figure 5.It is a multimedia data stream, comprises different types of data time-frame (frame).The data time-frame be synchronized mixes together.There is a time-frame flag in the place that each time-frame of AGS begins, is used for identifying the kind of time-frame.The present invention has defined the data time-frame of four kinds of fundamental types: sound, stroke (stroke), writings and image time-frame (frame).Yet, aspect the expansion in future, also can add the data data frame (frame) of newtype, such as: hyperlink and film etc.

The compressed mistake of each sound time-frame (frame), regular time length is all arranged, decide on the sound compression algorithm.But may there be regular length yet.So AGS is actual to be a time data stream.The time of any position of data stream can be decided by the number of sound time-frame before this position of counting.

The composition field of time-frame as shown in Figure 5.The time-frame grammer is a by type and deciding.First field of any time-frame is the flag of identification.

Time-frame is alternating expression (interleaved).The counting of playing subsystem use sound time-frame comes timing (timing) voice decoder and shows draw (drawing), writings and image.For example, stroke (stroke) time-frame between n and the n+1 sound time-frame will be displayed on nD and (n+1) time of D, and wherein D is the length of sound time-frame.

The time-frame grammer

The sound time-frame

Flag (flag)

The segments of sounds (may be different sizes) of compression

When recording, system from adlib receive original sound time-frame (for the sampling, the numeral, and incompressible).During original sound time-frame has regular length and fixes (duration).Recording subsystem uses the sound compression algorithm that each original sound time-frame is compressed into (compression) sound time-frame.For example, if system uses the ITUG.723 CELP-BASED standard sound press of uttering a word to contract, original sound time-frame has the length of 480 bit groups, during be 30 milliseconds (milliseconds).Sampling ratio per second 8k, and each sample is 16 bit PCM forms.Each original sound time-frame is compressed into 4bytes, and 20bytes or 24bytes correspond to quiet, the high compression pattern of low compression ratio when G.723 respectively.Therefore, the ratio of high compression ratio is 24: 1 (5.3kbits/second), and low compression ratio is 20: 1 (6.3kbits/second).G.723, noting, is one " loss (lossy) is arranged " compression algorithm (algorithm).This means, sound is distortion to some extent, and can not fully be recovered by demoder (decoder).

(algorithm) has some considerations about the compression algorithm that selects a sound.Most high compression ratio compression algorithm needs many computing powers and storer (memory).Therefore, need dynamical computer to compress sound immediately.In some cases, the preceding pre-filtering of compression needs, as: noise eliminating.This will increase the load of instant computing.Therefore, for computer faster, compression and pre-filtering can realize in instant mode.The original sound time-frame is that the time-frame with other types compresses and the time multichannel, and stroke (stroke) time-frame mainly directly points to AGS.

On the other hand, for computer at a slow speed, system can not use instant compression.System can store the temporary transient data stream A of original time-frame to, and sound flag (having only) is become other temporary transient data stream B with the time-frame time multichannel of other types.Compression and mixing A and B are AGS, can finish after a while.We will talk about, and in the part of recording subsystem (Record Sub system), when compression and multichannel Ying Yu finish.

Be noted that, the time frame grid type may change, if use the words of the compression algorithm of other types.The information of using the sound compression algorithm is to be stored in advance data stream.

Stroke (stroke) time-frame

Flag _ S or flag _ R

(number of points) counts

The type of pen

The color of pen

The size of pen

(x, y) ... or (x, y, p)

System allows the user directly on the blank or covering on the transparent window that webpage makes graphical annotation and draw.Unlike some " diagram audio data stream ", only allow the simple geometric figure and drawing is saved as meta data (instruction of drawing), drawing of the present invention is recorded as stroke (strokes).Stroke (stroke) is defined as a series of two-dimensional points (points), and it comprises the path (for example arbitrarily drawing lines with input media) that is pressed and is crossed by institute's stroke till decontroling from input media.

Two-dimensional points (x y) is level and vertical coordinate point with respect to webpage, but not absolute (form) point that shows.For example, with a window dimension 800*600, webpage may surpass this size (scroll bar is arranged), and for example the x value is greater than 800, or the y value is greater than 600.

For the better sound synchronism is arranged, system is divided into a lot of less strokes (stroke) time-frame with a stroke (stroke), before the morbid sound time-frame, as Figure 10 two (a).For example, be 30 seconds if the user draws a stroke (stroke), if not segmentation, 30 seconds stroke (stroke) will be decoded and shown (for example, 30 milliseconds of ms G.723) by single sound time-frame.The practice is preferably, stroke (stroke) average mark is dispersed in 30 seconds sound time-frame (for example, 30 seconds about 1000 time-frames (frame)).The other reason of segmented stroke (stroke) is to reduce the buffer size that stores single stroke.

Certainly, also can limit the size of stroke (stroke) time-frame.Less stroke (stroke) time-frame can have sound synchronism preferably, and only needs less buffer zone.But it also needs more computing power and will increase complexity simultaneously.

Except (one stroke (stroke) time-frame also comprises the attribute of input media simultaneously for x, y) point, and its attribute comprises the type (type) of pen, the color and the pen size (size) of pen.The type of pen might be solid (opaque fully), and mark (transparent fully, as to make the bit mode in background " AND " computing), ALPHA-transparent (for example, transparency from 1 to 255).Can use-1 expression solid, 0 expressive notation, and 1 to 255 realized for ALPHA-is transparent.The color of pen is the form with RGB.Wherein, R is red, and G is blue for green B.Each is an integer.The size of pen i.e. the thickness of pen.

Above-mentioned attribute is applied to the point of all strokes (stroke) time-frame.When input media detects pressure, the value of pressure be added to (x, y), become (x, y, p).The size attribute of the collaborative pen of pressure p can be used to control reference point (x, thickness y).Use point mode, adjustable thickness can improve the quality that stroke (stroke) is represented, shown in Figure 12 (b).

In stroke (stroke) time-frame, system uses two flags, and one is flag _ S, is used to refer to the beginning time-frame of stroke, and another is flag _ R, is used for remaining time-frame (frame).For the stroke performance of playing in the subsystem, such differentiation is important.When system decodes and demonstration strokes, system connects counting of stroke (stroke) time-frame, insert function via line or high power to be, such as: Beizier curve (curve).Simultaneously also on the bar line, do antialiasing (anti-alias) filtering, use the minimizing sawtooth effect.Two continuous stroke (stroke) time-frames that belong to identical stroke (stroke) will be connected end to end.Note,, on the same group (flag _ S, flag _ R) then being arranged not for the graphic plotting on the sketch plate as the sketch plate because system allows the user at the Web page rendering figure.

Also can come the compression stroke point via the approximation that detects consecutive point among the stroke.Because consecutive point are very near usually in the space, can represent a stroke (stroke) for (x_0, y_0), (x_1-x_0, y_1-y_0), (x_2-x_1, y_2-y_1) Replace (x_0, y_0), (x_1, y_1), (x_2, y_2) ..., the benefit of so doing is, we can use the less bit different vector of encoding, and utilize the still less coding decoder of (loss-less) (codec) of loss, such as: Huffman codec.Because the very near probability of x_k-x_k-1 is very high, we can design a code book and represent error amount.For example, code book seems to look like error codes:

00

110

211

3100

4101

5…

For the less error of normal generation, system can use less bit.Yet coding decoder is determined to increase the complexity of calculating.

The literal time-frame

Flag

Element ID

The flag of inserting or deleting

Limit frame (bounding rectangle)

Font ID

The literal size

Literal ... (may be blank, if deletion, or the preceding of element exists)

When recording, the user can place literal at blank or dynamically cover the transparent window of webpage.As user during in transparent window definition one zone, he can enter textbox, selects font and size.Perhaps, the user can place existing textbox on form, and can move everywhere.Textbox has a square boundary to correspond to webpage.Though do not put webpage when recording into, the user can be by using DHTML, and CSS and zindexing require to insert literal veritably to webpage.

The parameter of literal time-frame quite clear (self-explained).Textbox or element have an id (identification code).The incident that injects or delete the literal element is saved as the event information of an advance data stream (header stream), contains time, element i d, the flag of inserting or deleting, limit frame etc.It appears to, and information is replicated to data and advance data stream (header stream).In the advance data stream, the information of covering system is important, can be in order to improve the usefulness of the broadcast subsystem that will touch upon afterwards.

If time-frame is the existing literal element of deletion, then the bit group number of literal is zero.Perhaps, if promptly exist before the literal element, then need not to duplicate this literal.

The image time-frame

Flag

Element ID

Flag is inserted or deletion

Limit frame (Bounding Rectangle)

(for example, 0 is jpeg to the type of image, and 1 is gif, and 2 is png, and 3 is tiff ...)

The bit number of image shelves

The image shelves

The user also can insert or delete an image component at blank or dynamically cover transparent window on the webpage.Image file can be the image file of any compression, arbitrary few loss (loss-less) form, and as gif or png, or the person of diminishing (lossy) form, as jpeg.Explanation about the literal element also is applicable to image component.

Literal time-frame and the generally not segmentation of image time-frame when object is received in broadcast, can manifest immediately.

The expansion that other are possible

Except the data frame (frame) of above-mentioned four types, still may be via specifying different flags to add the time-frame of other types to data stream.For example, can add one " suspending time-frame (pause frame) ", tell that playing subsystem suspends and will how long continue, perhaps can add the address that one " hyperlink time-frame " specified hyperlink URL, the student then can access when playing (playback).Also can add " image time-frame ",, set up real film demand (VIDEO-ON-DEMAND) system with the time-frame synchronized mixes of sound and other types.

The data stream of segmentation (SDS)

Because data stream AGS comprises sound and graph data frame (frame), data stream (archives) size is very big usually.For data stream big on the website, could play (playback) after must waiting the download of entire stream, particularly therefore internet line at a slow speed always makes us being difficult to accepting.Preferably, can download and play data stream is carried out simultaneously, perhaps reduce as far as possible to postpone, but unfortunate, the function that most operating system is downloaded archives is closing function (blocking function).With the Microsoft's Window is example, can use API ' the s ISAPI of FTP-BASED or the API of HTTP-BASED, such as: " Imoniker " or " URLfileDownload " downloads archives from any website, yet these are to be downloaded action up to archives just can finish for closing function.Note that use HTTP-based downloads not worse than FTP.Because some FTP servomechanisms use proxy server (proxy) (wall of setting fire (firewall)), general FTP user's end also is not easy to carry out the wall (firewall) of setting fire completely.Though practicable self FTP or HTTP user side avoided sealing (blocking) problem, the cost of enforcement and time do not calculate, and the user may need to upgrade, in case the words that communications protocol changes.The thing that allows operating system do these low orders compares rationally on the contrary.

So the present invention uses simple segmentation algorithm to solve this class problem.

Before data stream uploaded to the website, system can be divided into data stream AGS many less data stream, was called the data stream SDS of segmentation, shown in Fig. 5 (b).Each SDS may be limited to a fixed size, perhaps regular time length.For example, but be less than two minutes length during selectional restriction one SDS.The number of SDS and the name of SDS be store in advance data stream (header stream).Name in fact only is a prefix.The name of SDS can be " data_0001.xxx ", " data_0002.xxx " or the like (xxx can be any incident).

After advance data stream and SDS upload to a website, play subsystem and can use sealing (blocking) URL function to download these archives.What the broadcast subsystem was at first downloaded is advance data stream (header stream).From preposition data stream (header stream), play subsystem and know the number of SDS and the prefix of stream name.Begin to download in regular turn whole SDS then.When first SDS was downloaded, the broadcast of data stream also began to carry out.Therefore, system is actually in data download stream and also plays this data stream.Just wait the download of an advance data stream and a SDS time delay, under the situation of major part, be less than 30 seconds, even internet line at a slow speed also is like this.

Use simple segmentation algorithm still to have, that is on the net servomechanism, do not have the needs of dedicated servo device sign indicating number in more benefit.Need the dedicated servo device unlike some other data stream medium, such as: form IIS medium servomechanism, the present invention can use any net servomechanism to deliver SDSs, no matter Unix or form NT, and its cost is very cheap.

The composition of AG advance data stream

Except data stream, system also produces simultaneously other data stream and claims advance data stream when recording.When playing from the website, advance data stream is first data stream that is downloaded, therefore be called " preposition " data stream, as Fig. 6, wherein dotted line is field (Field), information is made of many fields, and each information all has a flag F foremost, and being used to refer to this information is event information (EM) or information (IM).

Advance data stream (header stream) has two kinds of information (information).One is information (IM), and another is event information (EM).It is to organize the attribute of item to AG that information closes, and is used for narration group item.Event information produces when recording.When recording, except data stream, the user more sets up incident (event).Concerning playing subsystem, be necessary to know incident, to duplicate the program of recording fully.An incident can have a lot of event arguments.To all incidents, general parameter is the time, and the counting of the sound of correctly saying so is when incident takes place.

Advance data flows each information and occupies some fields, as shown in Figure 6.First field of any information is ID (flag) field, uses the type of identifying information, and for example, a heading message (IM) has two fields.One is ID (for example 0000), and another is the word string of title; The URL information (IM) that is used to explain the URL address of webpage also has two fields, and one is ID (for example 0001), and another is a url string.In addition, scrolling incident (EM) has four fields: one is ID (for example 1000), and one is sound time-frame counting (the scrolling incident takes place preceding), and one is X-OFFSET (skew), is Y-OFFSET at last.

In fact, the information of advance data stream saves as the full member (statemembers) of data structure usually, in order to represent AG group item; Have only as the user to store (saving), E-MAIL, or when uploading AG group, system just can produce advance data stream.In like manner, receive in case advance data stream is played subsystem, it is with decoded and save as the full member of data structure.

In the embodiments of the invention, advance data stream is a long word string just.Field with some special symbols separately.

It below is the detailed description of each information.

Information

Information is the attribute of AG group item or course.Classify title and the form of some IM down as.

Version (version of data stream format): flag, version number.

AG organizes the type of item: flag, type (normally AG organizes item, and root AG organizes item, and sub-AG organizes item, and response AG organizes item, and error (delta) AG organizes item or the like)

Title: flag, title

Author: flag, name

Date: flag, date

Explain (Comment): flag, explain

Acoustic coding demoder (codec): flag, coding decoder ID

The URL of webpage: flag, URL (if blank is blank (whiteboard) and embedded)

Embedded or the link: flag, type

Webpage font ID: flag, the ID of font

Webpage literal size: flag, literal size

The target screen size: flag, size (for example, 1024*768)

Whole time (sound counting): flag, counting

The number of segmentation: flag, number

The sub-AG that accompanys organizes the number of item: flag, number (having only root AG group item)

The title of the first sub-AG group item: flag, title (having only root AG group item)

The title of the second sub-AG group item: flag, title (..)

·..

·..

Most IM is quite clear.Some then need to be explained.

Version number is used in data stream format.If the change of format of data stream, system can give a new version number, to distinguish from old form.Error (delta) AG group item is similar to responds AG group, but its intention is inserted an existing AG group (for example, the part of variation is stored into error AG group, the repetition when avoiding editing).

The target screen size is important to the webpage that proposes.Web page contents can be via the browser reformatting, when the carrying form changes size (resized).Therefore, if the form size is different when recording and playing (playback), webpage is explained can't match correct position.Therefore, system stores screen size information flows in advance data, and plays the demonstration that subsystem can suitably be adjusted form with this.Screen size also influences the scrolling of webpage simultaneously.The preferably, screen size remains on 800*600 approximately, because the size of most display monitor central monitoring system is all on this size.

If organize item is an AG group item, and the information such as title of number, order and its sub-AG group item of accompanying are still arranged.Specify the reason of these information to be, allow and play the content that subsystem shows the AG course, even before sub-AG group item was downloaded, thus, the user can jump to special sub-AG group and begin to play (play sub-AG group that subsystem obtains switching to select data stream).

About the advance data stream format three specific characters are arranged.The first, the order of information is not that youngest is important, because each information has flag to discern.The second, via the different flags of definition, system can easily add more information.The 3rd, having information is same type, thereby identical flag is arranged.For example, in the root AG group item, the heading message of the sub-AG group of companion item.

Event information

The basic difference of event information (EM) and information (IM) is that event information must have a time parameter.The IM pass is the attribute to the group item, and the EM pass is to the change that shows (form).Classify the EM of some down as.

Scrolling: flag, time (sound counting), time-frame id, offset-x, offset-y

The sketch plate is opened and closed: flag, opens or closes at the time

Image event: flag, the time, image I D, id, image type, image opens or closes, limit frame (boundingrectangle)

The literal incident: flag, the time, literal ID, id, font size, the literal size, literal opens or closes, the limit frame

·...

As previously mentioned, the information of some incidents is to be replicated in the data stream.Problem is, for assorted youngest need be put these to event information or why not put all event informations in data stream.

The reasons are as follows.The webpage size is often greater than display window (most display monitor central monitoring system resolution is less than 1024*768).Therefore, browser provides spool to find the position of webpage.Among the present invention, the user can draw everywhere at webpage, and we have the incident of scrolling.Suppose that the user will play in the middle of data stream, time T-MID replaces from the beginning and plays.If play subsystem do not know scrolling before scan data stream skew (offset) at T-MID, then stroke (stroke) time-frame that whether need go to be plotted on the form and run into can't be learnt by system.Remember that the two-dimensional points that stroke (stroke) time-frame comprises is the absolute coordinates on the webpage.On the other hand, if scrolling skew is known at time T-MID by system, it can be made and draw stroke (stroke) time-frame or not via the decision of checking, if the words that the limit frame of stroke (stroke) time-frame and skew form intersect.

Though can find scroll information at T-MID by mat scan data stream, if scroll information is stored in the words in the data stream.This method and imperfection because must take time, when especially data stream is big (user will can not put up with), and furthermore, if data stream is not downloaded as yet, have just been had no way.Therefore, via using some storage areas (event information of advance data stream), our can be greatly improved usefulness of system.Similarly situation also is useful in image and literal time-frame.

Some incident informational needs are illustrated.Please remember that a webpage can comprise the frame page of a lot of scrollings (form in the webpage).When recording, the user can the whole webpage of scrolling or any subframe page or leaf.Therefore, system needs a frame page ID to discern which frame page and needs scrolling.Frame page ID is that whole form elements order of (comprising webpage itself) is replied from webpage by system, as long as record the subsystem rule identical with playing subsystem use diplopia window element back and forth.With DHTML is example, can use such as: the form object is enumerated at the interface of " IEnumElementCollecton " and " IEnumElement " and other functions.

Please remember that in identical time T 1, the user can insert image and literal element, and after time T 2 delete them.System specifies each image and ID of literal element (image I D or literal ID), use with data stream in ID coupling.Yet, if possible, on webpage, have simultaneously to surpass an identical image or literal element, therefore, second ID of event information promptly is used on this purposes.

Event information does not need to sort according to the time, and it allows the event information more than two that identical time parameter is arranged.

The AG data structure with make the result in fact

System uses a tree structure, as shown in Figure 9, represents AG group item and course.The node of tree corresponds to an AG group item.Tree is hierarchy.One node can have child node or father node (parent), decides on node type.For example, organize item if node is an AG, then the child node of this node is to belong to the sub-AG of accompanying of identical AG course to organize item, and the response group item of AG group item is to link this AG to organize child node of item or the like.

When AG course of system plays, it will play the root AG group item of course at first, then according to the order of node in tree, next be only sub-AG group item, restart unless the user jumps to arbitrary node of appointment.

Each node has a data structure, is called the AG data structure.Some members of data structure are the IM that are replicated in advance data stream (header stream).But, be not that each IM has a corresponding member in data structure.For example, for the AG course, the number of the sub-AG group item of accompanying may be decided by tree itself.When recording and playing (playback), some member is state (states), such as: whether organize item and be changed.

The most interesting member of AG data structure is to represent that of advance data stream (header stream) event information (EM).These members are called the incident member of data structure.Reduce the time of data stream (archives) operation when recording, incident member and use the important factor that is designed to of temporary transient data stream (archives) below will further specify.

Incident member and reduce the mechanism of data flow operations when recording

The incident that produces when recording is that (t, an array (EA) void*) represent that wherein t is the time (sound counting) that incident occurs, and void* represents to point to the index of arbitrary structures with the incident member.Structure is to decide (Event-dependent) with incident.For example, if incident is a scrolling incident, its structure is

Flag (for scrolling)

Time-frame ID

The X-skew

The Y-skew

That just is defined in advance data stream (header stream) scroll information.Other example is a sketch plate incident, and its structure is

Flag (for the sketch plate)

The flag of appearing and subsiding

In fact, when recording, system uses an array of incident array (EAA) to represent incident.The number of EAs is greater than three at least among the EAA.Reason is as follows:

System allows the user to edit existing AG data stream.The editor of an AGS means and inserts a new AGS to existing AGS, or from the existing a part of AGS of AGS deletion.Because the AGS data stream is relevant with the time, the editor of Ags archives irrelevant not as editor one and time are direct, with irrelevant archives such as literal archives or Word shelves of time.AGS edits other difficulty and in fact is, frame length is not fixing during AGS.Because the user can change recording time, insert one section new sound and graphical annotation (annotation), it often forces editing machine to go to scan the position that each bit obtains special insertion point.Then, existing AGS is cut into two sections, and the new AGS that increases arrives first, then, a remaining part to the second that increases.From time and calculated amount, scanning, section (slicing) and benefit increase very approaching, particularly when data stream is very long.It is simultaneously also quite general, and the user can first selection insert new AGS fragment, but as finding new AGS fragment (piece) and non-essential, also can restore (undo).Therefore, inserting new AGS fragment at once is not bright sage.For the complicacy that adds, its other problem is that each edits the period (session), and the user can produce various incidents.

For the time that reduces data stream operates, and improve usefulness, system uses the array of a lot of temporary transient data stream and incident array to solve problem.

Conceive as follows.Suppose that the user will edit an existing AGS.At the beginning, there are data stream AGS-O (having compression sound data frame (frame)) and incident array EA-O in system.If the user begins recorded voice and graphical annotations, insert (in the middle of AGS-O) or newly-increased (from the AGS-O tail end), system always sets up one group of new temporary transient data stream AGS-T1 (the sound time-frame can be compression or do not compress, and whether viewing system is instant compression) and incident array EA-T1.If the user stops to record, this is to (AGS-T1 is not EA-T1) with (AGS-O EA-O) merges at once.If the user restores nearest recording, system only abandon (AGS-T1, EA-T1).Suppose the user do not delete (AGS-T1, if EA-T1) and the user select to continue to record to increase newly, system only set up another group (AGS-T2, EA-T2).Then, when the user continues to record, two kinds of situations can take place then.

The first, if that the next one is recorded as is newly-increased, then (AGS-T2, EA-T2) newly-increased to (AGS-T1, EA-T1).In order to record, system set up one group new (AGS-T2, EA-T2).Notice that newly-increased data stream (archives) is simply and not consuming time.Increasing EA-T2 newly to EA-T1, in any case, should be considered.Because some incidents can be cancelled each other.For example, if there is a sketch plate (sketch board) t1 of incident to occur at EA-T1, reach the t2 of a sketch plate disappearance incident, and the difference of t1 and t2 is negligible (it is not without possibility), will be canceled mutually when then this two incident merges at EA-T2.Other example if the t1 of the incident of a scrolling at EA-T1 arranged, reaches the t2 of the incident of a scrolling at EA-T2,, and the difference of t1 and t2 is insignificant, reasonably the practice is, should eliminate in the scrolling incident of t1, only stays the scrolling incident at t2.

The second, if next is recorded as insertion, then (AGS-T1, EA-T1) will be inserted into (AGS-O, EA-O).Notice that if the sound time-frame of AGS-T1 not compression as yet (situation of back compression), before insertion AGS-O, system will compress and mix other time-frames.System sets up then that new (AGS-T1 EA-T1) records.

Therefore, we have three data stream (two are temporary transient (temp)) and three incident arrays, as shown in figure 13.

It is lower usually than the probability that newly-increased one new fragment arrives existing fragment to note that insertion one new fragment arrives existing fragment.What can infer is, most time, system just newly-increased temporary (temp) shelves have only the minority time to insert temporary (temp) shelves to existing person to existing person, thereby, can save time and computed losses, and improve the usefulness of system.

Certainly, record, we can continue to set up temporary (temp) data stream and incident array at every turn.But produce many temporary (temp) data stream and will become too complicated with the incident array, and in fact keep in (temp) data stream and incident array need with old combining.Complicacy and usefulness are a choice problem (tradeoff) concerning system.

If want the Version Control of implementation data stream, for example, revise (error) and original data stream and separately be stored.Then, this mechanism is also also by (AGS-O, EA-O) initial value is that blank is used.And it may need to define some new members in advance data stream and data structure arrive indication how to revise (insert, newly-increased, delete, or the like)

Above for recording.When the user will test or plays the data stream of recording, on the practice be by system copies (AGS-O, EA-O) to (AGS-O-C, EA-O-C) and merge (AGS-T1, EA-T1) and (AGS-T2, EA-T2) in (AGS-O-C, EA-O-C).Like this, we have temporary (temp) data stream (AGS-O-C EA-O-C) need not change original data stream as playing.

To the deletion of part data stream, the user can specify two time marks (MARKS) to delete therebetween data stream.Its practice with as playing (playback), at first set up one temporary (temp) (AGS-O-C, EA-O-C), then, the AGS-O-C of cutout and EA-O-C partly.Deletion EA-O-C needs consider more, is similar to merging.The benefit of this method is that the user can restore deletion, and therefore, data stream originally still can not be affected.If the user determines deletion, then, temporary (temp) (AGS-O-C, EA-O-C) can substitute original, and removing (AGS-T1, EA-T1) and (AGS-T2, EA-T2).

(broadcast during data recording stream) wherein mode is recorded and tested to the mechanism that this paragraph proposes just improvement.Remove this, still have a lot of additive methods also can finish, regard method as and decide.

Temporary (temp) simple stroke (stroke) archives, temporary (temp) image file and literal archives and the role that they improve in usefulness

When recording or playing (playback), system can produce simple stroke (stroke) archives.Stroke (stroke) archives are that the line by word string constitutes.Each word string comprises following information.

Stroke (stroke) time of origin (sound counting)

Limit frame (Boundary rectangle)

Stroke (stroke) point

Archives are to be received in when recording or playing (playback).

The reason of setting up these archives is to improve the usefulness of stroke (stroke) performance.Remember that stroke (stroke) time-frame is the time-frame mixing with other types in sound and the AGS.If the user will begin in the middle T-M time of data stream to record or play, from the incident member of data structure, system knows that skew comes scrolling blank or webpage.Were it not for simple stroke (stroke) archives, in order to represent the stroke of display window, system must flow by scan data, checks each stroke that runs into (stroke) time-frame, and calculate its boundary, look at that whether its boundary intersects at present display window.If show that then stroke (stroke) time-frame is on form.This road program is very consuming time.If there have been simple stroke (stroke) archives in system, then system just reads each stroke (stroke) line simply, checks whether it intersects at form.If be to show that stroke (stroke) is on form.Note, system still needs to scan AGS, yet it can skip stroke (stroke) time-frame, it goes without doing, and anything (is remembered, stroke (stroke) time-frame has the length of field indication stroke (stroke) time-frame, that length is just skipped to next time-frame, when it runs into a stroke (stroke) time-frame by system).

Note that have two simple stroke (stroke) archives of so temporary (temp), and one is blank or webpage, and another is sketch plate (sketch board).

For image time-frame and literal time-frame, identical action can be done for archives separately via storing them by system.

Though these archives may be spent many spaces, the usefulness that colored point space improves system is also quite worth, and especially present PC nearly all has very big storage volume.

Operator scheme

Former joints are form and the structures that produce data stream about system earlier, represent the inner structure of AG group item as tree-shaped node.

Following discussion is about operator scheme, its user interface and practice.

The operation of system is in idle, record and play mode between, as shown in figure 11.We also claim to operate in system in recording for to record subsystem, in like manner, also have and play and idle subsystem.

Idle mode and subsystem

When the user did not record or play an AG group item or course, we claimed system to be in idle mode.The figure of idle subsystem uses interface such as Figure 14.Blank or browser window on the webpage have menu, Facility Column, tree-shaped form and editor's form.

Generally, system originates in idle mode.In idle mode, useful function is:

Browsing page is explained at regional disk drive or on the website (annotate).This is to be controlled by the browser of the system of being embedded in to finish.System also provides the option that scans, such as: (forward) forward, homepage (home), be written into (load), (backward) backward stops (browsing) and tabulation in the past, as at any independently web browser, such as: the Explorer of Microsoft is the same normally to be seen.

Set up a new blank (whiteboard) and preparation and edit-record drawing, sound, the insertion of image shelves and textBox or the like.

Be written into an existing AG group item or course to tree structure.

Set up an AG course, it is to organize item and stroke and discharge other AG group items by setting up an AG, becomes the child group item of root AG group item.

Store an AG group item or course.

Adding and change the attribute-attention of an AG group item or course, is not that whole attributes can both be changed by the user.For example, the All Time of AGS promptly can't be changed by the user.System provides a properties dialog to allow the user add and changes attribute.

The figure that insertion is recorded is arbitrarily drawn, and image and textBox store and print-note to webpage, and when recording, figure in fact is to be inserted on the transparent window of a covering web browser form, rather than direct to being placed under the webpage.

Print

Store C-link AG course and embed the AG course to a C-.

Organization tree: the order that changes sub-AG group item is dragged and draws in use, deletes AG group item or the like from tree or course.

Setting up one has the AG group item of upload ability or course-user to specify a position (zone or far-end) to store SDS and advance data stream or the like.System is whole AGS of root and son group in the AGS of segmentation AG group or the AG course, and stores them to certain location.Perhaps, advance data stream can store at compression shelves (zip file) with SDSs and circulate.

AG of FTP group or course, or the zip shelves of delivering (post) configuration are to a website.

Watch the state of recording, such as: present reproduction time, recording time is with event information or the like.

Via the part of specifying two time marks (time mark may define) deletion AGS when playing subsystem.

Post an AG group item or course---system capable of automatic changing links AG group item and organizes item before E-MAIL, if webpage is the words of regional webpage to the embedded AG of one temporary (temp).In like manner, C-link AG course also together.

Editing e-mail address list (address books), or obtain from the Email User end software that is installed in system.

Enter recording mode

Enter play mode

·…。

Except these basic functions, if necessary, system also can easily add other function.Some basic functions are described as follows:

Properties dialog

System provides a properties dialog, and as Figure 15, the user can be used for adding and changing the attribute of an AG group item or course.It is quite obvious that great majority correspond to the attribute of AG group of advance data stream information.In dialog box, the user can specify whether AG organizes an embedded or link.If the URL of embedded address has in dialog box, and user's mark check dialog box is embedded in webpage, and then system changes link AG group to embedded AG group.Note, because webpage may comprise the element of a lot of links, such as: image and form or the like, the user must tell system where for comprising the data folder of whole elements.Utilize the function of stream Viewing device (browser), " the save as " of Explorer of Microsoft or the off-line stream Viewing device (browser) that some are proprietary, the user can store webpage to disk drive.If the user uses the explorer of Microsoft to store webpage, webpage and its element will be stored into the data folder that a regional webpage and comprised whole elements (the data folder name as " xxx_files ", xxx is the webpage name).Note that even most of advanced persons' webpage off-line stream Viewing device (browser) that some elements of webpage still can not be stored into the local disk machine.For example, if by ASP sign indicating number producing component, then the explorer of Microsoft can not store a little elements to the local disk machine.Furthermore, because the consideration of security, number of site does not allow some elements to be stored into the local disk machine.

Illustration becomes the some of webpage to webpage and figure

Next paragraph, free-format was drawn and directly is not inserted into the webpage of note when we will be illustrated as assorted youngest and record.Yet based on 2 reasons, inserting these figure is necessary to webpage.If a user to print with webpage on drawing, or only want to see graphical annotation on the webpage.As long as two, the explanation that plays sound as the user, and simultaneously can be by spool, click the webpage that (clicking) or the like controls note.

Finishing of inserting is earlier from data stream AGS or from simple stroke (stroke) shelves (an appearance aftermentioned) whole strokes (stroke) time-frame of decoding, save as image shelves form again, such as: but these users such as GIF or PNG can specify the form of a transparent colour in picture format.Utilize the technology of DHTML, CSS and Z-POSITIONING then, image is inserted on the webpage of explaining.Similar program also may be used on above the insertion of image and textBox.

Automatically set up an AG group item

AG group can be set up automatically, can take place when note surpasses a new webpage when the user records.System sets up a node simply on tree structure, represents the group item of note.

Automatically set up an embedded AG group item or an embedded AG course of C-

If AG group item is linked and webpage is local, and when the user will send out the Email of group item or be uploaded to a website, temporary (temp) group item of the AG group item that system will automatically change or sets of copies item to is embedded, that is webpage is stored into compound archives.Equally, if the AG course is C-link, be embedded then, and the compound file that stores all sub-AG groups of accompanying is organized the compound file of item to root AG with notified change root AG group and whole sub-AG group.

Record mode and subsystem

The method that system enters recording mode has two.One, the user can by on the menu record button or Facility Column begins to record.Perhaps, the user can directly be transformed into recording mode when play data stream.The figure of recording subsystem uses the interface as shown in figure 16.Wherein contain Facility Column (going up and a left side), editor's form (blank) or browse form (webpage), and status Bar.

On a webpage, in fact system covers above the webpage with a transparent window if the user wants recorded voice and figure (drawing image and literal).All figure is to be presented on the transparent window.Because form is transparent, figure looks like on the webpage that directly is plotted under it.Use transparent window to replace inline graphics the reasons are as follows to webpage.Though we can write a drama routine (script routine) and detect moving of mouse or pen, and this routine can use this information " draftings " automatically on webpage, however a lot of problems of this method.

One, some part of webpage, such as: hyperlink, mouse click or move and can respond in the past.

Two, the function of drama routine calling needs provide ability " drawing " by DHTML.Yet but there is not original graphic capability, such as: line, promptly needing is provided by DHTML.The conception that small square is set up in some patents propositions, the help of mat DHTML, CSS and Z-positioning (position), the tracking mouse moves and fills up the bar line.The conception of this patent is the color that construction one little square layer fills up pen, inserts these little squares to the Z-position webpage that is higher than the webpage element (just like an image component) and can write a drama sign indicating number.Jaggies and webpage that continuous small square forms overlap.Because the drama routine is the literal translation formula, therefore, the reaction of routine is slower.The user can see draw and show between delay, and the situation that should postpone will increase and cake more and more with the number of times of drawing.What is worse, slow reaction causes that also the drama routine is comparatively not sharp to the detection that mouse moves simultaneously, makes these squares in fact very sparse (if square is as the detection of a bit).The problem that involves in addition also has, and utilize and carry out filter function, such as: antialiasing (anti-aliasing) or the like also is difficult to improve the saw-tooth appearance that continuous square causes.

Three, some webpages are not supported the standard of DHTML, thereby can not insert the drama routine.

Based on a little problems, when recording, record subsystem and use a transparent form to cover webpage.Attention, browser window become non-active, and the information of mouse or pen input refers to the transparent window on top.

Get back to recording mode and subsystem now.The beginning of each notes system has one period prerecording period and recording the period of continuing.At first, be to record button (for example from idle mode) if the user records by pressing, then record subsystem and will do following action in the prerecording period.

Set up a new node and represent new AG group, if webpage or the blank that to be ready explaining are not selected tree nodes.

Changing figure uses interface (GUI) for recording subsystem

(notice that for the situation of back compression, AGS-T2 is actual to refer to original voice files, temporary (temp) data stream then comprises sound flag and other data frame (frame) to newly-increased temporary (temp) AGS-T2 to temporary (temp) AGS-T1.AGS-T1 is also similar), and merge temporary (temp) EA-T2, if (AGS-T2, EA-T2) group is not empty, resets that (AGS-T2 EA-T2) be sky to temporary (temp) EA-T1.

Check recording time, if the user (for example is intended to increase newly, the recording time of user's transition insertion expection is the tail end at AGS-T1), if and (AGS-T1, EA-T1) not empty, insert temporary (temp) AGS-T1 to AGS-O (noticing that the compressible sound time-frame of system before inserting is not if do immediately) and EA-O.

At the state of the present scrolling of each time-frame inside webpage or the blank, set up the scrolling event information to EA-T2.

Set up a transparent form, size is the words of a webpage as containing the browser window of webpage if explain target.

As be blank, the stroke of Xian Shiing (stroke) data frame (frame) then, image and textBox promptly exist at the preceding of blank recording time.As be webpage, then in stroke (stroke) data frame (frame) of transparent window demonstration, image and textBox promptly exist at the preceding of recording time, and their boundary is to be defined by the part that webpage shows at present.

Enter and record the period

The second, if the user is transformed into recording mode when playing (playback), there are two kinds of situations to take place.One, inserts new data and flow to existing data stream.Two, set up a new response group item and arrive broadcast group item.Concerning first kind of situation, prerecording work is:

Use interface to become in figure and record subsystem

Duplicate that (AGS-O-C is EA-O-C) to (AGS-O EA-O), (please remember that system uses temporary (temp) AGS-O-C and EA-O-C to play (playback)) if necessary.And remove (AGS-T1, EA-T1) with (AGS-T2, EA-T2).

Note wherein not needing to show previous figure and setting up a new transparent window, just continuous because they have prepared all.

Concerning the situation of setting up new response group item, prerecording work is:

Set up new response AG group item.Some information members of the data structure of new response AG group item inherit the attribute of original AG group item.For example, the target of note should be identical, and if be a webpage, originally the webpage URL of AG group should be URL that responds AG group or the like.

Set up a node to tree structure for responding AG group item, as the child group item of original AG group item.

The figure that stores the original AG of present display separation organizes a response AG group item to new AGS-T2.

Changing figure uses interface to record subsystem

The event information of setting up scrolling is to state the inside global information webpage or the white board of EA-T2 for the present scrolling of each time-frame.

Enter and record the period

The graphical content that note that these two kinds of situation display screens can't change to recording mode from play mode.Among the latter, originally the part figure of AG group item has been stored into new response AG group item.Stroke (stroke) data frame (frame) of original AG group item, the part figure of image and textBox is to be shown in computer screen.Respond AG group item when the user plays, this figure partly will show at the beginning immediately.

Whenever, record subsystem and enter and record the period, the user can do following thing.

At other blank or at the transparent window graphing that covers webpage.(this will produce stroke (stroke) time-frame to AGS, as simple stroke (stroke) archives).

Select different pen types (solid pen, mark-transparent fully, ALPHA-is transparent, the level pen, vertical pen or the like), the color (RGB) of pen, size (thickness with pen, the perhaps thickness of a size maximum, if this can detect the acting force of writing, that is thickness can change with the amount of force of stroke every bit.

Opening and close sound (microphone) input records.(open when sound, will produce voice data frame (frame))

Pull out (pop up) or (push down) the sketch plate of leaving behind is drawn.(this will produce event information, will produce stroke (stroke) data frame (frame) and draw, and the flag of stroke (stroke) is different from and is used for the flag that blank or webpage are painted)

Select the image shelves to insert any position of form.Image can move around and and can change size.(as the image time-frame, this will produce event information)

Open the word dialog box input characters.The user can select different fonts and font size.After closing edit routine, the user can movetext zone (size is determined by system) to any position, as image blocking.Notice that when the user inserts image or literal block, the sound input will be closed automatically.If want sound import, the user must open sound (this will produce event information and literal time-frame) subsequently.

The size of dynamically scrolling webpage (having only main body), or expansion blank, for example, by touching " expand " control knob on Facility Column or pressing the Page-Down key.(this will produce event information)

Print off display window

Send Email

·?…

In the practical operation, the user can freely add more function.As previously mentioned, record and to produce data frame and event information.The data frame of sound, stroke, literal and image time-frame for be synchronously and the time mix.Incident is to be stored in EA, after be stored in advance data stream (header stream) again.

Status Bar can show the state of recording, such as: present recording time, whether sound is for opening or closing or the like.

After the user stops to record, transparent window will be removed, and system is returned to idle mode.

Play mode and subsystem

Equally, system's mode of entering play mode has two.One, the user can open by the broadcast button on menu or the Facility Column and play its selected AG group item or course, and this area or far-end all can.Perhaps, the user can start the broadcast subsystem by touching ActiveX or residing in inserted component in the webpage.Play the data stream that subsystem begins the target of AG group on the download site or course, and subsystem begins to play the data that are downloaded, the data stream of segmentation, AGS simultaneously.Play user interface such as Figure 17 of subsystem, it comprises Facility Column (going up and a left side), editor's form (blank) or browser window (webpage), and status Bar.

For the AG course, normal playing program is to start from root AG group item, is sub-AG group item then in regular turn, unless the user requires system to start from arbitrary special group of item.

Each play (playback) also had two periods, play period and broadcast period in advance.

There is a little difference this locality and far-end AG group the pre-broadcast period between the two.Suppose that AG group or course are local (E-MAIL that an envelope is received), that is data stream is to be positioned at the local disk machine or to have downloaded and finished.Suppose that it is to be written into a tree structure that AG group item or course have.Pre-action of playing the period is:

According to the attribute of target screen size, change the size of browser window or blank.

Use interface to change in figure and play subsystem.

Regain stroke (stroke) data frame (frame) respectively, image and literal time-frame are to simple stroke (stroke) archives, and image file and literal archives are not as finishing as yet.(this usually occurs in, when AG group or course be this locality).If AG group item or course are far-end, stroke (stroke) time-frame and finishing of other recovery are to do at each received SDS.

Download webpage, if the target of explaining is a webpage.

Each framework of scrolling webpage or blank, by incident array, EA and now the scroll information of reproduction time decide the position.

Set up the transparent window that covers display window, if the target of explaining is a webpage.

Show stroke (stroke) data frame (frame), image and literal time-frame are up to present reproduction time, not to be exactly at transparent window at blank, (stroke is filed from simple stroke (stroke) and is regained, and time that they take place and present reproduction time are relatively, are similar to image and literal block; Yet the time of their generation may be replied from EA).Be not that whole stroke, image and literal all needs to show; System will decide the limit frame (boundary rectangle) of checking them with present display offset.

Check whether system needs to eject (pop up) sketch plate, in this way, then set up the sketch plate and show stroke (stroke) time-frame, allow the sketch plate be transferred to present reproduction time.

Scan data stream is up to present reproduction time (need not show stroke (stroke) data frame (frame) that runs into, image and lteral data frame (frame)).

Begin the decoding and the time-frame that plays sound, stroke (stroke) time-frame or the like.That is, enter and play the period.

If AG group item is a far-end, the action of playing the period in advance is:

As be a course, download the advance data stream of AG group or root AG group, and if download as yet and finish, then set up a shape shape node and carry this group item.

According to the attribute of target screen size, change the size of browser window or blank.

Use interface to become the broadcast subsystem in figure.

Download webpage, if the target of explaining is a webpage.

Download the SDSs of AG group item or course, and reply stroke (stroke) data frame (frame) respectively, image and literal time-frame are to simple stroke (stroke) archives, image file and literal archives.These actions are finished in background.(as being course, generally download will begin with the SDS of root AG group item, then, then is son group item in regular turn.Yet if the user selects from special sub-AG group item, download system will begin with this group corresponding SDSs.

Yet system downloads SDS can carry out in background, system whether will check simultaneously this SDS be received or do not have (note, each SDS for some time, system needs to download earlier several SDS of beginning, to determine that the time sum total is greater than reproduction time)

Behind the reception SDS, for the above-mentioned step 5 of AG group Xiang Zeyi of this locality.

Still some additional explanation of broadcast about the AG course.If the AG course is local, the user can select the group item in any tree structure, and root or arbitrary son group item begin.At this moment, the broadcast of course is with a selected AG group beginning.

When the AG course is the also same of far-end.When downloading the advance data stream of root AG group item, then its a sub-AG group whole title and position is known by system.At this moment, the user can stop to play (playback) and switch to other sub-AG group broadcasts.Then, system will suspend the download (as not finishing as yet) of other AG group items, and change downloading selected AG group item.Thereby the group item order of an AG course can be downloaded by instantaneously changing in response to user's action, and then promotes downloading efficiency.

In playing the period, system obtains time-frame one by one.Each time-frame will be by escape and broadcast or demonstration.The sound time-frame is used for showing other data frame (frame) simultaneously also as a timer.With stroke, image or literal time-frame is example, when the time-frame between sound time-frame N and N+1 is decoded, then can demonstrate the time of N*D, and wherein D is the time span of a sound time-frame.Except decoding and broadcast, system also supervises with EA and shows the time (sound counting), sees the incident that whether runs into.Some incidents, such as: insert image and all exist in EA and the data stream, so system can be used for verifying sequential.If incident comes across EA, system must produce this incident again.This usually occurs in the scrolling webpage, ejects the sketch plate, inserts image and deleted image or the like.

System except must according to the decoding of data stream and event information, play, show and produce again incident, system also needs to respond simultaneously user's interaction.Classify the action that the user can carry out when playing down as:

Suspend and the answer broadcast

Suspend and play and deliver problem (at this moment, system will set up the response AG group AG that plays a up till now group item, and the part figure of playing AG group item at present will become the initial graphics of responding AG group item).

Switch to and record (it is local having only when AG organizes item, and can edit, and system enters recording mode then)

Suspend and the definition time mark

Print

Send e-mails

The user also can add some functions to playing subsystem.

The present invention is by the AG data stream, data recordings such as sound, figure, literal, drawing are become AG group item, can do editor's action to elements such as sound, figure, literal, drawing, but also send Email is given the related personnel, is uploaded to the website and become CD-ROM etc. for people's download or broadcast, burning.Can play this data stream, and in playing process, if in playing, have problem, can be at problem points, question, and set up an AG data stream simultaneously, and become a sub-AG group item, problem is replied to the teacher, the teacher also can do the most effective answer at this problem.

With the distance learning is example, and teacher can be with the mode of course to teach on the similar blackboard, and AG course (being made of the AG group) is recorded in the sound of teaching, comment, graphic extension etc., sends the student to.Arbitrary AG group (for example a certain chapters and sections) that the student can open in the AG course is play, in the playing process, the student may be to some picture, noun or explanation are not understood, this moment, the student can be somebody's turn to do the background in playing, at any time insert (recording) another section AG group (i.e. response group item), for example go up at noun " Patent " and draw a circle with the pen circle, say simultaneously " this noun is assorted the one meaning " or the like, after recording, this response group item can be returned to teacher, teacher can be at this student's problem, do a response group item again, content can for, tap on the word of " Patent " with pen, say then: " this word of Patent is the meaning of patent ", should pass to the student by response group item again, when the student opens teacher's response group item, can on screen, see, the plotting motion that pen taps at the word of " Patent ", hear that simultaneously teacher says: " this word of Patent is the meaning of patent " the sound meaning, thereby the student can obtain the answer that he wants at once, plays the mode of transparency compared with the shadow bands teaching of commonly using or simple webpage, and this wants much more efficient.

Except distance learning, technology of the present invention also may be used on various different field, link up so long as utilize the long distance of sound graphics context mode, all can use technology of the present invention and promote communication efficient, for example the research staff on two ground is at the discussion of their plan case, and also available this mode is carried out.Perhaps someone is when online, some new noun or picture are not understood on the discovery webpage, when wanting to ask someone to explain, also webpage can be imported, record " point out picture on the webpage or the plotting motion and the related sound of literal with pen ", make one AG group, pass to the professional person, this professional person opens this AG group item, can be at once at problem, record its answer and make a response group item, replied.

The method of computer numeral teaching of the present invention is the method for coming digitizing (digitize) and classroom-simulating teaching by computer.Utilize this method, the instructor can use webpage or the computer blank (whiteboard) that can expand the different size size to draw, dynamically insert elements such as (or deletion) image and literal on (or from) blank (whiteboard) or webpage, and can record the sound of teaching simultaneously.This device can store education activities to a multiplex data stream and advance data stream (header stream).Data stream and note webpage are formed a sound figure (AG) group item.The user can be filed data stream, send Email, or passes on the website, plays for the people.This device comprises works (authoring), broadcast (playback), marshalling (organize) and the index of AG group item.The data stream of AG group item comprises (compressed) voice data frame (frame) of compression, (compressed) stroke (stroke) data frame (frame), image and the lteral data frame (frame) of compression.Advance data stream comprises information and event information (sequential is arranged), the display window (form) when being used for controls playing (playback).Via using event information and stroke temporary grade (the literal shelves of stroke), the present invention can reduce pretreated delay can begin sound import for the user and record stroke before, or before the user can begin to see the result of broadcast (playback).Inner data stream (archives) operation (inserting new data stream when for example, recording) of using multiple incident array to reduce complexity when editing of the present invention.This system uploads to the preceding of website in AG group item very long data stream is divided into a lot of data stream than subsection.This segment information (as the number of segmentation) and position be store in advance data stream (header stream).The data stream that data stream is divided into preposition shelves and segmentation can make the time less of broadcasting on the line, even also can reaching congested networking at a slow speed.AG group Xiang Qunke is construed as an AG course.The AG course can be can link person or embedded person.Link AG course is stored in the AG group item of the outside of AG course archives for some; Otherwise, be exactly embedded AG course.The user can be filed course, send e-mails or deliver on the website, as same single AG group item.

Claims (13)

1. the method for a computer numeral teaching is characterized in that, is included in to carry out the following step in the computer:

Set up a data stream, this data stream comprises a multiplex data stream and advance data stream, uses data such as carrying one sound, stroke, image and literal;

Edit this data stream, use and in this data stream, insert new sound, stroke, image and a lteral data, and delete an old sound, stroke, image and lteral data; And

Transmit this data stream, use and upload and download this data stream.

2, the method for computer numeral teaching as claimed in claim 1 is characterized in that, this computer is to be connected to a website, and this computer is able to this data stream is filed, and sends the Email of this data stream, and transmits these data and flow on this website and play for the people.

3, the method for computer numeral teaching as claimed in claim 1 is characterized in that, this data stream and is explained webpage and formed a sound figure (AG) group item, and this advance data stream is to comprise an information and an event information.

4, the method for computer numeral teaching as claimed in claim 1 is characterized in that, a user is able to this sound of works, broadcast, marshalling and index figure (AG) group item.

5, the method for computer numeral teaching as claimed in claim 1, it is characterized in that, this sound, stroke, data such as image and literal are the data for a compression, and be a content of courses, this content of courses is to be carried out on a computer blank by the person of giving lessons, and is able to a course group item is formed in this content of courses.

6, the method for computer numeral teaching as claimed in claim 5, it is characterized in that, this sound, stroke, data such as image and literal are a response content of being made by a student, this response content be this student at the problem that this content of courses proposed, and be able to this response content is formed a response group item.

7, a kind of method of computer numeral teaching is characterized in that, comprises the following step:

Provide a computer blank, an audio input device and a drawing input equipment;

Record one first sound of this audio input device and one first figure that this drawing input equipment is plotted in this computer blank, form one first data stream;

Transmit this data stream; And

Play this data stream, in playing, recorded one second sound of this audio input device and the second graph that this drawing input equipment is plotted in this computer blank, form one second data stream.

8, the method for computer numeral teaching as claimed in claim 7 is characterized in that, this audio input device is a microphone, and should the drawing input equipment be an input pen.

9, the method for computer numeral teaching as claimed in claim 7 is characterized in that, this computer blank is able to be replaced by a webpage, and when recording, this drawing input equipment is that this figure is plotted in a transparent window that is covered on this webpage.

10, the method for computer numeral teaching as claimed in claim 7 is characterized in that, this computer blank is a blank webpage.

11, the method for computer numeral teaching as claimed in claim 7 is characterized in that, the data of this data stream are the compressor, and this data stream is to comprise a multiplex data stream and advance data stream, and this advance data stream comprises an information and an event information.

12, a kind of method of computer numeral teaching is characterized in that, comprises the following steps:

Record a voice, the literal of a figure of drawing and a typewriting makes a data stream;

Transmit this data stream; And

Play this data stream.

13, the method for computer numeral teaching as claimed in claim 12 is characterized in that, this data stream one figure sound data stream.

Images