CN110874959A - Multi-terminal same-screen teaching system and teaching method - Google Patents
Multi-terminal same-screen teaching system and teaching method Download PDFInfo
- Publication number
- CN110874959A CN110874959A CN201811023491.6A CN201811023491A CN110874959A CN 110874959 A CN110874959 A CN 110874959A CN 201811023491 A CN201811023491 A CN 201811023491A CN 110874959 A CN110874959 A CN 110874959A
- Authority
- CN
- China
- Prior art keywords
- screen
- terminal
- accelerator
- student
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims description 31
- 230000005540 biological transmission Effects 0.000 claims abstract description 33
- 238000004891 communication Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims description 28
- 238000001514 detection method Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 9
- 239000003999 initiator Substances 0.000 claims description 5
- 230000000903 blocking effect Effects 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 3
- 230000000977 initiatory effect Effects 0.000 claims description 2
- 238000009877 rendering Methods 0.000 claims description 2
- 238000007726 management method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 239000010410 layer Substances 0.000 description 2
- 238000011022 operating instruction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000002344 surface layer Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/06—Electrically-operated educational appliances with both visual and audible presentation of the material to be studied
- G09B5/065—Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/08—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
- G09B5/14—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations with provision for individual teacher-student communication
Abstract
A multi-terminal on-screen teaching system comprises a teacher terminal, a student terminal, an accelerator, a wireless AP, a large-screen display device and a background server, wherein the wireless AP is accessed to a campus local area network or a wide area internet in a wireless or wired mode, the teacher terminal, the student terminal and the wireless AP are in wireless connection, and the wireless AP and the accelerator are in wired connection; video transmission between each terminal and the accelerator is carried out by adopting a dynamic block algorithm, and the terminals comprise a teacher terminal and a student terminal; the bottom layer communication among each terminal, the background server, the large-screen display device and the accelerator adopts an asynchronous input and output system, and the communication protocol adopts a TCP protocol.
Description
Technical Field
The invention belongs to the field of on-screen teaching, and particularly relates to a multi-terminal on-screen teaching system and a multi-terminal on-screen teaching method.
Background
With the popularization of intelligent terminals, teaching modes are changing, and the application of the intelligent terminals in a classroom becomes a research hotspot. More and more schools begin to use intelligent terminals to perform classroom teaching by means of projectors or electronic whiteboards. Although the teaching mode greatly improves the teaching diversity and interactivity compared with the traditional blackboard teaching, the teaching mode has the following defects due to the limitation of sites; students behind the seat have difficulty in seeing the white board or the projected content due to the occlusion of the students in front. Therefore, how to make each student obtain an equal experience of listening to lessons becomes an urgent problem to be solved.
The enhancement of the capability of the intelligent terminal equipment enables screen sharing and interaction between the terminal equipment to be possible. The multi-screen interaction means that the content displayed on the screen of one terminal device is transmitted to another terminal device for display by using technologies such as wireless transmission, and meanwhile, the screen of the receiving terminal device is controlled by the sending terminal device. In the prior art, a multi-screen interaction technology is mainly applied to a one-to-one scene, such as a case where mobile phone screen content is delivered to a television or a projection.
The prior technical scheme is as follows: DLNA, wiredisplay (miracast), AirPlay, etc., wherein DLNA: networking is carried out based on UPNP, the existing multimedia resources or files can be played wirelessly, and screen projection is not supported; WifiDisplay: the wireless screen projection can be realized, but the connection establishment process is slow, the operation is not easy, and the experience is not good; AirPlay: file playback and screen projection are supported, but more are only applicable to authenticated apple devices and some apple authorized partner devices.
In addition, the technical scheme is mainly applied to one-to-one screen projection scenes, cannot bear a plurality of terminals, cannot be automatically reconnected in unstable network environments and in scenes with disconnected lines, and does not support screen control.
Disclosure of Invention
The invention provides a multi-terminal same-screen teaching system and a teaching method thereof, aiming at solving the technical problem of realizing the switching between a one-to-many sharing screen and a small and large screen,
the technical scheme of the invention is as follows: a multi-terminal on-screen teaching system comprises a teacher terminal, a student terminal, an accelerator, a wireless AP, a large-screen display device and a background server, wherein the wireless AP is accessed to a campus local area network or a wide area internet in a wireless or wired mode, the teacher terminal, the student terminal and the wireless AP are in wireless connection, and the wireless AP and the accelerator are in wired connection;
the background server is a local server or a cloud server of a network cloud and is used for storing lesson opening information uploaded by the teacher terminal, the lesson opening information comprises parameters including subjects, textbooks, chapters and accelerator IP, when the teacher terminal is started, the background server pushes the lesson opening information to the student terminal, and the student terminal automatically enters a classroom to connect with an accelerator according to the received lesson opening information;
the accelerator is used for the same-screen transmission and large-screen output of screen content and audio data between the teacher terminal and the student terminals, playing various files stored in the accelerator as an independent terminal, and transmitting the screen content and the audio data to the teacher terminal and the student terminals in the same screen;
video transmission between each terminal and the accelerator is carried out by adopting a dynamic block algorithm, and the terminals comprise a teacher terminal and a student terminal;
the bottom layer communication among each terminal, the background server, the large-screen display device and the accelerator adopts an asynchronous input and output system, and the communication protocol adopts a TCP protocol.
A multi-terminal one-screen teaching method adopting the system comprises the following steps:
step 2, the teacher terminal and each student terminal are connected with the accelerator;
step 3, screen projection processing is carried out on the teacher terminal;
step 4, the accelerator displays screen projection processing and screen sharing processing;
step 5, each student terminal displays a sharing screen and answers the screen;
step 6: the teacher terminal is embedded with file service;
and 7: and (5) carrying out screen control processing on the teacher terminal.
The invention has the beneficial effects that:
1. one-to-many same-screen display in a wireless local area network is realized by adopting a wired/wireless network combined mode, adopting an asynchronous input and output system and applying means such as a dynamic block algorithm, frame frequency control, flow control and the like;
2. one-to-many same screens support disconnection reconnection, have high real-time performance and reliability, have controllable image quality and resolution ratio, and realize equal education. By expanding the instruction, richer interactive experience can be provided;
3. the screen control is realized, the constraint is relieved, a teacher does not need to be bound by a platform or an all-in-one machine, the teacher can move freely in a classroom, and the accelerator large screen is controlled to perform more interaction and communication with students;
4. and the system is compatible with various courseware resources such as EXE, Flash, teaching and assisting tools and the like.
Drawings
FIG. 1 is a system architecture diagram of the present invention;
FIG. 2 is a block diagram of the teacher's terminal according to the present invention;
FIG. 3 is a block diagram of an accelerator according to the present invention;
fig. 4 is a block diagram of a student terminal of the present invention;
fig. 5 is a schematic diagram of the configuration of the proactive model of the present invention.
Detailed Description
The invention will be further described with reference to the accompanying figures 1-5.
A multi-terminal on-screen teaching system comprises a teacher terminal, a student terminal, an accelerator, a wireless AP, a large-screen display device and a background server, wherein the wireless AP is accessed to a campus local area network or a wide area internet in a wireless or wired mode, the teacher terminal, the student terminal and the wireless AP are in wireless connection, and the wireless AP and the accelerator are in wired connection;
the wired/wireless network combined mode can well solve the problem of bandwidth bottleneck of a video source sending end under a pure wireless network environment (due to the fact that the number of student terminals is large, if the teacher terminal directly carries out on-screen delivery, due to the fact that the teacher terminal has limited processing capacity and network bandwidth and other problems, low-delay and smooth on-screen playing cannot be achieved), and bandwidth liability capacity of a wired network is fully developed.
The background server is a local server or a cloud server of a network cloud and is used for storing lesson opening information uploaded by the teacher terminal, the lesson opening information comprises parameters including subjects, textbooks, chapters and accelerator IP, when the teacher terminal is started, the background server pushes the lesson opening information to the student terminal, and the student terminal automatically enters a classroom to connect with an accelerator according to the received lesson opening information;
the accelerator is used for the same-screen transmission and large-screen output of screen content and audio data between the teacher terminal and the student terminals, playing various files stored in the accelerator as an independent terminal, and transmitting the screen content and the audio data to the teacher terminal and the student terminals in the same screen.
Wherein, the teacher terminal comprises a screen projection part, a screen control part, a teacher terminal control module and an embedded file service module, the teacher terminal control module is respectively connected with the screen projection part, the screen control part and the embedded file service module for data transmission,
the screen projection part comprises a scanning module, a mouse simulator, a screen recording module, a frame frequency control module, a coding module, a disconnection reconnection module and a communication module;
the screen control part comprises a gesture detection module, a video playing module, a decoding module, a disconnection reconnection module and a communication module;
wherein, the student terminal comprises a screen control part, a student terminal control module and a file management module, the student terminal control module is respectively connected with the screen control part and the file management module for data transmission,
the screen control part comprises a gesture detection module, a video playing module, a decoding module, a disconnection reconnection module and a communication module;
wherein, the accelerator comprises a same-screen accelerator module, a file management module, a screen projection part and a screen split part, the same-screen accelerator module is respectively connected with the file management module, the screen projection part and the screen split part for data transmission,
the screen projection part comprises a mouse detection module, a video playing module, a decoding module and a communication module;
the split screen part comprises a two-dimensional code generation module, a mouse simulator, a screen recording module, a coding module, a flow control module and a communication module;
wherein, the accelerator is a smart device in a classroom, such as a computer or tablet.
The video transmission between each terminal and the accelerator is transmitted by adopting a dynamic block algorithm, the terminals comprise a teacher terminal and a student terminal, in the transmission process, only bitmap blocks where changed pixel points are located need to be transmitted, and the dynamic block algorithm comprises the following specific steps:
step 2, starting from the origin, scanning along the positive direction of the X, Y axis, judging the scanning condition when the I-th line scanning is finished, if the I-th line finds a pixel change point, recording each pixel change point (P)Ix,PIy) If no pixel change point is found in the I-th line, entering a blocking stage and recording the line number If；
Step 3, passing the pixel change point (P)Ix,PIy) Determining the diagonal coordinates (R) of the transmission rectangleLx,RLy) And (R)Rx,RRy) Wherein, in the step (A),
RLx＝MIN(P1x,P2x,...,PIx,...)，
RLy＝MIN(P1y,P2y,...,PIy,...)，
RRx＝MAX(P1x,P2x,...,PIx,...)，
RLy＝MAX(P1y,P2y,...,PIy,...)
wherein, MIN (), MAX () represent the minimum function and the maximum function, respectively;
step 4, after the selection of the transmission rectangle is completed, the transmission rectangle is selected fromI thfStarting at line, starting scanning at N lines, and setting I as If+ N, returning to the step 2, generating a corresponding transmission rectangle, and when I is more than or equal to M, M is the whole row number of the bitmap, and the whole bitmap is scanned completely;
and 5, drawing a transmission rectangle at a designated position according to the diagonal coordinates, displaying a scanning moment buffer bitmap in other areas, rendering the received transmission rectangle to the designated area in the process of terminal display, and displaying the unchanged area at the same time as the last time.
The method can automatically judge the transmission rectangle according to the change area, greatly reduces the transmission data amount, reduces the requirements on network bandwidth and hardware resources, and ensures the real-time performance and the fluency of the same-screen transmission in the teaching process.
Wherein, the bottom layer communication among each terminal, the background server, the large-screen display device and the accelerator adopts an asynchronous input and output system, the communication protocol adopts a TCP protocol, the asynchronous input and output system comprises a channel, a buffer area, a selector and a forward shooting model,
the buffer is a data container, the buffer object is composed of an array for storing data and data read-write attributes, all data transmission is completed through the buffer, the buffer has four attributes for providing information about data elements contained in the buffer, and the attributes include:
capacity: a maximum number of data elements that the buffer can accommodate, the capacity being set at the time of buffer creation and not being able to be changed;
upper limit: the first element of the buffer that cannot be read or written represents a count of the existing elements in the buffer;
position: the index of the next element to be read or written, the position being automatically updated by the corresponding method;
marking: a memo position that is set by calling a mark;
the channel provides a direct connection with the I/O service for effectively transmitting data between the buffer and the file on the other side of the channel, the channel is bidirectional, and the channel can better reflect the real situation of the underlying system;
the selector is for maintaining a set of registered channels, a selection key inside of which encapsulates the registration relationship of a particular channel with a particular selector, the selection key being returned and providing indicia of the registration relationship, the selection key comprising two sets of bits indicating the channel operation of interest for the registration relationship and the operation for which the channel is ready, each selector object maintaining a set of three keys: a set of registered keys, a set of selected keys, a set of cancelled keys, a selector providing a ready selection, the ready selection and the multi-execution enabling a single thread to efficiently manage multiple channels simultaneously;
the proactive model is an asynchronous I/O model, in the proactive model, a terminal initiates I/O operation, an operating system copies network I/O data from a kernel cache region to a terminal region, and when an event separator monitors that I/O operation reading and writing are completed, a terminal thread is informed to process coding and decoding, so that the proactive model can better separate the I/O operation from service operation, and has higher performance.
The proactive model is composed of a processor, an interface and a thread pool,
wherein, the treater includes:
(1) a proactive initiator: the proactive initiator is any entity which initiates asynchronous operation in application, and registers an asynchronous channel which needs to execute asynchronous operation into an asynchronous operation processor, and the asynchronous operation processor calls back a method for completing the asynchronous operation processor after the operation is completed, so as to realize service logic processing;
(2) and a finishing processor: the completion processor interface is used for receiving the call of the asynchronous operation processor;
(3) an asynchronously operating processor: the asynchronous operation processor is responsible for managing a plurality of channels registered thereon, binding the thread pool, and initiating the next operation and issuing a completion notice according to the actual application condition.
Wherein the interface includes:
(1) the device comprises a proactive receiving interface, a monitoring server and an asynchronous operation processor, wherein the proactive receiving interface is used for binding a port, starting monitoring, receiving a request when a client initiates connection to a server, registering an interested operation of a client channel and simultaneously registering the asynchronous operation processor;
(2) the proactive connection interface forms an active connection interface of the client;
(3) the system comprises a proactive channel interface, a proactive receiving interface and a proactive channel interface, wherein the proactive channel interface provides an interface for an actual read-write channel in the network I/O, and the proactive receiving interface receives a connection request of a client and is used for read-write operation from a terminal to a server;
(4) the proactive processing interface manages the channels registered on the proactive processing interface, updates interest events, binds a thread pool and is used for task dispatch, and when I/O operation is completed, a corresponding method in the asynchronous operation processor is called back to be completed.
The asynchronous connection and read-write establishment of the asynchronous input and output system comprises the following steps:
step 2, the terminal initiates a connection request, the proactive group processing interface initiates an asynchronous receiving operation to receive the request from the terminal, and after the receiving operation is completed, a receiving completion method in the completion processor is called;
step 3, initializing a proactive channel interface by the finishing processor, packaging an asynchronous channel entity containing all the attributes of the link by the proactive channel interface, and finishing the read-write operation of two ends of the link through the asynchronous channel entity;
step 4, the event which is interested in the event is registered on the proactive group processing interface by the proactive channel, and after the event is finished, the reading completion method in the completion processor is called back again to complete the processing of the service logic;
step 5, after the link is established, the corresponding proactive channel of the link registers an interested reading event to the proactive group processing interface;
step 6, the terminal sends data;
step 7, the proactive group processing interface monitors the readiness of the I/O event and calls a thread to execute the I/O reading operation;
step 8, after the I/O reading event is finished, the front-shot group processing interface calls a reading completion method of a completion processor to execute decoding, coding and service logic processing;
step 9, registering the interested events of the proactive channel again;
step 10, after the service logic processing is finished, bringing back data to be sent to the terminal to a finishing processor;
step 11, returning the data to the completion processor, and executing the writing operation of the proactive channel;
and step 12, after the proactive group processing interface monitors that the I/O write operation is finished, calling a write completion method of the completion processor, and executing the next logic.
The asynchronous input and output system reads and writes data in a block mode, one data block is consumed in each operation, the data is much faster than the read and write data of one byte, the asynchronous input and output system supports non-blocking I/O operation, and a single thread can effectively manage a plurality of I/O channels simultaneously.
The asynchronous input and output system ensures real-time performance, high concurrency and reliability of communication, and the accelerator is also provided with a flow control module which dynamically adjusts the frequency of sending data according to the load condition of each student terminal communication channel, thereby effectively inhibiting the problem of network congestion.
A multi-terminal one-screen teaching method adopting the system comprises the following steps:
after the accelerator is started, a two-dimensional code containing accelerator IP information is generated and displayed, two service ports (51666 and 51667) are fixedly monitored, connection between the teacher terminal and each student terminal is waited, and the teacher terminal enters a ready state.
The screen projection port (51666 port) is used for displaying screen projection contents of the teacher terminal, and the screen splitting port (51667 port) is used for sharing screen contents of the accelerator.
Step 2, the teacher terminal and each student terminal are connected with the accelerator,
the teacher terminal obtains accelerator IP information by scanning the two-dimensional code and then is connected to a screen projection port of the accelerator, meanwhile, the teacher terminal pushes lesson opening information containing the accelerator IP information to each student terminal through a background server (the teacher terminal and the student terminals can be communicated through other networks), the student terminals are automatically connected to a screen division port of the accelerator, the teacher terminal and each student terminal report terminal equipment information to the accelerator, and the accelerator is associated with a communication channel thereof according to the collected equipment information to establish session management.
The teacher terminal can inquire and manage the session information of the student terminals through the accelerator. The teacher terminal and the student terminals both comprise disconnection reconnection modules, and can be automatically reconnected when the disconnection condition occurs due to unstable network.
Step 3, the teacher terminal performs screen projection processing,
after the teacher terminal is connected to the accelerator, screen recording is started, and recorded contents comprise audio (including MIC and system sound) and video (screen content), wherein the audio is encoded by AAC, and the video is encoded by H264.
Because the frame frequency of the Android native screen is uncontrollable, when the frame dynamic rate is high, the maximum FPS (FramesPerSecond) can reach more than 40 frames, great pressure is caused on the bandwidth, and the requirement of real-time transmission and playing cannot be met, the teacher terminal uses SurfaceView (a view object on the surface layer in Android) as a secondary dump to realize effective frame frequency control, so that the frame frequency can be adjusted freely within the allowable range of hardware performance, the aim of improving the image quality and the fluency under the condition of the same bandwidth is fulfilled, and the recorded content is sent to an accelerator as screen projection content.
Wherein, the parameters related to the encoding and transmission of the frame rate are adjustable, such as code rate, frame rate and resolution.
Step 4, the accelerator displays screen projection processing and screen sharing processing,
after receiving screen projection content sent by the teacher terminal, the accelerator decodes the screen projection content and outputs the decoded screen projection content to a large-screen display device for display; on the other hand, the screen of the accelerator is recorded and encoded in the background (the encoding quality is low compared with that of the teacher terminal, and the encoding can be realized by setting parameters), and the screen is shared and sent to each student terminal.
The screen of the accelerator is recorded and recoded for sharing, rather than the screen projection content sent by the teacher terminal is decoded and recoded for sharing, in consideration of the classroom teaching scene, there is a scene that the desktop of the accelerator is displayed to the student terminal or even the teacher terminal (for example, exe is opened by a PC, teaching and assistance tools, etc.), and at this time, the screen of the accelerator needs to be recorded certainly. In order to avoid processing two actions (screen projection by the teacher end and screen projection by the accelerator), the actions are combined into a processing mode, namely screen recording is carried out on the accelerator.
Generally, as the number of student terminals is large, in order to realize low-delay on-screen display, the encoding parameters adopted by the accelerator terminal are different from the screen recording parameters of the teacher terminal, and if the code rate setting is relatively low. In practical application, in order to adapt to different teaching environments, the coding quality of the accelerator end screen can be determined by setting relevant parameters (code rate, frame rate and resolution) on the same-screen software. When the number of the student terminals is small, the coding quality higher than the number of the student terminals can be set by selecting the related parameters, and when the number of the student terminals is large, the coding quality lower than the number of the student terminals can be set by selecting the related parameters.
Step 5, each student terminal displays a sharing screen and answers the screen,
the student terminal decodes the video data and the audio data received in real time and then plays the video data and the audio data, and the playing of the video stream and the playing of the audio stream can be matched according to time characteristics, for example, a uniform timestamp is generated during recording;
in the process of on-screen, an accelerator detects the connection state of a teacher terminal and each student terminal and the accelerator through heartbeat in real time, when the student terminals are disconnected with the student terminals, inactive student terminal channels can be closed, resources occupied by the student terminals are released, the student terminals can also detect the connection state between the student terminals and the accelerator, when a disconnection condition occurs, the student terminals can close the student terminals, reconnection operation is initiated after a specified time interval until the connection is successful, after the reconnection is successful, the accelerator sends recently reserved key frame information to the student terminals, and then sequentially sends real-time coding information to the student terminals, so that the latest on-screen content can be acquired quickly after the student terminals are disconnected and recovered.
The teacher terminal controls the student terminals to open or close gesture detection through an extension instruction, after the teacher terminal puts in the large screen, students answer the large screen, and the answering process is directly put in the large screen to be displayed.
Specifically, instructions are mutually transmitted between the teacher terminal and the student terminals through the accelerator or without the accelerator, and the specific instructions can be expanded and added at any time according to business requirements.
For example: teacher sends a GestureDetectEnable instruction for appointed student, student terminal receives the instruction after, start gesture detection, the student does the painting brush action on student terminal screen, student terminal transmits student's operating instruction to teacher terminal through the accelerator, teacher terminal carries out the analog operation to student's operating instruction at teacher terminal again, student's action just like this, just draw on teacher's end's drawing board, what put in on the big screen at this moment is teacher's screen, when ending with the screen, after teacher terminal sends the instruction of ending with the screen to each student terminal, student terminal automatic disconnection and accelerator's communication session.
Step 6: the teacher's terminal embeds a file service,
the teacher terminal is embedded with file service, files on the local computer are shared to the accelerator and the student terminals, and the accelerator downloads the files on the teacher terminal through the file management module and then plays the local computer.
The courseware resources of teachers are various, the types of files supported by the mobile terminal are limited, and many times, files which cannot be played by the mobile terminal can be downloaded by the accelerator and then played and displayed, so that the compatibility of the whole system to various files is greatly improved.
And 7: the teacher terminal performs a screen manipulation process,
the teacher terminal can also be used as a terminal for sharing the screen by the accelerator, is connected to a screen splitting port (51667) of the accelerator, starts a gesture detection function, plays a picture of a large screen of the accelerator on the teacher terminal at the moment, and can perform remote operation on the large screen by clicking and double-clicking gestures, such as file opening, video playing and the like.
The above-described embodiment merely represents one embodiment of the present invention, but is not to be construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
Claims (10)
1. A multi-terminal on-screen teaching system comprises a teacher terminal, a student terminal, an accelerator, a wireless AP, a large-screen display device and a background server, wherein the wireless AP is accessed to a campus local area network or a wide area internet in a wireless or wired mode, the teacher terminal, the student terminal and the wireless AP are in wireless connection, and the wireless AP and the accelerator are in wired connection;
the background server is a local server or a cloud server of a network cloud and is used for storing lesson opening information uploaded by the teacher terminal, the lesson opening information comprises parameters including subjects, textbooks, chapters and accelerator IP, when the teacher terminal is started, the background server pushes the lesson opening information to the student terminal, and the student terminal automatically enters a classroom to connect with an accelerator according to the received lesson opening information;
the accelerator is used for the same-screen transmission and large-screen output of screen content and audio data between the teacher terminal and the student terminals, playing various files stored in the accelerator as an independent terminal, and transmitting the screen content and the audio data to the teacher terminal and the student terminals in the same screen;
video transmission between each terminal and the accelerator is carried out by adopting a dynamic block algorithm, and the terminals comprise a teacher terminal and a student terminal;
the bottom layer communication among each terminal, the background server, the large-screen display device and the accelerator adopts an asynchronous input and output system, and the communication protocol adopts a TCP protocol.
2. The multi-terminal on-screen teaching system according to claim 1, wherein: the teacher terminal comprises a screen projection part, a screen control part, a teacher terminal control module and an embedded file service module, wherein the teacher terminal control module is respectively connected with the screen projection part, the screen control part and the embedded file service module for data transmission,
the screen projecting part comprises a scanning module, a mouse simulator, a screen recording module, a frame frequency control module, a coding module, a disconnection reconnection module and a communication module,
the screen control part comprises a gesture detection module, a video playing module, a decoding module, a disconnection reconnection module and a communication module;
wherein, the student terminal comprises a screen control part, a student terminal control module and a file management module, the student terminal control module is respectively connected with the screen control part and the file management module for data transmission,
the screen control part comprises a gesture detection module, a video playing module, a decoding module, a disconnection reconnection module and a communication module;
wherein, the accelerator comprises a same-screen accelerator module, a file management module, a screen projection part and a screen split part, the same-screen accelerator module is respectively connected with the file management module, the screen projection part and the screen split part for data transmission,
the screen projection part comprises a mouse detection module, a video playing module, a decoding module and a communication module,
the split screen part comprises a two-dimensional code generation module, a mouse simulator, a screen recording module, a coding module, a flow control module and a communication module.
3. The multi-terminal on-screen teaching system according to claim 1, wherein the dynamic blocking algorithm comprises the following specific steps:
step 1, caching a bitmap of a previous scanning moment of a video by a terminal, intercepting a new bitmap at the current moment, establishing a rectangular coordinate system by taking the upper left corner of the bitmap as an origin, the line scanning direction as the positive direction of an X axis and the scanning line increasing direction as the positive direction of a Y axis;
step 2, starting from the origin, scanning along the positive direction of the X, Y axis, judging the scanning condition when the I-th line scanning is finished, if the I-th line finds a pixel change point, recording each pixel change point (P)Ix,PIy) If no pixel change point is found in the I-th line, entering a blocking stage and recording the line number If；
Step 3, passing the pixel change point (P)Ix,PIy) Determining the diagonal coordinates (R) of the transmission rectangleLx,RLy) And (R)Rx,RRy) Wherein, in the step (A),
RLx＝MIN(P1x,P2x,...,PIx,...)，
RLy＝MIN(P1y,P2y,...,PIy,...)，
RRx＝MAX(P1x,P2x,...,PIx,...)，
RLy＝MAX(P1y,P2y,...,PIy,...)
wherein, MIN (), MAX () represent the minimum function and the maximum function, respectively;
step 4, after the selection of the transmission rectangle is finished, the I-th orderfStarting at line, starting scanning at N lines, and setting I as If+ N, returning to the step 2, generating a corresponding transmission rectangle, and when I is more than or equal to M, M is the whole row number of the bitmap, and the whole bitmap is scanned completely;
and 5, drawing a transmission rectangle at a designated position according to the diagonal coordinates, displaying a scanning moment buffer bitmap in other areas, rendering the received transmission rectangle to the designated area in the process of terminal display, and displaying the unchanged area at the same time as the last time.
4. The multi-terminal on-screen teaching system according to claim 1, wherein: the asynchronous input and output system comprises a channel, a buffer area, a selector and a proactive model, wherein the proactive model consists of a processor, an interface and a thread pool, and the processor comprises:
a proactive initiator: the proactive initiator is any entity which initiates asynchronous operation in application, and registers an asynchronous channel which needs to execute asynchronous operation into an asynchronous operation processor, and the asynchronous operation processor calls back a method for completing the asynchronous operation processor after the operation is completed, so as to realize service logic processing;
and a finishing processor: the completion processor interface is used for receiving the call of the asynchronous operation processor;
an asynchronously operating processor: the asynchronous operation processor is responsible for managing a plurality of channels registered thereon, binding the thread pool, and initiating the next operation and issuing a completion notice according to the actual application condition.
5. A multi-terminal on-screen teaching method according to any one of claims 1-2, comprising the steps of:
step 1, starting an accelerator, entering a ready state, and waiting for a teacher terminal and each student terminal to be connected with the accelerator;
step 2, the teacher terminal and each student terminal are connected with the accelerator;
step 3, screen projection processing is carried out on the teacher terminal;
step 4, the accelerator displays screen projection processing and screen sharing processing;
step 5, each student terminal displays a sharing screen and answers the screen;
step 6: the teacher terminal is embedded with file service;
and 7: and (5) carrying out screen control processing on the teacher terminal.
6. The multi-terminal on-screen teaching method according to claim 5, wherein the step 1 specifically comprises: after the accelerator is started, generating a two-dimensional code containing accelerator IP information for displaying, fixedly monitoring two service ports, waiting for the connection between a teacher terminal and each student terminal, and entering a ready state; the screen projection port is used for displaying screen projection contents of the teacher terminal, and the split screen port is used for sharing screen contents of the accelerator.
7. The multi-terminal on-screen teaching method according to claim 5, wherein the step 2 is specifically: the teacher terminal obtains accelerator IP information by scanning the two-dimensional code and then is connected to a screen projection port of the accelerator, meanwhile, the teacher terminal pushes lesson opening information containing the accelerator IP information to each student terminal through a background server, the student terminals are automatically connected to a screen splitting port of the accelerator, the teacher terminal and each student terminal report terminal equipment information to the accelerator, and the accelerator is associated with a communication channel thereof according to the collected equipment information to establish session management; the teacher terminal can inquire and manage the session information of the student terminals through the accelerator, the teacher terminal and the student terminals both comprise disconnection reconnection modules, and when the network is unstable and disconnection occurs, the teacher terminal and the student terminals can be automatically reconnected.
8. The multi-terminal on-screen teaching method according to claim 6, wherein step 3 specifically comprises: after the teacher terminal is connected to the accelerator, starting screen recording, wherein the recorded content comprises audio and video, the audio adopts AAC coding, and the video adopts H264 coding; the teacher terminal uses the SurfaceView as a secondary dump to realize effective frame frequency control, so that the frame frequency can be adjusted at will within the allowable range of hardware performance, and the coding and transmission related parameters of the frame frequency can be adjusted.
9. The multi-terminal on-screen teaching method according to claim 6, wherein step 4 specifically comprises:
after receiving screen projection content sent by the teacher terminal, the accelerator decodes the screen projection content and outputs the decoded screen projection content to a large-screen display device for display; on the other hand, the screen of the accelerator is recorded and encoded in the background, and is shared and sent to each student terminal;
the screen of the accelerator is recorded, recoded and then encoded, and is not encoded and shared after screen projection content sent by a teacher terminal is decoded, so that the situation that the desktop of the accelerator is displayed to a student terminal or even a teacher terminal in a classroom teaching scene is considered, and at the moment, the screen of the accelerator is recorded;
the coding quality of the accelerator end recording screen is determined by setting relevant parameters on the same-screen software.
10. The multi-terminal on-screen teaching method according to claim 6, wherein step 5 specifically comprises:
the student terminal decodes the video data and the audio data received in real time and plays the video data and the audio data, and the playing of the video stream and the playing of the audio stream can be matched according to time characteristics;
in the process of on-screen, an accelerator detects the connection state of a teacher terminal and each student terminal and the accelerator through heartbeat in real time, when the student terminals are disconnected with the student terminals, inactive student terminal channels are closed, resources occupied by the student terminals are released, the student terminals also detect the connection state between the student terminals and the accelerator, when a disconnection condition occurs, the student terminals close the channels, reconnection operation is initiated after a specified time interval until the connection is successful, after the reconnection is successful, the accelerator sends recently reserved key frame information to the student terminals, and then sequentially sends real-time coding information to the student terminals, so that the latest on-screen content can be quickly obtained after the student terminals are disconnected and recovered;
the teacher terminal controls the student terminal to open or close gesture detection through the extension instruction, and after the teacher terminal puts in the large screen, students answer the student terminal, and the answering process is directly put in and displayed on the large screen, and the instruction is mutually transmitted between the teacher terminal and the student terminal through an accelerator or not through the accelerator, and the specific instruction can be extended and added at any time according to business requirements.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811023491.6A CN110874959B (en) | 2018-09-03 | 2018-09-03 | Multi-terminal same-screen teaching system and teaching method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811023491.6A CN110874959B (en) | 2018-09-03 | 2018-09-03 | Multi-terminal same-screen teaching system and teaching method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110874959A true CN110874959A (en) | 2020-03-10 |
| CN110874959B CN110874959B (en) | 2021-11-05 |
Family
ID=69716925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811023491.6A Active CN110874959B (en) | 2018-09-03 | 2018-09-03 | Multi-terminal same-screen teaching system and teaching method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110874959B (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111897504A (en) * | 2020-07-14 | 2020-11-06 | 西安万像电子科技有限公司 | Data display method and device, computer readable storage medium and processor |
| CN111966316A (en) * | 2020-08-25 | 2020-11-20 | 西安万像电子科技有限公司 | Image data display method and device and image data display system |
| CN112306441A (en) * | 2020-11-10 | 2021-02-02 | 深圳市朗科科技股份有限公司 | Method and equipment for realizing same-screen transmission |
| CN112486450A (en) * | 2021-02-04 | 2021-03-12 | 江苏北弓智能科技有限公司 | Method and device for mobile cloud desktop on-screen interaction |
| CN113050826A (en) * | 2021-03-10 | 2021-06-29 | 深圳市绘王动漫科技有限公司 | Digital screen connected with cloud |
| CN113190270A (en) * | 2021-02-26 | 2021-07-30 | 深圳市捷视飞通科技股份有限公司 | Network course control system, method, computer device and storage medium |
| CN113419700A (en) * | 2021-07-16 | 2021-09-21 | 广州然也教育科技有限公司 | Semi-screen projection system and method based on classroom teaching |
| CN113612858A (en) * | 2021-09-08 | 2021-11-05 | 深圳市乐橙互联有限公司 | Multi-terminal cooperation system and cooperation method |
| CN113934396A (en) * | 2021-11-03 | 2022-01-14 | 上海商米科技集团股份有限公司 | On-screen display method, device and system |
| CN114143477A (en) * | 2021-12-06 | 2022-03-04 | 北京淳中科技股份有限公司 | Image annotation method and device |
| CN114419939A (en) * | 2021-06-08 | 2022-04-29 | 宁波升维信息技术有限公司 | Group screen projection display system for teaching |
| CN115134944A (en) * | 2021-03-26 | 2022-09-30 | 精工爱普生株式会社 | Communication connection method, information processing apparatus, and communication connection system |
| WO2023016425A1 (en) * | 2021-08-12 | 2023-02-16 | 广州视源电子科技股份有限公司 | Multi-screen interaction system and multi-screen interaction device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012138130A2 (en) * | 2011-04-05 | 2012-10-11 | 삼성에스디에스 주식회사 | Real-time lecture support system and method |
| CN104346964A (en) * | 2013-07-30 | 2015-02-11 | 上海好记星数码科技有限公司 | Real-time question answering method based on mobile and intelligent electronic instruments |
| CN105118347A (en) * | 2015-09-06 | 2015-12-02 | 苏州佳世达光电有限公司 | Teaching interactive projection system |
| CN105405325A (en) * | 2015-12-22 | 2016-03-16 | 深圳市时尚德源文化传播有限公司 | Network teaching method and system |
| CN105681885A (en) * | 2016-02-26 | 2016-06-15 | 杭州开迅科技有限公司 | Mobile terminal screen recording and live broadcasting device and method |
| CN207458375U (en) * | 2017-09-07 | 2018-06-05 | 北京乐易考教育科技集团有限公司 | A kind of wisdom tutoring system |
-
2018
- 2018-09-03 CN CN201811023491.6A patent/CN110874959B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012138130A2 (en) * | 2011-04-05 | 2012-10-11 | 삼성에스디에스 주식회사 | Real-time lecture support system and method |
| CN104346964A (en) * | 2013-07-30 | 2015-02-11 | 上海好记星数码科技有限公司 | Real-time question answering method based on mobile and intelligent electronic instruments |
| CN105118347A (en) * | 2015-09-06 | 2015-12-02 | 苏州佳世达光电有限公司 | Teaching interactive projection system |
| CN105405325A (en) * | 2015-12-22 | 2016-03-16 | 深圳市时尚德源文化传播有限公司 | Network teaching method and system |
| CN105681885A (en) * | 2016-02-26 | 2016-06-15 | 杭州开迅科技有限公司 | Mobile terminal screen recording and live broadcasting device and method |
| CN207458375U (en) * | 2017-09-07 | 2018-06-05 | 北京乐易考教育科技集团有限公司 | A kind of wisdom tutoring system |
Non-Patent Citations (1)
| Title |
|---|
| 林原、徐亚军: "一种为同屏显示设计的图像传输算法", 《电子技术与软件工程》 * |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111897504B (en) * | 2020-07-14 | 2024-03-12 | 西安万像电子科技有限公司 | Data display method and device, computer readable storage medium and processor |
| CN111897504A (en) * | 2020-07-14 | 2020-11-06 | 西安万像电子科技有限公司 | Data display method and device, computer readable storage medium and processor |
| CN111966316A (en) * | 2020-08-25 | 2020-11-20 | 西安万像电子科技有限公司 | Image data display method and device and image data display system |
| CN111966316B (en) * | 2020-08-25 | 2023-08-25 | 西安万像电子科技有限公司 | Image data display method and device and image data display system |
| CN112306441A (en) * | 2020-11-10 | 2021-02-02 | 深圳市朗科科技股份有限公司 | Method and equipment for realizing same-screen transmission |
| CN112486450A (en) * | 2021-02-04 | 2021-03-12 | 江苏北弓智能科技有限公司 | Method and device for mobile cloud desktop on-screen interaction |
| CN113190270B (en) * | 2021-02-26 | 2023-11-07 | 深圳市捷视飞通科技股份有限公司 | Network course control system, method, computer device and storage medium |
| CN113190270A (en) * | 2021-02-26 | 2021-07-30 | 深圳市捷视飞通科技股份有限公司 | Network course control system, method, computer device and storage medium |
| CN113050826A (en) * | 2021-03-10 | 2021-06-29 | 深圳市绘王动漫科技有限公司 | Digital screen connected with cloud |
| CN115134944A (en) * | 2021-03-26 | 2022-09-30 | 精工爱普生株式会社 | Communication connection method, information processing apparatus, and communication connection system |
| CN114419939A (en) * | 2021-06-08 | 2022-04-29 | 宁波升维信息技术有限公司 | Group screen projection display system for teaching |
| CN113419700A (en) * | 2021-07-16 | 2021-09-21 | 广州然也教育科技有限公司 | Semi-screen projection system and method based on classroom teaching |
| CN113419700B (en) * | 2021-07-16 | 2024-02-06 | 广州然也教育科技有限公司 | Semi-screen-throwing system and method based on classroom teaching |
| WO2023016425A1 (en) * | 2021-08-12 | 2023-02-16 | 广州视源电子科技股份有限公司 | Multi-screen interaction system and multi-screen interaction device |
| CN113612858A (en) * | 2021-09-08 | 2021-11-05 | 深圳市乐橙互联有限公司 | Multi-terminal cooperation system and cooperation method |
| CN113934396A (en) * | 2021-11-03 | 2022-01-14 | 上海商米科技集团股份有限公司 | On-screen display method, device and system |
| CN114143477A (en) * | 2021-12-06 | 2022-03-04 | 北京淳中科技股份有限公司 | Image annotation method and device |
| CN114143477B (en) * | 2021-12-06 | 2023-12-08 | 北京淳中科技股份有限公司 | Image labeling method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110874959B (en) | 2021-11-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110874959B (en) | Multi-terminal same-screen teaching system and teaching method | |
| CN110352451B (en) | On-screen teaching method, device, system and storage medium | |
| CN107104947B (en) | Multi-screen interaction method | |
| US8860776B2 (en) | Conference terminal, conference server, conference system and data processing method | |
| US20180293906A1 (en) | Method and system for recording and playback of web-based instructions | |
| CN105262974A (en) | Method for realizing wireless screen sharing of multiple users | |
| CN105763832A (en) | Video interaction and control method and device | |
| WO2022111421A1 (en) | Screen projection method and apparatus for application interface, device, and storage medium | |
| WO2022121302A1 (en) | Game live streaming processing method and electronic device | |
| CN110072133B (en) | On-screen display method and system | |
| CN111918080A (en) | Processing method and device for live broadcast teaching | |
| CN106933523A (en) | A kind of multi-screen interaction method and device | |
| CN113542660A (en) | Method, system and storage medium for realizing conference multi-picture high-definition display | |
| CN111385518B (en) | Video processing system, video processing method, video processing device, storage medium, and electronic device | |
| CN110769320B (en) | Method, system and device for sharing content by intelligent equipment | |
| CN111309657B (en) | Electronic equipment, control method thereof and intelligent classroom system | |
| JP2003337783A (en) | Communication system, communication device and communication method | |
| CN111356009B (en) | Audio data processing method and device, storage medium and terminal | |
| CN115022573A (en) | Desktop video conference system | |
| CN112672089B (en) | Conference control and conference participation method, conference control and conference participation device, server, terminal and storage medium | |
| CN210804824U (en) | Remote interactive teaching system with synchronous blackboard writing and live broadcasting functions | |
| CN108769565B (en) | Automatic switching method of picture layout, server and local recording and broadcasting system | |
| US6356295B1 (en) | Image transmission system | |
| CN113345281A (en) | Intelligent teaching system | |
| CN110662101A (en) | Network video multi-path display method and device based on UDP |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |