JPH04304543A - Picture processing system - Google Patents

Abstract

PURPOSE:To monitor the display state of a connected terminal equipment from an another terminal, and to transmit data to be displayed at the monitored terminal so as to be displayed, on a network. CONSTITUTION:Two kinds of terminals, those for a teacher 8 and for a student 7, are connected on a network 6. The terminal 7 transmits displayed picture data to a server 10. The server 10 stores the data as a picture file. The teacher terminal 8 reads the picture file from the server 10, and displays it. And also, the data inputted from the teacher terminal 8 are transmitted to the server 10, and stored. The student terminal 7 continues to monitor the transmission of the inputted data from the teacher terminal to the server 10, and displays the transmitted data.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention can, for example, display an image displayed on a certain terminal device connected to a communication network.
The present invention relates to image processing systems and the like that create images on other terminal devices.

0002

[Prior Art] Conventionally, in a computer-based educational system (CAI) in which student terminals and instructor terminals are arranged on a communication network, images displayed on the student terminal device are transferred to the instructor's terminal device. There was a system that displayed a reduced size on the device to make it easier for the instructor to understand the student's learning status.

0003

[Problem to be Solved by the Invention] In the above conventional example, the instructor only monitors the screen of the student's terminal, so even if he notices the student's mistake, he cannot send a message from the instructor's terminal to the student's terminal. This had the disadvantage that it was not possible to point out errors on the students' terminals and provide specific guidance.

The present invention has been made in view of the above-mentioned conventional example, and it is possible to monitor the display on another terminal from one terminal on the network and at the same time send an image to be displayed to the terminal being monitored. The purpose is to provide an image processing system that can.

[0005]

Means for Solving the Problems In order to achieve the above object, the image processing system of the present invention has the following configuration.

[0006] A first terminal device and a second terminal device have different functions.
An image processing system constructed on a network system in which one or more terminal devices are connected, wherein the first terminal device reads the image data sent from the second terminal device and displays the image data on the screen. a display means for displaying the
and a first sending means for sending input data to a second terminal device, and the second terminal device sends image data of an image being displayed to the first terminal device. a second transmitting means for transmitting the data; and a second display means for reading the input data transmitted by the first transmitting means, combining the input data with the image being displayed on the terminal device, and displaying the composite image again. has.

[0007]

[Operation] With the above configuration, it is possible to monitor the image displayed on the terminal device connected to the network, and when input is performed while viewing the image, the input data is transferred to the image being displayed on the terminal device being monitored. Superimposed display is possible.

[0008]

[Embodiment] [Embodiment 1] An embodiment of the present invention will be described below with reference to the drawings.

<Configuration of Embodiment> In this embodiment, a CAI system is realized using a local area network system (LAN) composed of a plurality of terminal devices and a network server. In this system, since the network server is also considered to be a terminal device with special functions, the terminal used by the user is called a client or client machine.

In FIG. 1, reference numeral 1 denotes the entire private telephone network used for telephone calls between teachers and students. 2 is the student phone, 3 is the teacher's phone, 4 is the telephone line, and 5 is the private branch exchange (PBX).
It is. 6 is the entire LAN used in the CAI system. 7 is the student client machine (student machine)
, 8 is a teacher's client machine (teacher machine), 9 is a LAN cable, and 10 is a network server. C
The AI classroom has many booths, each booth accommodates one student, one student machine, and one telephone for teachers to contact students. is set. During CAI classes, students sit at each booth in a predetermined order, and the teacher asks each student to
The phone number is now available.

FIG. 2 is a diagram showing the system configuration of the student machine 7, teacher machine 8, and server 10 terminal devices shown in FIG. Both are personal computers. In FIG. 2, 11 is a central processing unit (CPU) that controls the entire terminal device, 12 is a CRT display for displaying images, 13 is a keyboard for inputting character strings, and 14 is a mouse as a pointing device. . 15 is a memory for storing data and programs. The memory 15 has an operating system (
A read-only memory (ROM) 16 that contains part of the operating system (OS) and character fonts required to display characters on the CRT display 12, as well as a variable area and work area for loading and executing programs. Random access memory (RAM) 17, CRT display 12
Video RAM (VRA) stores data to be displayed on
M) Consists of 18. 19 is an external storage device that stores a large amount of programs and data in the form of files, 20
A LAN connection device 20 is connected to the LAN cable 9 and is in charge of data input/output control.

[0012] This embodiment is realized by a system having the above configuration.

<Explanation of operation> FIG. 3 shows the following steps in this embodiment:
FIG. 3 is a diagram for explaining data exchanged between a teacher and a student. The explanation will be given according to the circled numbers in the figure.

[0014]■. First, the image data that is the basis of the image displayed on the student machine 7 is sent to the server 10 and saved as an image file 70.

[0015]■. Second, the image file 70 is transferred to the teacher machine 8, and the image on the student machine 7 is displayed based on the data.

[0016]■. Third, the teacher operates the mouse and keyboard by intervening on the student's screen while looking at the screen. The mouse and keyboard data input at that time is sent to the server 10 and saved as an input data file 71.

[0017]■. Fourth, the student machine 7 is connected to the server 10
The input data file 71 stored in is fetched and displayed on the display.

By repeating these processes continuously, the teacher can intervene and give instructions to the student's screen. Students listen to the teacher on the phone and follow the teacher's instructions displayed on their computers.

During the above series of procedures, there are intermediate files stored in the server 10, and these files are as shown in FIGS. 4(a) and 4(b).

In this embodiment, these files are stored in the external storage device of the server 10. This external storage device is a disk device and is managed with a hierarchical directory structure. FIG. 4(a) shows how the image file 70 is stored. The image file 70 is "c:\cli"
＿vrm” file name in the directory “cli＿###
．． vrm”. “###” in the file name
The part indicates the number of the student machine 7 that sent the file. FIG. 4(b) shows the input data file 71
This is how it is stored. Input data file 71
is stored in the "c:\cli_cur" directory with the file name "cli_###.cur". Here, the "###" part indicates the number of the student machine 7 to which the input data file 71 is to be sent. This file is sent from the teacher machine 8 to each student machine 7, and the number part corresponds to the destination. In addition, in the configuration of many student machines and one teacher machine as in this embodiment, since the teacher can only intervene at one student at a certain time, it is preferable that the number of input data files 71 on the server 10 is one. It's normal. If there are multiple teacher machines 8, there may be multiple input data files 71.

FIG. 4(c) shows an example of the contents of the input data file, which includes the x and y coordinates of the mouse cursor, the on/off state of the mouse button, and data input from the keyboard. The data entered from the keyboard is data such as, for example, when a character string is input from the teacher machine 8, a character code string corresponding to the character string, and when an arrow key is pressed, the key code of the arrow key. In this embodiment, the input is limited to the mouse and keyboard, and is limited to the input data itself. However, in order to display it on the student machine 7 as an input data file, it is possible to send character strings and images read from the memory 15. You can also send a code that generates a tone to make the sound. Furthermore, if there is another input device, the data input from that device may be stored in the input data file 71.

<Operation of Teacher's Machine> When operating from the teacher's machine 8 in accordance with the exchange of files and data described above, the following procedure is performed.

FIG. 5(a) shows an example of a display screen on the CRT display 12 of the teacher machine 8. The screen displays of a large number of student machines 7 are scaled down and arranged by an expansion program (described later) that expands screen display data. The expansion program arranges reduced displays of the screens of each student machine 7 based on the image file on the server 10. 30 is the screen display area of the teacher machine 8, 31 is the title given to the function being executed, 32 is the reduced screen of each student machine, 33 is the number and name of the student machine, and 34 is the number pointed with the mouse of the teacher machine 8. 35 cursor indicating the point
is an end mark to abort the currently running program. When the teacher places the cursor 34 on the end mark 35 and clicks the mouse button, the development program ends. When the cursor 34 is placed on one of the reduced display screens and the mouse button is clicked, the reduced display is expanded to cover the entire screen and displayed in an enlarged manner. That is Figure 5
This is the state shown in (b).

In FIG. 5(b), the title 31 is the client machine number and student name of the student machine displaying the enlarged image, and a ``teacher intervention'' item 36 appears in the menu column. 37 is an enlarged screen display. In this case, the student named Ogawa, who is using student machine number 2, is learning about the structure of spiders and is being asked where the silk comes from. The student thinks that the spider's silk will come out from its mouth, so he places the mouse cursor 38 on the student machine 7 at the spider's mouth and is about to click the mouse button. At this point, the teacher decides to intervene in order to guide this student's learning, so she first calls telephone number 2 in the booth where the student named Ogawa is, and informs the student that she will intervene when the student answers the phone. . The teacher then positions the cursor 34 of the teacher machine 8 on the "teacher intervention" item 36 and clicks the mouse button. Then, the state shown in FIG. 5(c) is reached.

In FIG. 5(c), the title 31 displays the name of the educational software currently running on the student's machine, and the menu is a menu 39 for the educational software itself. Also,
The student's mouse cursor 38 disappears from this screen, and only the teacher's mouse cursor 34 is displayed. Furthermore, an "end intervention" menu 40 is added to the menu field for ending the teacher's intervention state. From the state shown in FIG. 5(c), the teacher points to a part of the screen with the mouse cursor 34 and uses the telephone 2 to tell the student,
The organ that makes the thread is located in the abdomen, and is excreted from a synthetic organ at the rear of the abdomen. In addition, Figure 5 (
The state c) shows the display on the teacher's machine 8, but the same screen is also displayed on the student's machine, except for the "end of intervention" menu 40. In other words, teachers and students teach and receive instruction while looking at a common screen.

In FIG. 5(c), when the teacher places the mouse cursor 34 on the "end intervention" menu 40 and clicks the mouse button, the intervention state ends and the state returns to the state shown in FIG. 5(b). Furthermore, when the teacher places the mouse cursor 34 on the end mark 35 in FIG. 5(b) and clicks the mouse button, the enlarged display ends and the display returns to the display in which the reduced screens are lined up as shown in FIG. 5(a). By the way, in the state shown in FIG. 5(c), the teacher moves the teacher's mouse cursor 34 to "File", "Edit", "
When the mouse button is clicked on one of the menus 39 such as "Display Format", the educational software on the student machine 7 executes the process corresponding to that menu. This means that exactly the same processing is executed on the student machine 7 as when the student operates the educational software alone.

Furthermore, if the teacher moves the mouse cursor 34 to the end mark 35 and clicks the mouse button from the state shown in FIG. The screen will display the OS prompt. At the same time, the teacher machine 8 also ends its state of intervening with the student and returns to the state shown in FIG. 5(b). However, the screen display 37 displays the prompt screen of the OS on the student machine, and the "Teacher Intervention" 36 menu is no longer displayed.

FIGS. 6A, 6B, and 6C show memory maps of the RAM 17 of the server 10, student machine 7, and teacher machine 8, respectively, while they are in operation. Figure 6 (
a) is a memory map of the server 10; Reference numeral 50 is a server OS that processes basic input/output control of the server and provides a multitasking environment. This also applies to client machines. A server-side LAN manager 51 performs various network processing such as LAN user management, data transfer, and security management. 52 is OS5
0 and the work area of the server-side LAN manager 51.

FIG. 6(b) is a memory map of the student machine 7. 55 is an OS for the client machine, which performs basic input/output control of the client machine, and 56 is a LAN manager on the client side, which controls the user's (student) L
It performs the login process to the AN and controls the exchange of data and messages between the client and the server. Reference numeral 57 is a screen saving program that sends the image data in the VRAM 18 of the student machine 7 to the server 10, converts it into a file, and saves it every 0.5 seconds. Reference numeral 58 is an input monitoring program, and when the teacher starts to intervene from the teacher machine 8 and the input data file addressed to his/her own machine continues to be saved on the server 10, it monitors the mouse cursor coordinates and mouse button on/off of that file.・This is a program that imports keyboard input data, etc., assuming that it is the same as data from the mouse and keyboard of your own machine. At this time, input from your own mouse and keyboard is ignored. 59 is the educational software that the students are learning. 60 is 55, 56, 57, 58, 59
This is the work area of each program.

FIG. 6(c) is a memory map on the teacher machine 8. 65 is the OS for the client machine, and 66 is the LAN manager on the client side, as shown in FIG. 6(b).
It performs the same processing as the OS 55/LAN manager 56 shown in . Reference numeral 67 is an expansion program that reduces and displays the screens of each student machine on the screen of the teacher machine 8 in a reduced size based on the screen display data in the image file of the student machine 7 stored on the server 10 by the screen save program 57. It has the function of Furthermore, according to the teacher's instructions, the screen of a specific student can be enlarged and displayed on the teacher's machine display, and the teacher can also intervene in the student's learning work for the purpose of guidance. The intervention referred to here means that the teacher machine 8
The process involves transferring input data, etc. to the server 10 and storing it as a file. And the above-mentioned Figure 6 (
As explained in b), the input monitoring program 58 of the student machine 7 takes in the data of the input data file as the same as the data from the student's own mouse and keyboard, and processes the educational software 59 using the data as input. 69 is a work area for each program 65, 66, 67, and 68. Note that the programs on each machine are executed in a multitasking environment, so that while one program is running on the same machine, another program is running in parallel.

With the configuration and operation described above, the teacher can not only monitor the student's learning situation but also give appropriate instructions to the student.

<Description of Flowchart> Next, the operation of the CPU 11 by each program will be explained according to the flowchart.

FIG. 7 is a flowchart of the screen storage program 57 of the student machine 7. The screen saving program is
As soon as the educational program is started, execution begins and the information on the display screen is sent to the server. The screen displayed for the student does not have the "end intervention" item in the menu column of FIG. 5(c).

[0034] Once the screen saving program has been started, '
Test whether “end mark” is selected (S1)
. If selected, immediately stops running the program and exits. Otherwise, the image data stored in the VRAM 18 is transferred to the server 10 (S2). Basically, the above steps are repeated, but image data is transferred at fixed time intervals. In this example, it is set to 0.5 seconds (S3), but it is necessary to consider the value depending on the number of terminal devices connected to the network and the performance of each terminal device.

The student machine 7 only processes this, but the server 10 that receives the image data is also compatible with this process. The server 10 stores the received image data as an image file 70 together with the sender's client number. Since data is sent one after another from the same client, old files are updated one after another with new data.

FIG. 8 is a flowchart of the expansion program 67 executed on the teacher machine 8. For the sake of explanation, several areas are defined in the work area 69. first,
"m" and "n" are taken as variables. Both are used for loop control. A "mode indicator" indicating the teacher's operation mode is also secured in the same work area. The modes include "reduction mode" which displays the student's screen in a reduced size, "enlargement mode" which selects and enlarges a specific student machine, and "intervention mode" which is included in enlargement mode but allows intervention with the student.
There is.

The expansion program is executed according to instructions from the operator, so if there is input, processing is performed according to the input data. First, it is tested whether a termination instruction has been given (S10). If it is finished, execution is stopped and the expansion program is finished. If not, the mode is tested to see if enlarged display is in progress (S11). If the display is not being enlarged, a test is made to see if an instruction to enlarge the display has been input (S12). If it was an instruction to enlarge the display,
Since the instruction also specifies the screen to be enlarged, that screen is enlarged and displayed (S13). Once enlarged, repeat from step 10. If the instruction is not to enlarge, the display for each student on the teacher machine 8 is updated. Therefore,
The number of currently connected student machines is set in the variable m, and 1 is set as the initial value in the variable n (S14). After completing the initial settings of the loop, execute the loop. The image file of client n stored in the server is read, and the display of student machine number n is updated based on the image data (S15). Add the value of n by 1 (S
16) Do this for all m terminals. When all m units are completed, repeat from step 10 (S17).

On the other hand, if it is determined in step 11 that the display is being enlarged, the operation in that mode is tested and processing is performed. First, test whether it is in "intervention mode" (S18
). If it is "intervention mode", it is tested whether "intervention end" is selected as an input (S19). If it is finished, the display of "intervention mode" shown in FIG. 5(c) is changed to the display of FIG. 5(b), which is a simple enlargement mode, and the execution of the input data storage program is stopped (S20). If the input is not "end of intervention", the input is an intervention for the student, and the input data is sent to the server by the activated input data storage program (S21). If the judgment in step 18 is that it is not "intervention mode", test whether the input data is "end mark" (
S22). If so, the enlargement mode ends and the display of the student machine returns to the reduced size and tiled display. For this purpose, the mode is set to "reduction mode" (S23), and step S14
Performs a reduced display from. If the input is not terminating, it is tested whether the mode change to "intervention mode" is selected (S24
). If so, change the mode to "intervention mode" and make the display as shown in Figure 5(c), and start the input data storage program to send subsequent input to the student machine (S25).
. This input has no particular meaning unless the mode is changed, and it is discarded and waits for the next input. As shown in this flowchart, there is an input data storage program as a secondary program used in the expansion program.

FIG. 9 is a flowchart of the input data storage program 68 on the teacher machine 8. This program is launched from the deployment program and begins execution. During execution, it waits for an instruction to stop it (S30), and if it is an instruction to stop it, it requests the server to delete the input data file stored in the server 10, and after it is deleted, it stops and ends (S31). Other input data is sent to the server 10 (S32) and saved as an input data file by the server. The timing of input data transmission may be set at fixed intervals, or, although not described in the embodiment, a "data transmission" item may be provided in the menu during execution of the expansion program to provide operational instructions.

The input data file created in the server 10 in this manner is taken into the student machine 7 and displayed. A monitoring program makes this possible. Figure 10
is a flowchart of the monitoring program 58 on the student machine 7.

[0041] The monitoring program starts running at the same time as the student's educational software is started. As already described, while displaying intervention data from the teacher machine, the machine's own input is discarded. This is done by testing the mode shared with the monitoring program in the input data processing of the educational software. This mode is referred to herein as "ignore mode," and the mode is set by the monitoring program. In educational software,
In "ignore mode", input from input devices such as the keyboard and mouse is discarded and ignored.

[0042] The executed monitoring program tests whether an instruction to end by selecting the "end mark" has been given (
S40). If it is finished, the "ignore mode" is canceled and the execution is finished (S41). If not, test whether the server 10 has an input data file 71 for the student machine running the program. Otherwise, there is no data from the teacher machine to display, so "ignore mode"
is canceled and the process returns from step S40 (S43).
. If there is an input data file to be displayed, enter "ignore mode" and read that file (S44
). The loaded file is opened and its contents are examined (S45), and if the contents are the same as the latest processed input data file, the loaded data is discarded as meaningless and the process returns to step S40. If the data is different, the data is displayed as if it were input from a keyboard or mouse (S46).

As described above, the existence of an input storage program, a monitoring program, and a server that supports the input data files handled by them makes it possible for teachers to give instructions to students through terminal devices. There is.

[0044]

[Other Examples] [Example 2] In Example 1, the local telephone network and LAN were completely independent networks, but by providing a communication device between the network server and PBX, it was possible to Naturally, it is possible to realize a system that can automatically call a target telephone from a teacher's or student's terminal device.

[Embodiment 3] In Embodiment 1, the explanation has been made assuming that there is only one teacher, but of course the same system can handle even if the number of teachers increases to two or three.

The present invention may be applied to a system made up of a plurality of devices, or to a device made up of one device. It goes without saying that the present invention can also be applied to cases where the present invention is achieved by supplying a program to a system or device.

[0047]

As explained above, the image processing system according to the present invention allows one terminal on a network to monitor the display on another terminal, and at the same time sends an image to be displayed to the monitored terminal. You can attach it.

[Brief explanation of drawings]

FIG. 1 is a system configuration diagram of an embodiment.

FIG. 2 is a system configuration diagram of a personal computer according to an embodiment.

FIG. 3 is an explanatory diagram of data exchange between a server, a student client machine, and a teacher client machine.

FIG. 4 is an explanatory diagram of a directory data file in the server.

[Figure 5-1]

[Figure 5-2]

FIG. 5-3 is an example of a screen display of a teacher's client machine.

[Figure 6-1]

[Figure 6-2]

FIG. 6-3 is a memory map of the server, student client machine, and teacher client machine.

FIG. 7 is a flowchart of a screen saving program.

[Figure 8-1]

FIG. 8-2 is a flowchart of an expansion program.

FIG. 9 is a flowchart of an input storage program.

FIG. 10 is a flowchart of a monitoring program.

[Explanation of symbols]

1... Local telephone network 2... Student telephone 3... Teacher's telephone 4... Telephone line 5... PBX 6... LAN 7... Student client machine 8... Teacher client machine 9... LAN cable 10... Server 11... CPU 12 …CRT display 13…Keyboard 14…Mouse 15…Memory 16…ROM 17…RAM 18…VRAM 19…External storage device 20…LAN connection device 30…Screen frame 31…Title 32…Reduced display 33…Client machine number and student name 34...Teacher's mouse cursor 35...End mark 36..."Teacher intervention" menu 37...Enlarged display 38...Student mouse cursor 39...Educational software menu 40..."Intervention end" menu 50...Server OS 51...Server side LAN Manager 52...Work area 55...Client side OS 56...Client side LAN manager 57...Screen saving program 58...Monitoring program 59...Educational software 66...Work area 65...Client side OS 60...Client side LAN manager 67...Deployment program 68... Input save program 69...work area

Claims (1)

[Claims] 1. An image processing system constructed on a network system in which one or more first terminal devices and second terminal devices having different functions are connected, wherein the first terminal device The device includes a display unit that displays image data sent from the second terminal device on a read screen, and a first device that sends input data to the second terminal device.
The second terminal device has a second sending device that sends the image data of the image being displayed to the first terminal device, and an input device sent by the first sending device. An image processing system comprising: a second display unit that reads data, combines the input data with an image being displayed on the terminal device, and redisplays the composite.