JP2009054134A - Computer, overwrite information processor, overwrite information processing method, program, storage medium, and teleconference method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology for connecting overwriting to a conference material position in a teleconference. <P>SOLUTION: This overwrite information processor performing the application for displaying a document is provided with a coordinate specifying means for specifying a coordinate in a document in a document display area, an input detection means for detecting an overwritten drawning in the document display area, and a storage control means for storing the coordinate and the drawing by associating them with each other. A display screen acquiring means has a document position information detection means for detecting the number of scroll lines and the length of scroll line in the document displayed by the program of the application by moving a scroll box. A teleconference system is provided with a position aligning means for aligning positions of the conference information prepared by a PC terminal unit by associating positional relation of the conference information displayed and output into a conference information display means and the information overwritten on the conference information with each other. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  In the present invention, when overwriting is performed on a display screen of an arbitrary application operating on a general-purpose OS (operating system) such as Windows (registered trademark), the overwritten drawing is associated with the display state of the application. In particular, it is possible to detect the scroll line number and scroll line length of a display document (image) displayed by an arbitrary application program by moving the scroll box. The present invention relates to a computer, an overwriting information processing apparatus, an overwriting information processing method, an electronic conference method, a program, and a storage medium.

In recent years, the number of methods for projecting digitized documents with projectors has been increasing in office presentations, and on the presentation device, the image being projected is written with a dedicated pen and the locus of the dedicated pen is detected. Electronic products that are stored electronically with the image being projected are also commercialized.
As a prior art related to the presentation device according to the present invention, as Patent Document 1, an information display medium has an image display surface that can rewrite information, and there is a peripheral portion of the image display surface. Identification information of the information display medium is recorded, and an x and y coordinate information layer indicating coordinate information on the image display surface is formed below the image display surface. Detecting identification information for identifying a display medium, and having a light emitting and receiving unit for detecting the coordinate information, the detected identification information and the coordinate information are processed in association with each other, stored in a memory, and transmitted to a personal computer Techniques to do this are disclosed.
As Patent Document 2, a handwritten writing information written on a writing unit of a white plate unit is scanned by a scanner unit, a written image is digitized, and a simple image that is combined with or associated with an image projected by a projector unit An inexpensive presentation equipment technology with a structure is disclosed.

As Patent Document 3, a document is analyzed, a character information generation unit that generates character information including information on the position, character size, and character type of each character, and character information on each line, main characters on each line are analyzed. Document structure information generation is provided by a line information generation unit that generates line information including character size, main character type, and score information of each line, and a document structure information generation unit that analyzes the line information and generates document structure information. Discloses a technique for grouping lines based on the score of line information and line continuity to generate document structure information.
As Patent Literature 4, the overwrite data to be displayed on the display document displayed on the display screen and the document coordinates indicating the position where the overwrite data is displayed as a relative position on the display document are stored in association with each other. Overwrite data storage means, document information detection means for detecting the display magnification of the display document and screen coordinates on the display screen, overwrite data using the display magnification of the display document and the screen coordinates of the display document A technique is disclosed that includes coordinate conversion means for converting document coordinates of overwrite data stored in the storage means into screen coordinates.

However, the above prior art has the following problems.
That is, in Patent Document 1, a technique for adding page information to a document for printing, automatically detecting page information, and overwriting and page alignment is known. A special device for identifying was required.
Further, in Patent Document 2, a method of separating and storing a PC image and handwritten information at the time of overwriting and displaying them in a superimposed manner is known, but the positional relationship between the image and the overwriting is unknown, and the display position of the image and the overwriting are overwritten. It was necessary to associate and manage.
In Japanese Patent Application Laid-Open No. 2003-228561, a method for calculating the position of a document by analyzing the character size and character type of the document is known, but the analysis takes an enormous amount of time.
Further, in a PC conference in which a plurality of PCs are connected to a network and common conference materials are displayed on all the PCs, explanation portions are marked for smooth progress of the conference, or characters are drawn for supplementary explanation. As a conventional technique related to this, a method is disclosed in which the server manages the window display state in the client-server method and unifies the size of the client window when there is a change (see, for example, Patent Document 4). ). In addition, a method of overwriting electronic mail, transmitting separately from electronic mail, and displaying electronic mail and overwriting is disclosed (for example, see Patent Document 5).
Further, a method of instructing by a button or key input for unifying the window position (display position, width, height) is disclosed (for example, see Patent Document 6). Further, it is disclosed that the mouse operation is invalidated by displaying a full-screen transparent window, and the screen is unified by file transfer and file display (see, for example, Patent Document 7).

However, the method of Patent Document 4 requires a server for managing the window display state, and the display position is not managed. Further, in the case of Patent Document 5, it is a condition that the display state of the electronic mail at the time of display is the same as that at the time of overwriting. Further, in the method of Patent Document 6, when the content displayed in the window is updated, the display content cannot be updated. In the case of Patent Document 7, the conference material needs to be filed in units of pages.
Furthermore, a document is created faster than handwriting by a document creation program such as Microsoft Word (registered trademark), and has superiority to handwriting from reuse, Internet distribution, and the like. However, since everything is created by operating the mouse and keyboard, the expressive power is limited and the expressive power is inferior to handwriting.
If the position information of the displayed document can be acquired, characters and figures can be drawn on the document by handwriting on the tablet or Tablet PC assuming a pen input operation, and the document and handwritten characters can be associated, saved, displayed, edited, etc. It becomes possible to make use of the advantages of both. However, only a few such as Microsoft Office (registered trademark) provide the position information of the display document necessary for this purpose through the API.

The program generally has a scroll bar as a tool for displaying the target document, and the scroll display is performed by dividing the document into line units, and is reflected in the scroll box position.
However, this line length is set to a different value for each program and cannot be used easily. That is, some programs (such as Microsoft Excel (registered trademark)) have different movement amounts (scroll line length) for each scroll. In this case, the document position is simply specified from the scroll line length and scroll box position information. It was something that could not be done.
Also, in the conventional technology, when the document is scrolled and the page is changed, this overwriting has no meaning and will be erased. However, when the overwritten page is displayed again, the previous overwriting is performed. Most of the time. Further, when overwriting is performed on a document, it is not possible to understand the meaning even if only overwriting is stored and displayed, so it is necessary to store the overwriting associated with the overwritten document. This cannot be associated unless position information such as the page number and line number of the display document is known. There are programs that provide location information for this document, but most programs do not.

Further, according to Patent Document 8, the overwrite position information is retained and displayed at the time of enlargement / reduction display and scroll display. However, once the display of the application (APL) is changed, it disappears and the same image is redisplayed by APL. In addition, it does not have information such as the display position and display magnification, and the overwrite data cannot be aligned and displayed.
Further, when the conference material is scrolled and the page is changed, the relationship between the display information and the overwrite information is lost and has no meaning, so the overwrite data is saved and deleted. When a conference material that has been overwritten again is displayed, in order to display this overwriting, it is necessary to know where the conference material was overwritten and where it is currently displayed. There is a program (Microsoft Office) that provides an API for acquiring a page number, a display position, a display magnification, and the like during display, but almost no program provides this function.
In addition, the overwriting has a meaning for the conference material at the time of overwriting, and it becomes an obstacle when the display change (scrolling, page turning) of the conference material is performed. For this reason, when there is no need to redisplay, it is necessary to delete, and when necessary, a storage operation is required.
JP 2000-215000 A JP 2001-016384 A JP 2001-265762 A JP 2004-094679 A JP-A-8-185302 JP 11-143794 A JP 2000-259535 A JP 2003-91490 A JP 2001-31264 A

  The present invention has been made in view of the above-described conventional problems. As a first object, the present invention provides a document position detection function capable of detecting document position information even in a program that does not provide position information. It is intended to provide. A second object of the present invention is to associate overwriting and meeting material position in an electronic conference. A third object of the present invention is to eliminate the need to save overwrite information for conference information in an electronic conference.

In order to achieve the above object, according to the first aspect of the present invention, in a computer, the number of scroll lines and the scroll line length of a display document displayed by an arbitrary application program operating on the operating system are detected by moving the scroll box. And document position information detecting means.
The invention described in claim 2 is an overwriting information processing apparatus in which an application for displaying a document is executed, and a display screen acquisition means for acquiring information on a display screen on which the application is executed, From the display screen on which the application is executed, coordinate specifying means for specifying coordinates in the document in the document display area displayed on the display screen, input detection means for detecting overwriting in the document display area, Storage control means for associating and storing the coordinates and the drawing, and the display screen acquisition means detects the number of scroll lines and the scroll line length of the display document displayed by the application program by moving the scroll box. Document position information detecting means is provided.

According to a third aspect of the present invention, the document position information detecting means draws a straight line in the document area, detects the amount of movement of the drawing line due to the scroll box movement, and detects the scroll line length. .
According to a fourth aspect of the present invention, the document position information detecting means extracts a specific image of a document area, detects the amount of movement of the specific image due to scroll box movement, and detects the scroll line length. To do.
According to a fifth aspect of the present invention, the document position information detecting means acquires an image of a document display area, displays the acquired image so as to overlap the document area, and executes scroll box movement. .
The invention according to claim 6 is characterized in that the document position information detecting means detects a scroll line length of the document and detects line coordinates of all lines of the document.
The invention according to claim 7 is characterized in that the document position information detecting means detects a current line number by moving the scroll box until the scroll box position changes.

The invention according to claim 8 is characterized in that the document position information detecting means displays a transparent window on the document and scrolls the document by moving a scroll box.
According to a ninth aspect of the present invention, the document position information detecting means detects scroll information for each program, stores it, automatically recognizes a program for displaying a document, and eliminates the need to detect scroll information. Features.
According to a tenth aspect of the present invention, the document position information detecting unit detects and stores the number of scroll lines and scroll line coordinates of the document in advance and does not need to detect the number of scroll lines and line coordinates of the document when performing position detection. It is characterized by.
The invention according to claim 11 is characterized in that the coordinate specifying means has a function of specifying a drawing corresponding to a document display area on a display screen based on the coordinates stored in the storage means. .
The invention according to claim 12 is an overwriting information processing method for executing an application for displaying a document, the step of acquiring information on a display screen on which the application is executed by a display screen acquiring unit, and coordinates A step of specifying coordinates in the document in the document display area displayed on the display screen from a display screen on which the application is executed by the specifying means; and a step of detecting drawing on the document display area by the input detection means And storing the coordinates and the drawing in association with each other by the storage control means, and acquiring the information on the display screen includes the number of scroll lines of the display document displayed by the program of the application, scrolling Document position information for detecting line length by scroll box movement And having a output step.

According to a thirteenth aspect of the invention, when executing an application for displaying a document, a step of acquiring information on a display screen on which the application is executed by a display screen acquiring unit, and a step of executing the application by a coordinate specifying unit Specifying the coordinates in the document of the document display area displayed on the display screen from the displayed display screen, detecting the drawing in the document display area by the input detection means, and the storage control means Storing the coordinates and the drawing in association with each other, and the step of acquiring the information on the display screen includes moving the scroll box with the number of scroll lines and the scroll line length of the display document displayed by the program of the application. A document position information detecting step detected by It characterized a program for realizing the overwriting information processing method comprising the computer.
According to the fourteenth aspect of the invention, when executing an application for displaying a document, a step of acquiring information on a display screen on which the application is executed by a display screen acquiring unit, and a step of executing the application by a coordinate specifying unit Specifying the coordinates in the document of the document display area displayed on the display screen from the displayed display screen, detecting the drawing in the document display area by the input detection means, and the storage control means Storing the coordinates and the drawing in association with each other, and the step of acquiring the information on the display screen includes moving the scroll box with the number of scroll lines and the scroll line length of the display document displayed by the program of the application. A document position information detecting step detected by Characterized in that it is a storage medium storing a program for realizing the overwrite processing method by a computer equipped.

The invention according to claim 15 connects a plurality of PC terminals as computer apparatuses each having a conference information display means on a network as PC conference participation terminals, and one of the plurality of PC terminals is connected. In an electronic conference method for performing an electronic conference by using a PC conference operation right terminal as needed and outputting a display document as conference information displayed on the PC conference operation right terminal and its overwriting information on each of the other PC terminals. The document position information detection unit of the PC terminal displays the position of the display document displayed on the conference information display unit by an arbitrary application program operating on an operating system. Horizontal scroll bar information, vertical scroll bar information, display window information The number of scroll lines and scroll line length of the displayed document. Detected by movement of the crawl box and sent out, and the position matching means of the other PC terminal detects the received horizontal scroll bar information, vertical scroll bar information, display window information, and movement of the scroll box. Corresponding to the received information, the display state of the display document as the conference information displayed on the conference information display means of the own device is set to the same display state as the PC terminal of the transmission source.
According to a sixteenth aspect of the present invention, a plurality of PC terminals as computer devices each having a conference information display means are connected as a PC conference participation terminal on a network, and one of the plurality of PC terminals is connected. In an electronic conference method in which a PC conference operation right terminal is used at any time, and a display document as conference information displayed on the PC conference operation right terminal and its overwriting information are output to a screen on each of the other PC terminals and an electronic conference is performed. The document position information detection means of the PC terminal detects a change in the position and size of a work window in which an arbitrary application program operating on an operating system displays the display document on the conference information display means, and the display The number of scroll lines and the scroll line length of the document are detected by moving the scroll box and sent. Then, the position matching means of the PC terminal displays the display state of the work window displaying the display document as the meeting information displayed on the meeting information display means of the own machine in correspondence with the received information. The display state is the same as that of the transmission source PC terminal.

Further, the invention according to claim 17 is the electronic conference method according to claim 15 or 16, wherein the document position information detecting means of the PC terminal is used as conference information displayed by an arbitrary application program operating on the operating system. When a change in the display document to another display document is detected, the PC conference participation terminal acquires the changed file of the other display document as the conference information and re-sets it to the same state correspondingly. It is characterized by displaying.
The invention according to claim 18 is the electronic conferencing method according to any one of claims 15 to 17, wherein the document position information detecting means of the PC terminal displays an arbitrary application program operating on an operating system. When overwriting a displayed document as conference information to be performed, the coordinates on the screen for overwriting operations such as overwriting characters are acquired, the coordinates on the screen are converted into document coordinates, and the scrolling of the displayed document is performed. The number of lines and the scroll line length are detected by the movement of a scroll box, transmitted to the PC conference participation terminal, and the document coordinates are stored in the own machine. The overwritten contents are drawn with the coordinates on the screen on the display document as the meeting information displayed above, and the document is displayed on the document material. While overwriting the same, the scroll box in correspondence with the detected information by moving in the same display state, and characterized in that it saved by converting the screen coordinates to document coordinates.

Further, the invention according to claim 19 is the electronic conference method according to claim 18, wherein the overwriting information processing apparatus is configured such that the overwritten content of the displayed document displayed on the meeting information display means is all or part of the information. If deleted, obtain the coordinates on the screen of the overwrite deletion operation, transmit to the PC conference participation terminal, and delete the deleted document coordinates from the document coordinates stored in the own machine, The document position information detecting means of the PC terminal erases the overwritten character by the screen coordinate, makes the overwritten content on the document material the same, and calculates the document coordinate of the overwritten character erased based on the screen coordinate. The deleted document coordinates are deleted from the stored document coordinates.
According to a twentieth aspect of the present invention, in the electronic conference method according to the eighteenth or nineteenth aspect, when the display document is changed, the display area document space being displayed is calculated, and the saved document coordinates are displayed on the screen according to the calculation result. The overwriting content on the conference material is made the same by converting to coordinates and redrawing.
According to a twenty-first aspect of the present invention, in the electronic conference method according to any one of the eighteenth to twentieth aspects, the overwriting information processing apparatus displays a display document file that has been overwritten as necessary. The overwriting information is redrawn from the stored overwriting information to the position at the time of overwriting, and thereafter, editing processing such as addition and deletion of the overwriting can be performed.

According to the present invention, document position information can be detected even in a program that does not provide position information.
As a result, it becomes possible to detect the position information of the document displayed by an arbitrary program that does not have an API that provides the position information of the display document, and the handwriting input to the electronic document is associated with the document position. Efficiency can be used for electronic data to improve the efficiency of many operations. (Example: When instructing to modify a created electronic document, modify the xxx from the Lth line of the Nth page to xxxx and create a separate instruction document. Printing, correcting handwriting on paper → drawing directly on a document, etc.).
Further, since the scroll line length of an arbitrary application program is detected and the document position can be detected, it can be used for many applications.
In addition, since the document display is fixedly displayed even when the document position information is detected, the discomfort caused by the scroll display can be eliminated, and the convenience can be improved.
In addition, since the document position of an application whose scroll line length is different for each scroll can be detected, it can be used for many applications.

In addition, since the original display state is restored after the document position information is detected, an operation for correcting the display position is not necessary, so that convenience can be improved.
In addition, since the displayed document can be scrolled and displayed only by operating the document position detection control application program, it is possible to eliminate the trouble of erroneous operation and switching between the two, thereby improving convenience. be able to.
Further, since scroll information detection at the time of document position information detection can be omitted, discomfort caused by scroll display can be eliminated, and convenience can be improved.
Further, since detection of scroll line number (position) information at the time of document position information detection can be omitted, it is possible to eliminate discomfort caused by scroll display and to prevent waste of time, and to improve convenience.
Further, according to the electronic conference system according to the present invention, the PC conference operation right terminal notifies the PC conference participation terminal when the display position of the conference information and the overwrite information is changed. By displaying the changed meeting information and the overwrite information at the same position, it becomes possible to quickly reflect the image of the operation PC on the participating PC, so that a smooth PC meeting is possible and the meeting Efficiency can be improved.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
A first aspect of the present invention relates to an apparatus, and scrolls a document (image) displayed by an arbitrary application program operating on a general-purpose OS (operating system) such as Windows (registered trademark) on various display apparatuses. The present invention relates to a document position detecting device that detects the number of lines and the scroll line length by moving a scroll box. A second aspect of the present invention to be described later relates to an electronic conference system.

  FIG. 1 is an explanatory diagram showing various display device examples to which the present invention is applied. FIG. 1A shows an example of a display device in which a tablet is connected to a PC (personal computer), a document is corrected by handwriting on a document created by an arbitrary application program, and the contents of the change are drawn. FIG. 1B shows an example of a display device that corrects a document by handwriting on a document created by an arbitrary application program using a pen of a tablet PC and draws the changed contents, and FIG. A display device that improves the efficiency of lectures and lectures by attaching a touch panel to large displays such as PDPs and rear projectors (inch, 60-inch, etc.) and overwriting them on materials created by any application program with your fingers. Fig. 1 (d) shows an example of a display-integrated coordinate input device). And shows a display device example to improve the efficiency of the conference by overwriting onto article created by any application program.

FIG. 2 is a schematic view of an embodiment in which the document position information detection device according to the present invention is applied to the display device shown in FIG. As shown in FIG. 2, the display device 1 is a large display (50 inches, 60 inches, etc.) PDP and a liquid crystal rear projector, and is a display-integrated coordinate detection device in which a touch panel 3 is disposed on the front surface of the display. The touch panel 3 detects a coordinate indicator such as a finger or a pen touching the display surface.
A personal computer (PC) 5 that provides a display image is connected to the display device 1, a position is calculated based on information detected by the touch panel 3, and coordinate data is obtained via a PC_I / F such as RS232C or USB. A coordinate calculation control device 7 for notifying the PC 5 is connected. The personal computer 5 has a RAM for storing data and the like, a ROM for storing a program, and a CPU for executing the program.

Next, with reference to FIG. 3, a light blocking detection method will be described as an example of a coordinate detection method for a touch panel. FIG. 3 is an explanatory diagram of a light blocking detection method in a touch panel. As shown in FIG. 3, a plurality of light blocking detection type touch panels are arranged in a row according to the detection accuracy in the X-axis and Y-axis directions (the number is the same as 1024 × 768 pixels which is the resolution of the XGA display in the figure). The coordinate calculation control device includes a light emitting diode (LED) 9 that is arranged and emits infrared light, and a phototransistor 11 that receives infrared light emitted from the LED 9 and converts it into an electrical signal and transmits it to the touch panel control device. 7 sequentially emits the LEDs in the X-axis direction and the Y-axis direction, and when the electrical signal transmitted from the phototransistor is equal to or less than a specific amount, calculates the coordinate position from the light-emitting LED, assuming that a coordinate indicator exists.
Here, when the coordinate indicator 13 is in the position shown in FIG. 3, the infrared light emitted from the second LED A on the X axis is blocked by the coordinate indicator and is not received by the phototransistor, and the X coordinate “2” is detected. Similarly, the infrared light emitted from the third LED B on the Y axis is blocked by the coordinate indicator, and the Y coordinate “3” is detected without being received by the phototransistor.

Still another light blocking detection method as a coordinate detection method will be described with reference to the drawings.
4 and 5 are explanatory diagrams of another embodiment of the light blocking detection method on the touch panel 3 that is substantially the same as the method shown in FIG. The display-integrated coordinate input device 110 of this type is mainly composed of a coordinate input device 111, a display device 112, and a host device 113. The coordinate input device 111 sets the position of the coordinate pointing object touched on the screen installed so as not to hide the screen of the display device 112 as the number of pixels (pixels) of the display device 112 (X-axis direction 1024, Y-axis direction 768, etc. ) Are detected periodically (5 ms, 10 ms, etc.) and sent to the host device 113 via PCI / F (communication means such as RS232C, USB). The display device 112 displays the RGB image data output from the host device 113 on the liquid crystal screen and the PDP screen.

FIG. 5 is an explanatory diagram showing a coordinate detection method in the coordinate input device 111 of FIG. The left coordinate indicator detector 115 and the right coordinate indicator detector 116 emit irradiation light by a semiconductor laser, detect light reflected by the light retroreflective member 114, convert it into analog data, and perform coordinate detection control. Send to device 117. When there is no coordinate indicator such as a finger or a pen on the screen, the irradiation light is reflected and received in the opposite direction by 180 degrees by the light retroreflective member 114 (see the arrow on the right side of the screen).
On the other hand, when a coordinate indicator exists on the screen, the irradiated light is blocked and not reflected by the light retroreflective member 114, and the left coordinate indicator detector 115 and the right coordinate indicator detector 116 do not receive light. The left light blocking angle (θL) and the right light blocking angle (θR) are detected by the left coordinate indicator detector 115 and the right coordinate indicator detector 116 in the direction in which no light is received (light blocking angle). The X coordinate and Y coordinate of the coordinate indicator are calculated from W, θL, and θR according to the following formula.
X = W × tan θR / (tan θL + tan θR)
Y = H−W × tan θL × tan θR / (tan θL + tan θR)
Thereafter, the detected coordinates are notified by communication means such as RS232C and USB, which are standard equipment on the host PC 113.

Furthermore, overwriting information coordinate detection methods in various display device examples other than those described above will be described with reference to the drawings. 6 and 7 are explanatory diagrams of another example of the coordinate detection method of the overwriting information. As another method for detecting the coordinates of the overwriting information, there is a method in a tablet PC 15. The electronic pen and the sensor built in the tablet PC 15 detect the position (coordinates) of the electronic pen on the display, and emulate mouse click and drag operations. With a horizontal display and an electronic pen, handwritten characters that are difficult to operate with a mouse can be drawn as if writing on paper.
The tablet PC 15 is composed of a liquid crystal display, a keyboard, and a mouse, and can be operated in the same manner as a general notebook PC. However, instead of the keyboard and mouse operation, the tablet PC 15 is touched on the screen with the electronic pen 16 to operate the mouse. A touch panel 17 is provided so that it can be operated instead, and text input can be performed by handwritten character drawing or character recognition instead of the keyboard.
The left side of FIG. 6 shows a state where the screen (display) of the tablet PC 15 is opened, and the right side of FIG. 6 shows a state where the screen is rotated by 180 degrees and folded. In coordinate detection, the tempered glass 19, the liquid crystal display unit 21, and the electromagnetic induction sensor unit 23 are overlapped to press the touch panel with a pen or detect a change in the electric field on the touch panel side due to static electricity and replace it with coordinates. It has a structure. A tempered glass 19 is installed on the surface directly in contact with the electronic pen to prevent the liquid crystal display unit 23 from being damaged. An electromagnetic induction sensor unit 23 for detecting the position of the electronic pen 16 is installed under the liquid crystal display unit 21 so that the image quality of the display is not impaired.

Next, an electromagnetic induction coordinate detection method using the electromagnetic induction sensor unit 23 will be described.
As shown in FIG. 7A, a resonance circuit 25 is built in the electronic pen 16, and an induced electromotive voltage is generated in the coil 27 by a magnetic field generated by a current flowing in the sensor coil 27 of the electromagnetic induction sensor unit 23, and is composed of a capacitor or the like. The resonance circuit 25 is excited and a resonance current flows. FIG. 7A is an explanatory diagram of an electromagnetic induction coordinate detection method by the electromagnetic induction sensor unit 23.
The sensor coil 27 is installed at a density proportional to the coordinate detection accuracy. In the figure, a part of the sensor coil for detecting the X-axis direction is shown. Although not shown, a sensor coil for detecting the Y-axis direction is installed in a direction perpendicular thereto.
The coordinate detection control device 29 controls the transmission / reception selector switch 31, connects the transmission / reception selector switch 31 to the AC power source 33, passes a current from the AC power source 33 to the sensor coil 27, controls the X-axis coil selector switch 35, and switches at high speed. The current is passed through the sensor coil 27 sequentially. As a result, a magnetic field is sequentially generated on the sensor coil 27. This magnetic field causes the electronic pen 16 to resonate.
Next, the transmission / reception change-over switch 31 is connected to the AC power source 33, and the X-axis coil change-over switch 35 is sequentially switched at high speed. At this time, an induced electromotive voltage is generated in the sensor coil 27 by a magnetic field generated by the resonance current of the electronic pen 16. Then, the induced electromotive voltage generated in the sensor coil 27 is amplified and measured (V).

FIG. 7B shows the distribution of induced electromotive voltages detected at this time. FIG. 7B is an induced electromotive force distribution diagram in the electromagnetic induction sensor unit 23. Here, when the electronic pen 16 is on the sensor coil “2”, the induced electromotive voltage received from the sensor coil “2” is the highest. Thus, the X coordinate is determined by detecting the highest point of the induced electromotive voltage distribution. Although not shown, the Y coordinate is determined in the same manner.
Then, the detected coordinates of the electronic pen 16 are notified to the PC 5 by an I / F (a general-purpose I / F such as RS232C or USB, or a dedicated I / F). The coordinate detection control device is built in the display device. Here, a form in which the display is rotated and folded is shown, but there is a form in which the display unit can be removed.

Next, coordinate input control will be described with reference to FIG. FIG. 8 is an explanatory diagram regarding coordinate input control, (a) is a conceptual block diagram, and (b) and (c) are explanatory diagrams showing states of coordinate data and mouse data.
As shown in FIG. 8A, the “coordinate data” is detected by the touch panel 3 when the coordinate indicator touches the display surface, and the coordinate calculation control device 7 calculates the position and transmits it to the PC 5. The touch panel driver 41 receives “coordinate data” transmitted from the coordinate calculation control device 7 through communication means such as RS232C, USB, etc., converts it into “mouse data” defined by the OS, and provides the mouse I / O provided by the OS. Output to F43.
The “mouse data” has the same format as the data output by the mouse connected to the PC 5, and the “application program” is the same as the mouse operation.
That is, a normal mouse has a right button, a left button, and a mouse having a middle button, but since there is no touch panel, the touch operation is treated as button ON and the detach operation is treated as button OFF. As for (left), it is assumed that the user selects it according to the properties provided by the touch panel driver.
The “application program” to which the execution right is given by the OS receives “mouse data” from the mouse I / F 43. The function defined in the information under the mouse coordinates (menu, button, etc.) is executed. As a result of the execution, the screen display is changed, “image data” is updated, converted into RGB signals, and displayed as an image on the display. FIGS. 8B and 8C show schematic structures of coordinate data and mouse data.

[Screen Display Information]
With reference to FIG. 9, screen display information such as scroll box position information, display image, and window position information that can be acquired by an I / F (API) provided by the OS (in the case of Windows (registered trademark)) will be described. . All window information being displayed (upper left origin coordinates, width, height, window handle, window class, window style, window title) can be acquired.
The window handle is a window identifier that is uniquely assigned to the window being displayed, whereby the window information is obtained and set. When the vertical scroll bar is created as a window, it is identified from the window class “ScrollBar”, (window width <window height), and when attached to a window, from the window style “WS_VSROLL”. When the horizontal scroll bar is created as a window, it is identified from the window class “ScrollBar”, (window width> window height), and when attached to the window, from the window style “WS_HSCROL”.
The upper left origin coordinates (HX, HY), width (HW), and height (HH) of the horizontal scroll bar can be acquired.
The horizontal scroll length (HL) indicates the full width including the non-display portion of the document, and can be acquired by an I / F provided by the OS.
The horizontal scroll display width (HB) indicates the width of the display portion of the document, the horizontal scroll box position (HP) indicates the position of the left end of the display document, and can be acquired by an I / F provided by the OS.
The upper left origin coordinates (VX, VY), width (VW), and height (VH) of the vertical scroll bar can be acquired.
The vertical scroll length (VL) indicates the vertical length of all pages of the document including the non-display portion, and can be acquired by an I / F provided by the OS.
The vertical scroll display length (VB) indicates the vertical length of the display portion of the document, and the vertical scroll box position (VP) indicates the position of the upper end of the display document, which can be acquired by an I / F provided by the OS.
The window in contact with the scroll bar becomes the window (document display area) controlled by the scroll bar, and the upper left origin coordinates (WX, WY), window width (WW), and window height (WH) can be acquired.
The program name and display document file name can be acquired from the window title information.

Next, the positional relationship between the scroll bar information and the display document will be described with reference to FIG. FIG. 9 is an explanatory diagram of the positional relationship between the scroll bar information and the display document.
As shown in FIG. 9A, the vertical scroll box position (VP), vertical scroll box width (VB), and vertical scroll length (VL) are obtained by an API (application interface) provided by the OS (in the case of Windows (registered trademark)). This is possible and shows the vertical positional relationship of the document display area in the entire document.
VP: Length from the first page to the upper end of the document display area VL: Vertical length of the entire document VB: Vertical length of the display portion As shown in FIG. 9B, the horizontal scroll box position (HP), horizontal scroll The box width (HB) and the horizontal scroll length (HL) can be acquired by an API provided by the OS, and indicate the positional relationship in the horizontal direction on the display page of the document display area.
HP: Length from the left end of the display page to the left end of the document display area HL: Length of the display page in the horizontal direction HB: Length of the display portion in the horizontal direction FIG. 9C shows the positional relationship of the document display area.

A method for converting screen coordinates into document coordinates will be described with reference to FIG.
The horizontal scroll length (HL) indicates the total length of the displayed document in the horizontal direction, and the horizontal scroll display width (HB) indicates the width of the portion displayed on the screen.
The vertical scroll length (VL) indicates the page length in the vertical direction of the display document, and the vertical scroll display length (VB) indicates the height of the portion displayed on the screen.
The positional relationship between the entire document page area (document coordinate space) and the screen display portion is shown.
Based on the information provided by the OS, the vertical scroll box position (VP) is converted into the number of scroll lines (LN), and the scroll line length of the information provided by the OS (LL: pixel length as a basic unit to be scrolled by pressing the scroll bar button) ) To the document Y coordinate (SY) with the top edge of the first page as the origin.
Document Y coordinate (SY) = LN * LL + (Y−WY)
Based on information provided by the OS, the horizontal scroll box position (HP) is converted to the distance (XP: pixel length) from the left edge of the document page, the horizontal scroll length (HL) is converted to the pixel length (XL), and the screen X coordinate is converted to the document. Conversion to document X coordinate (SX) with the center of the page as the origin (left side-, right side +).
Document X coordinate (SX) = (X−WX) − (XL / 2−XP)
Based on the information provided by the OS, the vertical scroll length is converted into the number of scroll lines to obtain the total number of document lines (TL).

A method of converting document coordinates to screen coordinates will be described with reference to FIG.
Based on the information provided by the OS, the vertical scroll box position (VP) is converted into the number of scroll lines (LN), and the document origin Y coordinate (DY) of the document display area is calculated from the scroll line length (LL) of the information provided by the OS. To do.
Display area origin document Y coordinate (DY) = LN * LL
The horizontal scroll box position (HP) is converted into a distance (XP: pixel length) from the left edge of the document page by information provided by the OS, and the horizontal scroll length (HL) is converted to pixel length (XL) by the information provided by the OS. Conversion and document center coordinates (DC) are calculated.
Document center coordinates (DC) = (XL / 2−XP) + WX
When (WX <DC <(WX + WW)), the display area document space is as follows.
DY ≦ document Y coordinate ≦ (DY + WH) − (DC−WX) ≦ document X coordinate ≦ + (WW + WX−DC)
When (DC ≦ WX), the display area document space is as follows.
DY ≦ document Y coordinate ≦ (DY + WH) + (WX−DC) ≦ document X coordinate ≦ + (WX−DC + WW)
When ((WX + WY) ≦ DC), the display area document space is as follows.
DY ≦ document Y coordinate ≦ (DY + WH) − (DC−WX) ≦ document X coordinate ≦ − (DC− (WX + WY))
When the document coordinates (SX, SY) are within the display area document space, the coordinates are converted to screen coordinates (X, Y) as follows.
Screen X coordinate = DC + Document X coordinate screen Y coordinate = Document Y coordinate−DY + WY

Next, the position information of the scroll box will be described with reference to FIG.
As shown in FIG. 12A, the upper left corner coordinates (WX, WY) of the document display area are acquired by an API provided by the OS.
As shown in FIG. 12B, when the scroll box position (HP0, VP0) indicates pixel coordinates, the document coordinates (DX, DY) of the specific position (X0, Y0) are calculated as follows.
DX = HP0 + (X0−WX)
DY = VP0 + (Y0-WY)
As shown in FIG. 12C, when the scroll box position (HP1, VP1) is reached by the scroll operation, the screen coordinates (X1, Y1) of the specific position are calculated as follows.
X1 = (DX-HP1) + WX
Y1 = (DY−VP1) + WY

[Change display magnification]
Next, a display magnification conversion method when the document display magnification is changed will be described with reference to FIG.
FIG. 13 shows the display state when the document coordinates (SX0, SY0) are acquired.
When the document is enlarged (reduced), the distance from the left end of the document page is enlarged (reduced) in proportion to the display magnification, but the information for calculating the left end coordinate of the document page is not provided by the OS.
When obtaining the document coordinates, the total number of lines (TL0) is calculated and stored from the vertical scroll length (VL0).
The total number of lines (TL1) is calculated from the vertical scroll length (VL1) when the display magnification is changed.
Since the scroll line length (LL) is the same regardless of the display magnification, the display magnification is proportional to the total number of lines in the document as follows.
Display magnification = TL1 / TL0
Document coordinates (SX0, SY0) are as follows when the display magnification is changed.
Document X coordinate (SX1) = SX0 * (TL1 / TL0)
Document Y coordinate (SY1) = SY0 * (TL1 / TL0)

Next, the scroll information will be described with reference to FIG.
Here, there is a difference in the amount of movement at the time of scroll execution and the meaning of the scroll bar information for each application program. The scroll box position is moved to the top (left end) and scrolled in line units. ) Is detected to obtain scroll information.
FIG. 14A shows a case where the line number matches the scroll box position.
Scroll type: line position FIG. 14B shows a case where the scroll box position increases for each of a plurality of lines.
Scroll type: block position FIG. 14C shows a case where the scroll box position increases in line length.
Scroll type: pixel position FIG. 14D shows a case where the line length is indefinite.
Scroll type: unspecified position As shown in E of FIG. 14, the program name and version information of the application program for displaying the document are acquired by the API provided by the OS.
Here, the scroll information is saved in the scroll information file, and when the position detection is executed, the program name and version information of the target application program are acquired. If the scroll information is saved in the scroll information file, the saved information is input. Skip detection.

Next, scroll line length detection (drawing method) will be described with reference to FIG.
Here, the movement amount (line length) of the document by one scroll operation is detected. At the time of scrolling, generally, the area remaining by scrolling is not redrawn, the image is moved, and only the newly appearing part is drawn. Also, the image on the document area on the transparent window moves simultaneously when scrolling.
As shown in FIG. 15A, a full-screen size transparent window (the background is transmitted and displayed) is displayed.
As shown in FIG. 15B, the upper left coordinates (WX, WY), width (WW), and height (WH) of the document area are acquired by the API provided by the OS, and the horizontal line (line length: L) drawing position ( X, Y0) is calculated.
X = WX + WW / 2−L / 2
Y0 = WY + WH-32

As shown in FIG. 15C, the length (L), thickness (from the start coordinates (X, Y0) in colors (R: 254, G: 1, B: 2, etc.) that do not appear in the normal document or image. 1) Draw a horizontal line.
As shown in FIG. 15D, the vertical scroll box position is moved downward (LINEDOWN) by an API provided by the OS.
At this time, the document image and the horizontal line drawn on the transparent window move upward by the scroll amount.
As shown in E of FIG. 15, a display image having a width (L) and a height (1) in which the start coordinates are sequentially (X, Y0), (X, Y0-1)... (X, YL). When all of the acquisition and acquisition data are drawing colors, it is determined as a moving position by scrolling, and a vertical line length (VLL) is calculated.
VLL (vertical line length) = Y0-YL
As shown in F of FIG. 15, a vertical line having a start coordinate (X0, Y), a length (L), a thickness (1), and a color (R: 254, G: 1, B: 2) is drawn. Similar to the detection of the scroll line length, a display image having a width (1) and a height (L) in which the start coordinates are sequentially (X0, Y), (X0-1, Y)... (XL, Y) is acquired. By doing so, the horizontal line length (HLL) is detected.
X0 = WX + WW-32
Y = WY + WH / 2−L / 2
VLL (horizontal line length) = X0-XL

Next, a scroll line length detection method (image extraction method) based on an image extraction method will be described with reference to FIG.
Generally, an area remaining by scrolling is not redrawn, but some application programs redraw all. At this time, the drawing in the document area on the transparent window is lost by redrawing. Therefore, the scroll line length detection line cannot be detected, and the scroll line length cannot be detected.
As shown in A of FIG. 16, the upper left coordinates (WX, WY), width (WW), and height (WH) of the document area are acquired by the API provided by the OS, the start coordinates (X, Ys), and the width ( The reference image of the area (WW) and height (1) is acquired. (* Maximum vertical line length is the maximum value actually detected)
X = WX
Ys = WY + WH-MVL (maximum vertical line length)
A display image having a width (WW) and a height (1) with the start coordinates sequentially (X, Ys + 1), (X, Ys + 2), (X, Ys + 3),. Search for an image that matches the reference image.

As shown in FIG. 16B, when matching images exist, the start coordinates are sequentially set to (X, Ys-1), (X, Ys-1), (X, Ys-1), ... (X , Y0), a reference image of a width (WW) and height (1) region is acquired, a display image is acquired in the same manner, and a search is made for a matching image.
As shown in FIG. 16C, when there is no matching image at the start coordinate (X, Y0), the reference image is set.
This indicates that the same image as the reference image does not exist in the scroll movement region.
As shown in FIG. 16D, the vertical scroll box position is moved downward (LINEDOWN) by the API provided by the OS.
The start coordinates are sequentially ((X, Y0-1), (X, Y0-2), (X, Y0-3), ... (X, YL). A display image is acquired, searched for an image that matches the reference image (matches at start coordinates (X, YL)), and a vertical line length is calculated.
VLL (vertical line length) = Y0-YL
As shown in E of FIG. 16, a reference image of the start coordinate (Xs, Y), width (1), and height (WH) area is acquired (* the maximum horizontal line length is the maximum value of the actually detected value. To do).
Xs = WX + WW-HML (maximum horizontal line length)
Y = WY

As shown in F of FIG. 16, the width (1) and the height where the start coordinates are sequentially (Xs + 1, Y), (Xs + 2, Y), (Xs + 3, Y),... (Xs + MHL-1, Y). The display image of (WH) is acquired and searched for an image that matches the reference image.
As shown in G of FIG. 16, when there is a matching image, the start coordinates are sequentially set to (Xs-1, Y), (Xs-2, Y), (Xs-3, Y),. , Y), a reference image of the width (1) and height (WH) region is acquired, a display image is acquired in the same manner as F, a search is made for a matching image, and a reference display image is obtained if there is no match.
The horizontal scroll box position is moved one line to the right (LINERIGHT) by an API provided by the OS.
The start coordinates are sequentially ((X0-1, Y), (X0-2, Y), (X0-3, Y),... (XL, Y)) of width (1) and height (WH). A display image is acquired, searched for an image that matches the reference image (matches at start coordinates (XL, Y)), and a horizontal line length is calculated.
HLL (horizontal line length) = X0-XL

Next, a scroll screen fixed display method at the time of scroll execution will be described with reference to FIG.
As shown in FIG. 17A, the upper left corner coordinates (WX, WY), width (WW), and height (WH) of the document area are acquired.
As shown in FIG. 17B, an image of the document area is acquired by an API provided by the OS.
As shown in FIG. 17C, a transparent window having a full screen size is displayed on the document.
As shown in FIG. 17D, the acquired image of the document area is displayed at the same position as the document area.
As a result, when scroll control is performed by moving the vertical scroll box position and the horizontal scroll box position, the screen is not displayed in a scroll but is fixed.

Next, a method for detecting the current line number (current line number) will be described with reference to FIG.
As shown in FIG. 18A, when the scroll type indicates “block position”, the scroll box position obtained from the scroll bar information has the same value regardless of where the line position is L, L + 1, L + 2, L + 3, or L + 4. It becomes. Also, there is no API for obtaining the line position in the block. The vertical scroll box position indicates the upper end of the display portion, and the horizontal scroll box position indicates the left end of the display portion. However, the display contents are different when the line position is “L” and “L + 4”.
As shown in FIG. 18B, LINEUP (LINELEFT) is executed by the API provided by the OS, and the number of lines (N) until the scroll box position (N) decreases is detected.
As shown in FIG. 18C, the scroll box position is moved to the head, LINEDOWN (LINERIGHT) is repeated until the scroll box position becomes “N”, and the number of lines (S) is detected.
As shown in FIG. 18D, the current line number = S + (N−1).
As shown in E of FIG. 18, when the scroll box position is “0”, the number of lines (N) is detected by LINEDOWN (LINERIGHT), the number of lines up to the scroll box position “1” (S) is detected, and the current line Calculate the number.
Current line number = SN

Next, the line coordinate detection method will be described with reference to FIG.
As shown in FIG. 19A, (1) when the vertical scroll type is “unspecified position”, the vertical line coordinates are obtained by moving the vertical scroll box position to the top, LINEDOWN, vertical scroll line length detection, and addition. Is detected.
As shown in FIG. 19B, when the horizontal scroll type is “unspecified position”, the horizontal line coordinates are detected by moving the horizontal scroll box position to the left end, repeating the LINEIGHT, detecting the horizontal scroll line length, and adding. .

Next, position detection file registration will be described with reference to FIG.
As shown in FIG. 20A, a “position detection file” dialog is displayed when a position detection file is registered.
Click the “Register” button to display the “Register” dialog and enter the position detection file.
The “Open” button displays a file selected from the list display, and executes position detection control based on information stored in the position detection registration file.
With the “Delete” button, the registered file information selected from the list display is deleted.
As shown in FIG. 20B, the file input by the “registration” dialog is displayed. As shown in FIG. 20C, the program name and version information of the application program for displaying the file are acquired by the API provided by the OS.

As shown in D of FIG. 20, the window information being displayed (upper left corner coordinates (WX, WY), width (WW), height (WH), window class, window title)) is searched by an API provided by the OS. Window information (window class, window title) is acquired with the window having the largest area (WW * WH) as the document area window.
As shown in E of FIG. 20, the window information being displayed is searched, and the window information (window class, window title) of the horizontal scroll bar adjacent to the document area is acquired.
As shown in F of FIG. 20, the window information being displayed is searched, and the window information (window class, window title) of the vertical scroll bar adjacent to the document area is acquired.
As shown in G of FIG. 20, scroll information necessary for position detection control is detected and stored.
When registered in the position detection file at the time of position detection control execution, registration information is used and detection of scroll information is omitted.

Next, an application example of the position detection technology described so far will be described with reference to FIG.
Here, as an application example of position detection control, drawing characters (graphics) on a displayed document by handwriting in presentations, etc., and in electronic conferences, etc. that are performed using a plurality of terminal devices between remote spaces Shows the case of improving efficiency by clearly explaining the explanation and supplementary explanation. However, here, assuming that the work window is set to be displayed at a predetermined position and a predetermined size (the entire screen is acceptable) and that the predetermined document is displayed in this work window, detailed description is limited to detecting the scroll operation of the document. To do.
Position detection control is also performed in other display change operations that are generally required in meetings, etc., but these other position detection control controls will not be discussed here, and another electronic conference system will be installed later. This will be described as an example. Also, there is a technique for notifying other connected devices of the position detection result (including transmission via WAN etc.) and displaying the display contents with the same display contents on the other device (display synchronization). This will be described later in an example of an electronic conference system, and is omitted here.

As shown in FIG. 21A, the “application section” displays a toolbar according to the purpose.
As shown in FIG. 21B, the “position detection control” executes a position detection control function in response to a request from the application unit.
As shown in FIG. 21C, at the start of position detection, the program name, version information, and document file name are acquired from the displayed window information.
As shown in FIG. 21D, when the scroll information of the display program is not registered in the “scroll information file”, the scroll information is detected and stored in the scroll information file.
If registered, the registration information is input and detection of scroll information is omitted.
As shown in E of FIG. 21, the document file registered in the “position detection registration file” is searched.
If registered, the registration information is input, and the detection of scroll information, the number of scroll lines, and line coordinates is omitted.

As shown in F of FIG. 21, handwritten characters (graphics) on the document are drawn by linearly connecting the mouse coordinates at the time of the drawing operation.
The mouse coordinates at this time are converted to document coordinates and saved.
As shown in FIG. 21G, scrolling is performed by moving the scroll box upward by the “up” button, downward by the “down” button, right by the “right” button, and left by the “left” button. Display.
As shown in H of FIG. 21, the upper left document coordinates (DX, DY), document area width (DW), and document area height (DH) of the display document are calculated, and document coordinate data in the same area exists. The document coordinates are redrawn by converting the screen coordinates and connecting the coordinates with a straight line.

Next, a functional configuration of document position information detection control will be described with reference to FIG. FIG. 22 is an explanatory diagram of the document position information detection control in the PC 5 shown in FIG. 6, and (a) is a functional configuration block diagram of the document position information detection control in the PC 5. Note that the following document position information detection control is specifically executed by the CPU based on a program stored in the ROM of the PC 5.
In FIG. 22A, the functional configuration of the document position information detection control is a function provided by converting and saving the screen coordinates by the mouse operation on the document to the document coordinates via the function provided by the position detection control unit. The position detection start control unit 53 for inputting or detecting information necessary for position detection from the scroll information file or the position detection registration file, the number of scroll lines in the position detection file, and the line coordinates A position detection file control unit 55 for detecting and saving the position detection file, inputting the position detection control of the displayed document according to the registered information, starting the position detection control, and the mouse. Document coordinate calculation control unit 57 that converts screen coordinates by operation to document coordinates, and a screen that converts document coordinates to screen coordinates A standard calculation control unit 59, a display document region calculation control unit 61 that detects the upper left document coordinates, document width, and document height of the screen display document region, and scrolls the document downward, upward, right, and left. A scroll control unit 63, a document coordinate display control unit 65 that displays a document in an area including document coordinates, a position detection end control unit 67 that performs end processing of position detection control, and an area that can be referred to by each control unit The position detection control table 69 that holds information necessary for position detection control, the scroll information table 71 that holds scroll information and is set in an area that can be referred to from each control unit, and the area that can be referenced from each control unit And a vertical line coordinate table 73 that holds the document Y coordinate for the vertical line number, and an area that can be referred to by each control unit, and the document X for the horizontal line number. And a horizontal line coordinate table 75 for holding a target.
22B, 22C, 22D, and 22E are explanatory diagrams showing the contents of the position detection control table 69, the scroll information table 71, the horizontal line coordinate table 75, and the vertical line coordinate table 73, respectively. It is.

Hereinafter, only the position detection control unit will be described with reference to flowcharts.
Next, the processing operation of the position detection start control will be described with reference to FIG.
Here, at the start of position detection, the screen coordinates (X, Y) of the target document area are executed as input values.
23, window information (upper left corner coordinates, width, height, window handle) of the screen coordinates (X, Y) is acquired by the API provided by the OS, and the window handle is acquired from the position detection control table “document area identification”. To "".
In step S103, the window information being displayed is searched, the window information (window handle) of the vertical scroll bar adjacent to the document area is acquired, and the window handle is set in the position detection control table “vertical scroll bar identification”.
In step S105, the window information being displayed is searched, the window information (window handle) of the vertical scroll bar adjacent to the document area is acquired, and the window handle is set in the position detection control table “vertical scroll bar identification”.

In step S107, "current vertical line number detection" processing is executed, the current vertical line number of the display document is acquired, and set in the position detection control table.
In step S109, "current horizontal line number detection" processing is executed, the current horizontal line number of the display document is acquired, and set in the position detection control table.
In step S111, the program name and version information of the program for displaying the document are acquired by the API provided by the OS and set in the position detection control table.
In step S113, the file name and file path of the display document are acquired by the API provided by the OS.
In step S115, it is searched whether the position detection information of the display document is registered in the “position detection registration file”.
In steps S117 and 119, scroll information is input and set in the scroll information table.

If the vertical scroll type is “unspecified position” in step S121, the vertical line coordinate table is input and set in the vertical line coordinate table.
If the horizontal scroll type is “unspecified position” in step S123, the horizontal line coordinate table is input and set in the horizontal line coordinate table.
In steps S125, 127, and 131, it is searched from the “scroll information file” whether or not the scroll information of the display program is registered, and if it is registered, it is set in the scroll information table.
If the vertical scroll type is “unspecified position” in steps S133 and 135, “vertical line coordinate detection” processing is executed, and a vertical line coordinate table is set.
If the vertical scroll type is not “unspecified position” in step S137, “vertical scroll line number detection” processing is executed, the number of vertical scroll lines is acquired, and set in the position detection control table.

If the horizontal scroll type is “unspecified position” in steps S139 and 141, “horizontal line coordinate detection” processing is executed, and a horizontal line coordinate table is set.
If the horizontal scroll type is not “unspecified position” in step S143, “horizontal scroll line number detection” processing is executed, the horizontal scroll line number is acquired, and set in the position detection control table.
If scroll information is not registered in step S145, “vertical position detection information detection” processing is executed, vertical scroll information and vertical line coordinate information are acquired, and set in the table.
If the scroll information is not registered in step S147, “horizontal position detection information detection” processing is executed, horizontal scroll information and horizontal line coordinate information are acquired, and set in the table.

In step S149, the detected scroll information is stored in a “scroll information file”.
In step S151, a “vertical line display” process is executed to display the current vertical line number position.
In step S153, "horizontal line display" processing is executed to display the current horizontal line number position.
In step S155, transparent screen display and position detection control of the full screen size are started.
24A, 24B, 24C, 24D, and 24E are explanatory diagrams showing the input values and the state of each table in the flowchart of the position detection start control operation.

Next, the position detection file control operation will be described with reference to FIG.
In step S201 of FIG. 25, a “position detection file” dialog is displayed and processing is started.
In steps S203, 205, 207, and 209, when a “registration” button is pressed, a “registration dialog” is displayed and a registered file name is input.
In steps S211 and S213, the input file is displayed.
In step S215, the program name and version information of the program that displays the file are acquired.
In step S217, the window information being displayed (upper left corner coordinates (WX, WY), width (WW), height (WH), window class, window title)) is retrieved by the API provided by the OS, and the area (WW * WH) is displayed. ) To obtain window information (window class, window title) with the largest window as the document area window.

In step S219, the window information being displayed is searched, and the window information (window class, window title) of the horizontal scroll bar adjacent to the document area is acquired.
In step S221, the window information being displayed is searched, and window information (window class, window title) of the vertical scroll bar adjacent to the document area is acquired.
In step S223, a “vertical position detection information detection” process is executed to acquire vertical scroll information and vertical line coordinate information.
In step S225, a “horizontal position detection information detection” process is executed to acquire horizontal scroll information and horizontal line coordinate information.
In step S227, the acquired information is stored in the position detection registration file.
In step S229, the display file is closed.
In steps S231 and S233, the “delete” button is pressed to delete the selected file from the position detection registration file.
In steps S235, 237, 239, and 241, by pressing the “Open” button, the “registered file start” process is executed, the file is displayed, and the position detection control is started. FIG. 26 is an explanatory diagram showing the state of the position detection registration file in the flowchart of the position detection file control operation.

Next, the document coordinate calculation control operation will be described with reference to FIG.
Here, the screen coordinates are executed as input values.
In step S301 of FIG. 27, a “display document area calculation control” process is executed to acquire display document coordinates.
In step S303, the upper left screen coordinate (X, Y) of the document area is detected by an API provided by the OS based on the document area identification.
In step S305, document X coordinate = screen X coordinate−X (upper left upper screen X coordinate) + DX (upper left upper document X coordinate).
In step S307, document Y coordinate = screen Y coordinate−Y (upper left upper screen Y coordinate) + DY (upper left upper document Y coordinate).
In step S309, the document coordinates are passed to the request source.
FIGS. 28A, 28B, and 28C are explanatory diagrams showing states of input values, display document coordinates, and document coordinates in the flowchart of the document coordinate calculation control operation, respectively.

Next, the screen coordinate calculation control operation will be described with reference to FIG.
Here, the document coordinates are executed as input values.
In step S401 in FIG. 29, a “display document area calculation control” process is executed to acquire display document coordinates.
In step S403, the upper left screen coordinate (X, Y) of the document area is detected by an API provided by the OS based on the document area identification.
In steps S405 and 407, if the document X coordinate is within the display area, it is converted to the screen X coordinate.
Screen X coordinate = Document X coordinate-DX (Upper left upper document X coordinate) + X (Upper left upper screen X coordinate)
In step S409, coordinates outside the document area are set to invalid (-1).
In steps S411 and 413, if the document Y coordinate is within the display area, it is converted to the screen Y coordinate.
Screen Y coordinate = Document Y coordinate-DY (Upper left document Y coordinate) + Y (Upper left screen Y coordinate)
In step S415, the coordinates outside the document area are set to invalid (-1), and the screen coordinates are passed to the request source.
FIGS. 30A, 30B, and 30C are explanatory diagrams showing states of input values, display document coordinates, and document coordinates, respectively, in the flowchart of the screen coordinate calculation control operation.

Next, the display document area calculation control operation will be described with reference to FIG.
In step S501 in FIG. 31, the document area width (WW) and height (WH) are acquired by the API provided by the OS by document area identification.
In steps S503, 505, and 507, in the case of the horizontal scroll type “pixel position”, the horizontal scroll box position is acquired and set as the upper left document X coordinate.
In steps S509 and 511, in the case of the horizontal scroll type “unspecified position”, the horizontal line coordinate of the current horizontal line number is set as the upper left document X coordinate from the horizontal line coordinate table.
In step S513, in the case of the horizontal scroll type “line position or block position”, “current horizontal line number * horizontal line length” is set as the upper left document X coordinate.

In steps S515, 517, and 519, in the case of the vertical scroll type “pixel position”, the vertical scroll box position is set as the upper left document Y coordinate.
In steps S521 and S523, in the case of the vertical scroll type “unspecified position”, the vertical line coordinate of the current vertical line number is set as the upper left document Y coordinate from the vertical line coordinate table.
In step S525, in the case of the vertical scroll type “line position or block position”, “current vertical line number * vertical line length” is set as the upper left document Y coordinate.
In steps S527, 529, and 531, the document area width (WW) is set as the document width.
In step S533, the document area height (WH) is set as the document height.
In step S535, the display document coordinates are passed to the request source.
32A, 32B, 32C, 32D, and 32E are explanatory diagrams showing states of tables and display document coordinates in the flowchart of the display document area calculation control operation.

Next, the scroll control operation will be described with reference to FIG.
Here, scroll display is performed with the scroll direction as an input value.
In steps S601, 603, 605, and 607 of FIG. 33, if the current vertical line number is smaller than the number of vertical scroll lines, scroll display is performed by moving the vertical scroll box position downward (LINEDOWN) via the API provided by the OS. . In steps S609, 611, 613, and 615, when the current vertical line number is not 0 (first), scroll display is performed by moving the vertical scroll box position upward (LINEUP).
In steps S617, 619, 621, and 623, when the current horizontal line number is smaller than the number of horizontal scroll lines, scroll display is performed by moving the horizontal scroll box position to the right (LINERIGHT) via the API provided by the OS.
In steps S625, 627, 629, and 631, when the current horizontal line number is not 0 (first), scroll display is performed by moving the horizontal scroll box position to the left (LINELEFT).
34 (a), (b), and (c) are explanatory diagrams showing input values and states of the respective tables in the flowchart of the scroll control operation.

Next, the document coordinate display control operation will be described with reference to FIG.
Here, a document whose document coordinates are near the upper left corner of the document area is displayed with the document coordinates as input values.
In steps S701 and S703 in FIG. 35, if the horizontal scroll type is not “unspecified position”, a horizontal line number (H) corresponding to the document X coordinate is calculated.
H = document X coordinate / horizontal line length If the vertical scroll type is not “unspecified position” in step S705, the vertical line number (V) corresponding to the document Y coordinate is calculated.
V = document Y coordinate / vertical line length In the case of the horizontal scroll type “unspecified position” in steps S707, 709, 711, the horizontal line number that first satisfies “document X coordinate ≧ horizontal line coordinate” is obtained from the horizontal line coordinate table. Search for.

In steps S713, 715, 717, 719, and 721, in the case of the vertical scroll type “unspecified position”, the vertical line number that first satisfies “document Y coordinate ≧ vertical line coordinate” is searched from the vertical line coordinate table.
In steps S723, 725, 727, 729, 730, and 731, the horizontal scroll box is moved to the right until the horizontal line number (H) corresponding to the document X coordinate is reached.
In steps S733 and 735, the horizontal scroll box is moved to the left until the horizontal line number (H) corresponding to the document X coordinate is reached.
In steps S737, 739, 741, 743, and 745, the vertical scroll box is moved downward until the vertical line number (V) corresponding to the document Y coordinate is reached.
In steps S747, 749, and 751, the vertical scroll box is moved upward until the horizontal line number (V) corresponding to the document Y coordinate is reached.
36 (a), (b), (c), (d), and (e) are explanatory diagrams showing the input values and the state of each table in the flowchart of the document coordinate display control operation, respectively.
Next, the position detection end control operation will be described with reference to FIG.
In step S801 in FIG. 37, as the position detection end control, the transparent window having the screen size is deleted, the position detection control is ended, and the application program can be operated.

Next, the registration file start operation will be described with reference to FIG.
Here, the file name is displayed with the file name registered in the position detection registration file as an input value, and position detection control is started.
In step S901 in FIG. 38, a program for displaying a file with a file name and a file path is activated to display the file.
In step S903, "current vertical line number detection" processing is executed, the current vertical line number of the display document is acquired, and set in the position detection control table.
In step S905, "current horizontal line number detection" processing is executed, the current horizontal line number of the display document is acquired, and set in the position detection control table.

In step S907, the document area identification (window handle) is acquired from the document area window information (window class, window title) and set in the position detection control table.
In step S909, the vertical scroll bar identification (window handle) is acquired from the vertical scroll bar information (window class, window title) and set in the position detection control table.
In step S911, the horizontal scroll bar identification (window handle) is acquired from the horizontal scroll bar information (window class, window title) and set in the position detection control table.
In step S913, scroll information is input and set in the scroll information table.

If the vertical scroll type is “unspecified position” in step S915, the vertical line coordinate table is input and set in the vertical line coordinate table.
If the horizontal scroll type is “unspecified position” in step S917, the horizontal line coordinate table is input and set in the horizontal line coordinate table.
In step S919, a “vertical line display” process is executed to display the current vertical line number position.
In step S921, a “horizontal line display” process is executed to display the current horizontal line number position.
In step S923, full-screen size transparent window display and position detection control are started.
39A, 39B, and 39C are explanatory diagrams showing the input value, the position detection registration file, and the position detection control table in the flowchart of the registration file start operation, respectively.

Next, the current vertical line number detection operation will be described with reference to FIG.
In step S1001 of FIG. 40, the “document display fixing” process is executed so that the screen is not scroll-displayed by moving the scroll box position.
In step S1003, the current position (B) of the vertical scroll box is acquired.
In steps S1005, 1007, and 1009, if the current position is “0”, the vertical scroll box position is moved downward.
If the current position is not “0” in step S1011, the vertical scroll box position is moved upward.
In steps S1013, 1015, and 1017, the movement is repeated until the vertical scroll box position becomes a value different from the current position (B), and the number of lines is acquired.
In step S1019, the vertical scroll box position is moved to the top.
In steps S1021, 1023, 1025, 1027, 1029, and 1031, the total number of lines until the vertical scroll box position becomes “1” is acquired.
In step S1033, current vertical line number = total number of lines−number of lines, and in step S1035, the total number of lines until the vertical scroll box position becomes the current position (B) is acquired.
In step S1037, the current vertical line number = total number of lines + (number of lines−1), and in step S1039, the “document display fixed release” process is executed so as to perform scroll display.
In step S1041, the current vertical line number is passed to the request source.

Next, the current horizontal line number detection operation will be described with reference to FIG.
In step S1101 of FIG. 41, the “document display fixing” process is executed so that the screen is not scroll-displayed by moving the scroll box position.
In step S1103, the current position (B) of the horizontal scroll box is acquired.
In steps S1105, 1107, and 1109, if the current position is “0”, the horizontal scroll box position is moved to the right.
If the current position is not “0” in step S1111, the horizontal scroll box position is moved to the left.
In steps S1113, 1115, and 1117, the movement is repeated until the horizontal scroll box position becomes a value different from the current position (B), and the number of lines is acquired.
In step S1119, the horizontal scroll box position is moved to the left end. In steps S1121, 1123, 1125, 1127, 1129, and 1131, the total number of lines until the horizontal scroll box position becomes “1” is acquired.
In step S1133, the current horizontal line number = total number of lines−number of lines. In step S1135, the total number of lines until the horizontal scroll box position becomes the current position (B) is acquired.
In step S1137, the current horizontal line number = total number of lines + number of lines−1. In step S1139, the “document display fixed release” process is executed and scrolled.
In step S1141, the current horizontal line number is passed to the request source.

Next, the vertical line position display operation will be described with reference to FIG.
Here, the vertical line number is moved to the same position as the input value.
In step S1201 of FIG. 42, the “document display fixing” process is executed so that the screen is not scroll-displayed by moving the scroll box position.
In step S1203, the vertical scroll box position is moved to the top.
In steps S1205, 1207, 1209, and 1211, the downward movement is repeated until the vertical scroll box position becomes the same as the vertical line number.
In step S1213, a “document display fixed release” process is executed and scroll display is performed.
FIG. 43 is an explanatory diagram showing the state of input values in the flowchart of the vertical line position display operation.

Next, the horizontal line position display operation will be described with reference to FIG.
Here, the horizontal line number is moved to the same position as the input value.
In step S1301 of FIG. 44, the “document display fixing” process is executed so that the screen is not scroll-displayed by moving the scroll box position.
In step S1303, the horizontal scroll box position is moved to the left end.
In steps S1305, 1307, 1309, and 1311, the rightward movement is repeated until the horizontal scroll box position becomes the same as the horizontal line number.
In step S1313, a “document display fixed release” process is executed and scroll display is performed.
FIG. 45 is an explanatory diagram showing the state of input values in the flowchart of the horizontal line position display operation.

Next, the vertical position detection information detection operation will be described with reference to FIG.
In step S1401 of FIG. 46, “vertical line length drawing method detection” processing is executed to acquire the vertical line length.
If the detection is successful in steps S1403 and 1405, the vertical line length detection method is set to the drawing method.
If it is not successful in step S1407, the vertical line length detection method is set to the image extraction method.
In step S1409, a “vertical line coordinate detection” process is executed to detect the document Y coordinate for each line.
In step S1411, the number of vertical scroll lines is set in the scroll information table.
In step S1413, the vertical line length (line 1 document Y coordinate-line 0 document Y coordinate) is calculated for all lines.
If the vertical line lengths are not the same in steps S1415 and 1417, the vertical scroll type is set to “unspecified position”.

In step S1419, if they are the same, the vertical line length is set in the scroll information table.
In steps S1421, 1423, and 1425, the vertical scroll box position (P0) before movement and the vertical scroll box position (P1) after movement are acquired.
In steps S1427 and 1429, if the vertical line length and the vertical scroll box position difference (P1-P0) are the same, the vertical scroll type is set to “pixel position”.
In step S1431, the maximum value of the vertical scroll box position is acquired from the vertical scroll bar information.
In steps S1433 and 1435, when the number of vertical scroll lines is the same as the maximum value, the vertical scroll type is set to “line position”.
If the number of vertical scroll lines is greater than the maximum value in step S1437, the vertical scroll type is set to “block position”.
47 (a) and 47 (b) are explanatory diagrams showing the state of each table in the flowchart of the vertical position detection information detection operation.

Next, the horizontal position detection information detection operation will be described with reference to FIG.
In step S1501 of FIG. 48, a “horizontal line length drawing method detection” process is executed to acquire the horizontal line length.
If the detection succeeds in steps S1503 and 1505, the horizontal line length detection method is set to the drawing method.
In step S1507, if not successful, the horizontal line length detection method is set to the image extraction method.
In step S1509, a “horizontal line coordinate detection” process is executed to detect the document X coordinate for each line.
In step S1511, the number of horizontal scroll lines is set in the scroll information table.
In step S1513, the horizontal line length (line 1 document X coordinate-line 0 document X coordinate) is calculated for all lines.

In steps S1515 and 1517, if the horizontal line length is not the same, the horizontal scroll type is set to “unspecified position”.
In step S1519, if they are the same, the horizontal line length is set to the horizontal line length. In steps S1521, 1523, and 1525, the horizontal scroll box position (P0) before movement and the horizontal scroll box position (P1) after movement are acquired.
In steps S1527 and 1529, if the horizontal line length and the horizontal scroll box position difference (P1-P0) are the same, the horizontal scroll type is set to “pixel position”.
In step S1531, the maximum value of the horizontal scroll box position is acquired from the horizontal scroll bar information.
If the number of horizontal scroll lines and the maximum value are the same in steps S1533 and 1535, the horizontal scroll type is set to “line position”.
If the number of horizontal scroll lines is greater than the maximum value in step S1537, the horizontal scroll type is set to “block position”.
49A and 49B are explanatory diagrams showing the states of the tables in the flowchart of the horizontal position detection information detection operation.

Next, the vertical scroll line number detection operation will be described with reference to FIG.
In step S1601 of FIG. 50, the “document display fixing” process is executed so that the screen is not scroll-displayed by moving the scroll box position.
In step S1603, the vertical scroll box position is moved to the top.
In step S1605, the maximum value of the vertical scroll box position is acquired from the vertical scroll bar information.
In steps S1607, 1609, 1611, 1613, and 1615, the downward movement of the vertical scroll box is repeated until the vertical scroll box position reaches the maximum value, and the number of vertical scroll lines is detected.
In step S1617, a “document display fixed release” process is executed and scroll display is performed.
In step S1619, the number of vertical scroll lines is passed to the request source.

Next, the horizontal scroll line number detection operation will be described with reference to FIG.
In step S1701 of FIG. 51, the “document display fixing” process is executed so that the screen is not scrolled by moving the scroll box position.
In step S1703, the horizontal scroll box is moved to the left end.
In step S1705, the maximum value of the horizontal scroll box position is acquired from the horizontal scroll bar information.
In steps S1707, 1709, 1711, 1713, and 1715, the horizontal scroll box rightward movement is repeated until the horizontal scroll box position reaches the maximum value, and the number of horizontal scroll lines is detected.
In step S1717, a “document display fixed release” process is executed and scroll display is performed.
In step S1719, the number of horizontal scroll lines is passed to the request source.

Next, the vertical line coordinate detection operation will be described with reference to FIG.
Here, vertical line coordinates are detected and set in the vertical line coordinate table using the vertical line length detection method as an input value.
In step S1801 in FIG. 52, the vertical scroll box position is moved to the top.
In step S1803, the maximum value of the vertical scroll box position is acquired from the vertical scroll bar information.
In steps S1805, 1807, and 1809, the sum of the vertical line lengths up to the line number is set as a vertical line coordinate in the vertical line coordinate table.
In steps S1811, 1813, 1815, and 1817, if the vertical line length detection method is “drawing method”, “vertical line length drawing method detection” processing is executed to detect the vertical line length.
If the vertical line length detection method is “image extraction method” in step S1819, “vertical line length image extraction method detection” processing is executed to detect the vertical line length.
In steps S1821, 1823, and 1825, the number of vertical scroll lines is set in the vertical line coordinate table.
FIGS. 53A and 53B are explanatory diagrams showing the input values and the state of the vertical line coordinate table in the flowchart of the vertical line coordinate detection operation, respectively.

Next, the horizontal line coordinate detection operation will be described with reference to FIG.
Here, horizontal line coordinates are detected and set in the horizontal line coordinate table using the horizontal line length detection method as an input value.
In step S1901 in FIG. 54, the horizontal scroll box position is moved to the left end.
In step S1903, the maximum value of the horizontal scroll box position is acquired from the horizontal scroll bar information.
In steps S1905, 1907, and 1909, the sum of the horizontal line lengths up to the line number is set as a horizontal line coordinate in the horizontal line coordinate table.
If the horizontal line length detection method is “drawing method” in step S1917, the “horizontal line length drawing method detection” process is executed to detect the horizontal line length.
If the horizontal line length detection method is “image extraction method” in step S1919, “horizontal line length image extraction method detection” processing is executed to detect the horizontal line length.
In steps S1921, 1923, and 1925, the number of horizontal scroll lines is set in the horizontal line coordinate table.
FIGS. 55A and 55B are explanatory diagrams showing the input value and the state of the horizontal line coordinate table in the flowchart of the horizontal line coordinate detection operation, respectively.

Next, the vertical line length drawing method detection operation will be described with reference to FIG.
In step S2001 of FIG. 56, the upper left coordinates (WX, WY) of the document area are acquired by the document area identification (window handle).
In step S2003, the document area width (WW) and height (WH) are acquired from the document area identification (window handle).
In step S2005, the start point coordinates (X0, Y0) for horizontal line drawing are calculated.
X0 = WX + WW / 2-L (horizontal line length) / 2
Y0 = WY + WH-32
In step S2007, a horizontal line of (4) start point coordinates (X0, Y0), length (L), thickness (1), and color (R: 244, G: 1, B: 2) is drawn.
In step S2009, the vertical scroll box is moved downward.
In steps S2011, 2013, and 2015, a document display area having start point coordinates (X, Y), width (L), and height (1) within the range of (X = X0) and (WY <Y ≦ Y0-1). An image is acquired, and an image that matches all the horizontal line colors (R: 244, G: 1, B: 2) is searched.
In steps S2017, 2019, and 2021, if the line cannot be detected, the vertical line length is set to “0”.
In step S2023, the Y-direction movement amount (Y0-Y) of the horizontal line is set as the vertical line length.
In step S2025, the vertical line length is passed to the request source. FIG. 57 is an explanatory diagram showing the state of the position detection control table in the flowchart of the vertical line length drawing method detection operation.

Next, the horizontal line length drawing method detection operation will be described with reference to FIG.
In step S2101 in FIG. 58, the upper left coordinates (WX, WY) of the document area are acquired by document area identification (window handle).
In step S2103, the document area width (WW) and height (WH) are acquired from the document area identification (window handle).
In step S2105, the start point coordinates (X0, Y0) for vertical line drawing are calculated.
X0 = WX + WW-32
Y0 = WY + WH / 2−L (vertical line length) / 2
In step S2107, a vertical line of starting point coordinates (X0, Y0), length (L), thickness (1), and color (R: 244, G: 1, B: 2) is drawn.
In step S2109, the horizontal scroll box is moved to the right.

In steps S211, 2112, and 2115, a document display area image having start point coordinates (X, Y), width (1), and height (L) is acquired within the range of (WX <X ≦ X0-1), and the horizontal line color Search for images that all match (R: 244, G: 1, B: 2).
If it cannot be detected in steps S2117, 2119, and 2121, the horizontal line length is set to zero.
In step S2123, the vertical line movement amount in the X direction (X0-X) is set as the horizontal line length.
In step S2125, the horizontal line length is passed to the request source.
FIG. 59 is an explanatory diagram showing the state of the position detection control table in the flowchart of the horizontal line length drawing method detection operation.

Next, the vertical line length image extraction method operation will be described with reference to FIG.
In step S2201 of FIG. 60, the upper left coordinates (WX, WY) of the document area are acquired by the document area identification (window handle).
In step S2203, the document area width (WW) and height (WH) are acquired from the document area identification (window handle).
In steps S2205 and 2207, the start point coordinates (X0, Y0) of the display image are calculated.
X0 = WX
Y0 = WY + WH-Maximum vertical line length (MVL)
In step S2209, a document display area image having start point coordinates (X0, Y0), width (WH), and height (1) is acquired as a reference image.
In step S2211, 2213, a document display area image having start point coordinates (X, Y), width (WH), and height (1) is acquired within the range of (X = X0) and (Y0 + 1 ≦ Y <Y0 + MVL). An image that matches all the reference images is searched.
In Steps S2215, 2217, 2219, 2221, and 2223, if there is a display image that matches the reference image, it is searched whether there is a display image that matches the reference image in the range of (Y0> WY).
In step S2225, if there is a display image that matches the reference image, the vertical line length is set to “0”.
In step S2227, the vertical scroll box position is moved downward.
In steps S2229, 2231, 2233, and 2235, within the range of (X = X0) and (Y0-MVL <Y ≦ Y0-1), the start point coordinates (X, Y), the width (WH), and the height (1) The document display area image is acquired, and an image that matches all the reference images is detected.
In step S2237, the movement amount (Y0-Y) of the reference image in the Y direction is set as the vertical line length.
In step S2239, the vertical line length is passed to the request source.
FIG. 61 is an explanatory diagram showing the state of the position detection control table in the flowchart of the vertical line length image extraction method detection operation.

Next, the operation of the horizontal line length image extraction method will be described with reference to FIG.
In step S2301 of FIG. 62, the upper left coordinates (WX, WY) of the document area are acquired by document area identification.
In step S2303, the document area width (WW) and height (WH) are acquired by document area identification.
In steps S2305 and 2307, the start point coordinates (X0, Y0) of the display image are calculated.
X0 = WX + WW- Maximum horizontal line length (MHL)
Y0 = WY
In step S2309, a document display area image having start point coordinates (X0, Y0), width (1), and height (WH) is acquired as a reference image.
In step S2313, a document display area image having start point coordinates (X, Y), width (1), and height (WH) is acquired within the range of (X0 + 1 ≦ X <X0 + MHL), (Y = Y0), and the reference image Search for images that all match.

In steps S2315, 2317, 2319, 2321, and 2323, if there is a display image that matches the reference image, it is detected whether there is a display image that matches the reference image in the range of (X0> WX).
In step S2325, if there is a display image that matches the reference image, the horizontal line length is set to “0”.
In step S2327, the horizontal scroll box position is moved to the right.
In steps S2329, 2331, 2333, and 2335, within the range of (X0−MHL <X ≦ X0−1) and (Y = Y0), the start point coordinates (X, Y), the width (1), and the height (WH) The document display area image is acquired, and an image that matches all the reference images is searched.
In step S2337, the X-direction movement amount (X0-X) of the reference image is set as the horizontal line length.
In step S2339, the horizontal line length is passed to the request source.
FIG. 63 is an explanatory diagram showing the state of the position detection control table in the flowchart of the horizontal line length image extraction method detection operation.

Next, the document display fixing operation will be described with reference to FIG.
In step S2401 of FIG. 64, the upper left coordinates, width, and height of the document area are acquired by document area identification.
In step S2403, an image of the document area is acquired.
In step S2405, a transparent window having a full screen size is displayed on the document.
In step S2407, the document image is displayed at a position overlapping the document area on the transparent window.
FIG. 65 is an explanatory diagram showing the state of the position detection control table in the flowchart of the document display fixing operation.
Next, the document display fixing release operation will be described with reference to FIG.
In step S2501 of FIG. 66, the document image is deleted.
In step S2503, the transparent window is deleted.
FIG. 67 is an explanatory diagram showing the state of the position detection control table in the flowchart of the document display fixing release operation.

[Second Embodiment]
Described above is a technique for grasping the display state of a document on a specific device (overwrite information processing device) including a display function and displaying the same content on another display device or on the display device of another terminal. did.
The second aspect of the present invention relates to an electronic conference system. As shown in the system configuration diagram of FIG. 68, the electronic conference system has various conference forms such as (a) multipoint conference, (b) room conference, and (c) desk conference. A plurality of connected terminal devices are used, and the display of each of these terminal devices must be synchronized. As a terminal device capable of display synchronization for a conference system described below, an apparatus including the configuration of the overwriting information processing apparatus described in the first embodiment is used.
Subsequently, an embodiment (second embodiment) of an electronic conference (multi-point conference) system according to the present invention that uses position detection control in a display change operation including some functions that have not been described so far will be described.
In the second embodiment, when a document in a specific device is overwritten, and when the overwritten content is changed (erased and / or additionally overwritten), a technique for detecting the overwritten content The position detection control of the display including this and the position detection result are notified to other connected devices (including transmission via WAN etc.), and the detection contents are reflected on the display unit of the other devices. Control to display the same contents in a document (display synchronization).
Hereinafter, an electronic conference system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 69 shows a schematic block diagram of the PC conference system of this embodiment. In the electronic conference system of this embodiment, as shown in FIG. 68 (a) as a multipoint conference, conference participation locations (conference participation terminals) are connected via the WAN. However, the technique described below is a technique that can also be applied to the case where display contents are synchronized between a plurality of connected devices in the (b) room meeting or (c) desk meeting in FIG.

The second embodiment is an electronic conference system that acquires a document as conference information and position information of overwriting the document. Further, in order to save the display document position and the overwrite position in association with each other, saving the screen image for displaying the document and the overwrite is executed. When overwriting and scrolling display, the overwriting gets in the way, so the screen image is saved and the overwriting is erased.
The apparatus used in this embodiment has the same configuration as the various terminals already described. An electronic conference system is configured by appropriately combining the various terminals described so far for use by a large screen device for conference participants to view or for each conference participant who wants to conference separately from each other. Yes. In addition, although not shown in figure, what is connected via LAN or the internet network may be mixed as each terminal device.

In particular, in FIG. 68, the tablet PC 1 is connected to a remote value, PC terminal in the LAN via a LAN (Local Area Network), a router, and a WAN (Wide Area Network). A PC terminal having a large display-integrated coordinate input device (PDP, liquid crystal rear projector, etc.) as a display is installed in a conference room or the like and is shared by a plurality of people. The tablet PC 2 is connected to another PC terminal from the outside by a wireless modem.
The tablet PC 3 outputs an image to the projector, displays it on the screen, and shares the screen. A general PC is less user-friendly than a pen operation, but is operated with a mouse. The screen unification method in the PC conference at the multi-point having the above configuration is the right scope. A room conference using a tablet PC by wireless LAN connection is also effective.
In addition, PC operation data is effective via a network, and voice is effective for a PC conference on a desk using a telephone. The document creation program for creating the conference document includes “page information” (“page number” in Microsoft Word, “sheet number” in Excel, “slide number” in PowerPoint, etc.), “display magnification” (50%, 100). %)), A display file name acquisition function shall be provided. This feature is currently provided by Microsoft Word, Excel, and PowerPoint. Further, it is assumed that the document creation program exists in each PC terminal. These programs are registered trademarks of Microsoft Corporation in the US and are commercially available.

Since each of these terminal devices has been described in the previous embodiment, overlapping description will be omitted below. Each terminal device uses the same function as the document position detection device described in the first embodiment, the overwriting function for the conference document, and / or the overwriting content performed on another device for which the description is suspended as the display content of the own device. It is assumed that the function to be reflected is provided according to the role of each device.
When the screen including the overwriting is scrolled or the like, it is necessary to redraw if the scroll display is necessary to the extent that the overwriting position remains, and the previous overwriting is necessary. Alternatively, the same contents (including overwriting) of the same document (s) are always displayed on all the terminals participating in the conference after displaying the saved image and returning to the document display to display the scroll part. The same location) is displayed and supports smooth electronic conferences.

[PC conference control]
The PC conference control method will be described with reference to FIG.
In the illustrated system, a plurality of PCs (conference participation terminals) are connected by a WAN. In the figure, only two general PCs (PC1, PC2) are shown, but other conference participation terminals may be connected. Further, as a conference participation terminal, a large-screen display device for cooperation such as the projector described above may be connected. Reference numeral 200 denotes a PC screen, and 201 denotes a document creation program that operates as a Windows (registered trademark) standard application (APL) program installed in the PC. In the following description, for the sake of convenience, one PC (PC1 shown in the upper part of the figure) in the figure is an operation right PC, and the other PC is a non-operation right PC. Note that the operation right PC is assigned to an appropriate device during the conference, but can be temporarily assigned to other conference participation terminals according to the progress of the conference.
The PC conference control program 202 installed in each conference participation terminal PC1 and PC2 includes an initialization control unit 204, a display monitoring control unit 205, an overwrite control unit 206, and a network control unit 207. The control unit 206 and the network control unit 207 operate independently.

The “initialization control unit 204” initializes the PC conference control program 202, inputs a conference participation PC, activates the display monitoring control unit 205, the overwrite control unit 206, and the network control unit 207, and starts a PC conference. Then, the “display monitoring control unit 205” detects the conference material file displayed by the document creation program, and distributes the conference file to the non-operation right PC.
Further, the display position, size, vertical scroll box position, and horizontal scroll box position of the document creation program window are monitored, and if there is a change, the change is notified to the non-operation right PC. The “overwrite control unit 206” draws a mark or minutes on the conference material by handwriting operation and saves the drawing data in an overwrite file. The “network control unit 207” receives and controls packet data transmitted from other PCs participating in the conference (transmission of packet data is performed on the network as necessary by each control unit).
The control data 208 is input / output by the initialization control unit 204, the display monitoring control unit 205, the overwrite control unit 206, and the network control unit 207 to control the PC conference. The conference file 209 is a conference material displayed by the document creation program and is automatically distributed from the operation right PC in addition to the operation right PC. The overwrite file 210 stores overwrite data drawn at the time of a PC conference.

[PC conference control data]
Next, the PC conference control data will be described with reference to FIG. Various packet data 211 is data transmitted / received via the network (LAN 10), and control data 208 is input / output by the independently operated network control unit 207, display monitoring control unit 205, and overwrite control unit 206. Data are shown.
The conference start packet 212 is created by the initialization control unit that hosts the conference and transmitted to the participating PCs to start the conference. The conference file packet 213 is created by the display monitoring control unit 205 of the PC that hosts the conference, distributed to the conference participation PCs, starts the document creation program 91, and displays it as conference material.
The PC screen unification packet 214 is used by the display supervisory control unit with the operation right to monitor changes in the window display position, size, vertical scroll box position, and horizontal scroll box position of the document creation program. To do. The overwrite packet 215 is transmitted when the overwrite control unit 206 overwrites the conference material and displays the overwrite on another PC. The overwrite erase packet 216 is transmitted when the overwrite control with the operation right is overwritten and erases the overwrite of another PC. The operation right packet 217 is used to turn over the conference material page, request the overwriting operation right to the host PC, and notify the transfer of the operation right. The conference is ended by the host PC by the conference end packet 218 and the absence is notified by the participating PC.
The PC conference is controlled by the conference control data 219. The current display state is indicated by the PC screen unified control data 220. Overwriting is controlled by overwriting control data 221.

[Overwrite operation]
Here, the overwriting operation will be described with reference to FIG. As shown in the figure, the handwritten character can be drawn with the “pen” button of the overwrite toolbar, and the handwritten character can be deleted with the “erase” button (see FIG. 71 [1]).
The “APL” button allows normal program operation. At the time of overwriting and erasing, a transparent window having a full screen size (transparent background is displayed) is created and displayed (see FIG. 71 [2]). At the time of a character drawing operation with the pen (mouse), lines are sequentially drawn on the transparent window between the drag coordinates in the color and thickness set for the pen. The drawing coordinates (drag coordinates at the time of overwriting operation) are converted into document coordinates (scroll space coordinates) and saved as stroke data in the overwriting file. (See FIG. 71 [3]).
The drawing coordinates are transmitted to the conference participation PC for each stroke by the overwrite packet 215 (see FIG. 71 [4]). The PC that has received the overwrite packet 215 draws a line between the drawing coordinates sequentially with the pen color and the pen thickness (see FIG. 71 [5]). The drawing coordinates are converted into document coordinates and stored as stroke data in the overwrite file 210 (see FIG. 71 [6]).

Next, the overwriting stroke data will be described with reference to FIG. In FIG. 72, (X0, Y0), (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), (X5, Y5), (X6, Y6) are one stroke. Indicates the drag trajectory. For handwritten character drawing, a line is drawn with a set color and thickness between two points ((X0, Y0)-(X1, Y1), (X1, Y1)-(X2, Y2),...)). Draw by
The drag trajectory coordinates by the character drawing operation are converted into document coordinates and stored in the overwriting file in stroke units (pen down (mouse button ON) to pen up (mouse button OFF)). To erase the handwritten character, a rectangular area having a size set for the eraser with the pen touch point as the center is displayed as an eraser cursor. The eraser cursor is overwritten in the rectangle with the mouse coordinate at the center and the eraser cursor length on one side. By dragging onto the handwritten character, the handwritten character in the rectangular area is erased.
A state in which the (X3, Y3)-(X4, Y4) line segment and the (X4, Y4)-(X5, Y5) line segment are erased and divided into two strokes by the handwritten character erasing operation is shown. Delete the deleted stroke data from the overwrite file. When a stroke is divided into a plurality of strokes by erasing, it is divided and stored into a plurality of strokes.
When a PC that has received the overwrite file via the network receives the overwrite packet, it draws the drawing coordinates in the pen color and pen thickness in order, converts the drawing coordinates into the document coordinates, and saves it in the overwriting file. Then, overwriting and redrawing will be performed as described later.

[Start meeting hosting]
A conference host start operation and the like by the PC hosting the conference will be described. A conference participation PC is designated by the host PC, and the conference is started by connecting. Other participating PCs wait for connection from the host PC.
A method for starting conference sponsorship will be described with reference to FIG. The PC conference control program 202 is activated by means provided by the OS, such as double-clicking on a desktop icon, and the “conference selection toolbar” is displayed.
When the “sponsor” button 203 is pressed, a “conference partner dialog box” is displayed (see FIG. 73 [1]). The network address of the conference partner is input and registered (see FIG. 73 [3]). The network address of the registered conference partner is displayed (see FIG. 73 [4]). Drag the network address to delete it. When the setting of the conference partner is completed (OK), the “conference start toolbar” is displayed (see FIG. 73 [5]).
The “overwrite toolbar” is displayed by the “start” button, and the conference is started (see FIG. 73 [6]). The network address of the PC participating in the conference is notified to the conference participating PC by the conference start packet 212 (see FIG. 73 [7]).

The conference material file path being displayed is acquired from the title bar of the APL window that displays the conference material, the file data is input, and the conference file packet 213 is distributed to the conference participation PCs (see FIG. 73 [8]). The upper left origin coordinate, width, height, horizontal scroll box position, and vertical scroll box position of the APL window displaying the conference material are acquired, and the screen display of the conference participating PCs is unified by the PC screen unification packet 214 (FIG. 73 [9] ]reference).
Note that the PC operation rights (page turning of conference materials, scrolling display, overwriting, etc.) are given to the host PC at the beginning of the conference, but the PC operation rights are assigned to other participating PCs as will be described later. Can be moved temporarily. Incidentally, a mode in which a plurality of PCs can be operated simultaneously is possible, but since it is not possible to know which PC display content is displayed as conference material, in this embodiment, the display content of the PC having the operation right is used as conference material. .
Note that the “APL” button allows the user to leave the conference material display screen and view another page independently. However, when the user returns to the conference material again, the display contents of the PC with the operation right are displayed.

[Conference start operation]
Next, a conference participation start method will be described with reference to FIG. The PC conference control program 202 is activated by means provided by the OS, such as double-clicking on the desktop icon, and the “conference selection toolbar” is displayed (see FIG. 74 [1]).
The “participation” button displays a “conference connection dialog box”, waits for reception of the conference start packet, and erases it by reception (see FIG. 74 [2]). The conference member information is stored in the conference control data 219 (see FIG. 74 [3]).
Upon reception of the conference file packet 213, a file is created, a file path is passed, a program for displaying conference material is started, and conference material is displayed (see FIG. 74 [4]). With the PC screen unified packet 214, the display of the APL window for displaying the conference is corrected to the upper left origin coordinates, width, and height, and the position is corrected to the horizontal scroll box position and vertical scroll box position (see FIG. 74 [5]). The operation right toolbar is displayed (see FIG. 74 [6]).

[PC screen unification method]
Next, the PC screen unification method will be described with reference to FIG. In the conference participation terminal device having the operation right, the following processing is executed at appropriate intervals.
Changes in the upper left origin coordinates (X, Y), width (W), height (H), horizontal scroll box position (PH), and vertical scroll box position (PV) of the window displayed on the screen are monitored. That is, the APL window displaying the conference document is searched, and the window title, window class, window upper left origin coordinate (X, Y), window width (W), and window height (H) are acquired.
A document display window (generally a child window of the APL window having the maximum size of the APL window) is searched, and document display window information (window class, window title) is acquired. When the document display window is in the “WS_VSCROLL” window style, the vertical scroll bar window class and the window style are set to “0”. If not, a vertical scroll bar adjacent to the document display window is searched, and vertical scroll bar window information (window class, window title) is acquired.
When the document display window has the “WS_HSCROLL” window style, the horizontal scroll bar window class and the window style are set to “0”. If not, the horizontal scroll bar adjacent to the document display window is searched, and the horizontal scroll bar window information (window class, window title) is acquired.
Based on the horizontal scroll bar information, the vertical scroll bar information, and the document display window information, the upper left document coordinates (PH, PV) of the document being displayed are calculated. If there is a change, the PC screen unified packet 214 is created and transmitted to the conference participation PC (see FIG. 75 [1]).

The operation result is reflected in the drawing state in the PC received via the network. Upon receiving the PC screen uniform packet, the conference participant PC corrects the display of the window to the upper left origin coordinates (X, Y), width (W), height (H), horizontal scroll box position (PH), vertical scroll box position ( The scroll box is moved to (PV) (see FIG. 75 [2]).
That is, the document coordinates at the upper left corner of the document display area (PH: upper left upper document X coordinate, PV: upper left upper document Y coordinate).
The horizontal scroll bar information is acquired, the horizontal scroll length (HL) is converted into the pixel length (XL), the distance (XP) from the left end of the document is calculated by the following, and the OS provides the horizontal scroll box to the position of XP To move.
XP = XL / 2− | PH |
* | | Means an absolute value The scroll line length (LL) of the information provided by the OS (LL) The number of scroll lines (LN) is calculated as follows, and the vertical scroll box to the position corresponding to the number of scroll lines is calculated by the I / F provided by the OS. Moving.
LN = PV / LL

On the other hand, a processing method for displaying a document at a specific position, which is executed on the other conference participation PC connected to the network, will be described with reference to FIG.
The APL window is displayed at the upper left origin coordinate (X, Y), window width (W), and window height (H).
Document display window information (window class, window title) is acquired from the document display window class and window title.
The vertical scroll bar information is obtained from the vertical scroll bar window class and window title, the vertical scroll box position corresponding to the upper left document Y coordinate (PV) is calculated, and the vertical scroll box is moved to the same position.
Searches the horizontal scroll bar attached to the document display window, the adjacent horizontal scroll bar, obtains horizontal scroll bar information, calculates the horizontal scroll box position corresponding to the upper left document X coordinate (PH), and moves the horizontal scroll box to the same position. Moving.
As a result, the PC screen display is kept the same as the operation right PC.

[Overwrite redraw]
When there is an overwriting, the above-described overwriting file is overwritten and redrawn in the PC that has received the above-described overwriting file through the network (and in its own department, if necessary). Executed. The overwrite redrawing method will be described with reference to FIG.
Next, an overwriting redrawing method will be described with reference to FIG. Changes in the horizontal scroll box position, vertical scroll box position, window display upper left origin coordinate, width, and height are monitored, and if there is a change, a PC screen unified packet 214 is transmitted (see FIG. 77 [1]). When there is a display change, obtain horizontal scroll bar information, vertical scroll bar information, document display window information, calculate the display area document space, convert the document coordinates stored in the overwrite file to display magnification, and display the document coordinates as the display area If it is within the document space, it is redrawn by converting it to screen coordinates and connecting the coordinates with a stroke color and stroke thickness (see FIG. 77 [2]). Then, a “PC screen unified packet” is created and a screen update request is made.
Upon receiving the PC screen unified packet 214, the conference participation terminal corrects the display to the window position, width, and height based on the packet information, moves the scroll box to the vertical scroll box position, the horizontal scroll box position, the horizontal scroll box position, and the vertical scroll. The scroll space displayed on the screen is calculated from the box position, the overwrite data is searched from the overwrite file to the same space, and the document coordinates are converted into screen coordinates and displayed (see FIG. 77 [3]).

Next, the operation right transfer method will be described with reference to FIGS. 78 and 79. Here, the PC operation right (turning the page of the conference material, scrolling display, overwriting, etc.) is given to the host PC at the start of the conference at the beginning of the conference. When the participation PC becomes necessary as the conference progresses, the participating PC requests an operating right from the hosting PC using the “operating right” button (see FIG. 78 [1]). The operation right request by the “operation right (request) packet” is transmitted to the host PC (see FIG. 78 [2]).
When the host PC receives the operation right request packet, an “operation right dialog box” is displayed (see FIG. 78 [3]). The operation right is given by the “permit” button, and the operation right request is rejected by the “reject” button. When the host PC permits the operation right, an “operation right (movement) packet” is transmitted to all participating PCs (see FIG. 78 [4]).
When the operation right (transfer) packet is received, an “operation right transfer dialog box” is displayed and the operation right transfer is notified (see FIG. 78 [5]). When the operation right is acquired, the display is changed from the operation right toolbar to the overwrite toolbar, and when the operation right is lost, the display is changed from the overwrite toolbar to the operation right toolbar (see FIG. 78 [6]).
When the operation right request is rejected by the “reject” button, an “operation right (rejection) packet” is transmitted to the operation right request PC (see FIG. 79 [7]). When the operation right (rejection) packet is received, an “operation right rejection dialog box” is displayed (see FIG. 79 [8]).

[Overwrite edit]
Subsequently, an overwriting result editing process performed as necessary after the PC conference is described.
The overwrite editing method will be described with reference to FIGS. When the PC conference ends, a save confirmation dialog box for handwritten characters (graphics) overwritten during the PC conference is displayed (see FIG. 80 [1]).
The overwrite data is saved in the designated overwrite file (see FIG. 80 [2]). The overwritten conference material is displayed by starting the program that created the conference material (see FIG. 80 [3]). Start the overwrite editing program, display the open dialog box, and input the overwrite file for the conference material (see FIG. 80 [4]).
Displays the overwrite editing toolbar, obtains the upper left origin coordinates, width, height, horizontal scroll box position, and vertical scroll box position of the window displaying the conference material, calculates the scroll space (display area document space) on the screen, this space Overwrite to the file is retrieved and displayed from the overwrite file (see FIG. 81 [5]).
In other words, the document coordinates saved in the overwrite file are converted to display magnification, and if the document coordinates are within the display area document space, they are converted to screen coordinates, and the coordinates are drawn by connecting the stroke color and stroke thickness. . “Pen” erases and edits additional drawing, and “Erase” erases and edits. Save the edited override.

[Overwrite setting]
Next, the overwriting setting will be described with reference to FIG. From “Thickness”, the thickness of the line at the time of character drawing is selected (see FIG. 82 [1]). From “Color”, the color of the line at the time of character drawing is selected (see FIG. 82 [2]). The selected color and thickness are displayed in the viewer (see FIG. 82 [3]). Clicking on the eraser (pen touch) and dragging the displayed handle selects the size of the eraser cursor (see FIG. 82 [4]).
The conference system and the overall operation of the second embodiment have been described above. In the following, the process of the conference described so far will be described for each process by using a flowchart.

[Initialization control flow]
The initialization control operation will be described with reference to the flowchart of FIG. First, the conference control data is cleared to 0, the network address of the own PC is set (step S3171), and the PC screen unification control data is cleared to 0 (step S3172). Subsequently, overwriting control data is set to 0 clear, pen color, pen thickness, and default value for the eraser cursor (step S3173). Further, a conference selection toolbar is displayed (step S3174), and set while the toolbar is not controlled (step S3175). Mouse data from pen down (mouse button ON) to pen up (mouse button OFF) on the conference selection toolbar is input (step S3176).
It is determined whether the tool bar is not being controlled (step S3177). If the tool bar is not being controlled, the tool bar is set to be during the tool bar control (step S3178). Furthermore, the mouse coordinate button is selected and displayed (step S3179), and the click button information is saved (step S3180). The button has two display forms, a normal display and a selection display (inverted display, etc.).
If the toolbar is not being controlled in step S3177, it is further determined whether the mouse button is OFF (step S3181). If the mouse button is OFF, it is further determined whether the coordinates are on the click button (step S3182). If the coordinates are not on the click button, the tool bar control is set to OFF (step S3183). If the coordinates when the mouse button is OFF are different from the click button, the click button is returned to the normal display and the button function is not executed. (Step S3184).
If the coordinates are on the click button in step S3182, the conference selection toolbar is deleted (step S3185), and it is determined whether the click button is hosted (step S3186). If the click button is sponsored here, “conference sponsoring control” is executed by clicking the “sponsor” button (step S3187). If the click button is not sponsored in step S3186, it is further determined whether the click button is participating (step S3188). If it is participating, “conference participation control” is executed by clicking the “join” button (step S3189). .

[Conference Host Control Flow]
Next, the conference sponsoring control operation will be described with reference to the flowchart of FIG. First, the conference partner dialog box is displayed (step S3191), and the conference partner network address is input (step S3192). Subsequently, the host PC network address and the operation right PC network address are set as the own PC network address in the conference control data, the operation right is set, and the input conference partner network address is set as the participation PC network address (step S3193).
Further, a conference start toolbar is displayed (step S3194), and set while the toolbar is not controlled (step S3195). Mouse data from pen down (mouse button ON) to pen up (mouse button OFF) on the conference start toolbar is input (step S3196). Subsequently, it is determined whether the tool bar is not being controlled (step S3197). If the tool bar is not being controlled, the tool is set to be in the tool bar control by pen-down (step S3198), and the mouse coordinate button is selected and displayed (step S3199). ), The click button information is stored (step S3200).

The button has two display forms, a normal display and a selection display (inverted display, etc.). (If the toolbar is not being controlled in step S197, it is further determined whether the mouse button is OFF (step S3201). If the mouse button is OFF, it is further determined whether the coordinates are on the click button (step S3202). If the coordinates are not on the click button, the setting is made while the toolbar is not controlled (step S3203), and if the coordinates when the mouse button is OFF differ from the click button, the click button is returned to the normal display and the button function is executed. If there is a click button upper coordinate in step S3202, the conference start toolbar is deleted (step S3205), and it is determined whether the click button is started (step S3206).
If it is a start click button here, the “start” button activates “network control”, starts packet data reception (step S3207), activates “display monitoring control”, and starts normal monitoring of the window display. (Step S3208). Furthermore, “overwrite control” is activated, an overwrite toolbar (PC with operation right) is displayed (step S3209), a conference start packet is created from the conference control data, and transmitted to all participating PCs (step S3210).

[Conference participation control flow]
Next, the conference participation control operation will be described with reference to the flowchart of FIG. First, a conference connection dialog box is displayed, and it is waited for connection by the host PC (step S3211). Thereafter, “network control” is activated and packet data reception is started (step S3212).
Subsequently, it waits for the network control to receive the conference start packet and notify the conference start packet (step S3213). If the conference start packet is received, the conference connection dialog box is deleted by receiving the conference start packet (step S3214). Subsequently, “display monitoring control” is activated, window display normal monitoring is started (step S3215), “overwrite control” is activated, and an operation right toolbar (PC without operation right) is displayed (step S3216).

[Display monitoring control flow]
Next, the display monitoring control operation will be described with reference to the flowchart of FIG. First, it is determined whether the PC is the host PC (step S3221), and if it is the host PC, the APL program is started (step S3222), and the APL window information being displayed (the window where the APL title is displayed) is acquired. (Step S3223).
Subsequently, it is confirmed whether the APL window is displayed, and when the APL window is displayed, the document display window information is acquired (step S3224), the horizontal scroll box position is further acquired (step S3225), and the vertical display is performed. The scroll box position is acquired (step S3226). Then, it is determined whether it is a conference file display (step S3227), and the process waits for a file name to be set in the window caption information (step S3228).
Subsequently, the path of the file name of the caption information is searched, the conference file is input (step S3229), the conference file data and the conference file name are set in the “conference file packet”, and transmitted to all participating PCs (step S3230). Subsequently, it is determined whether it is an end notification (step S3231). In other words, the display monitoring control is ended by receiving the conference end packet from the host PC or by the end notification by the toolbar “end” button (step S3231).
If it is not an end notification, it is determined whether there is an operation right (step S3232), and APL window information (upper left origin coordinate, width, height) is acquired (step S3233). Then, the document display window information is acquired (step S3234), the horizontal scroll box position is acquired (step S3235), and the vertical scroll box position is acquired (step S3236).
Subsequently, whether or not there is a change is determined (step S3238). If there is a change, if the acquired APL window information and scroll box position are different from the PC screen unification control data, a “PC screen unification packet” is created based on the acquired value. It transmits to all other PCs (step S3239). Further, the acquired value is set in the PC screen unification control data (step S3240), and waits to monitor the display state of the APL window periodically (every 10 ms, every 100 ms, etc.) (step S324Z).

[Network control flow]
Next, the network control operation will be described with reference to the flowchart of FIG. The network control is ended by receiving a conference end packet from the host PC or by an end notification using a toolbar “end” button (step S3241).
If it is not an end notification, it waits for packet data to be received (step S3242). If packet data is received, it is determined whether it is a conference start packet (step S3243). If it is a conference start packet, upon reception of the “conference start packet”, the host PC, the operation right PC, the participating PC network address, the number of participating PCs are set in the conference control data, and no operation right is set (step S3244), and the conference starts. The packet reception is notified to the “conference participation control” (step S3245). If it is not a conference start packet in step S3243, it is further determined whether it is a conference file packet (step S3246). If it is a conference file packet, a “conference file packet” is received to create a conference file and save the conference file data. The conference file name is set in the conference control data (step S3247). Subsequently, the conference file display program is activated by the created conference file (step S3248).
If it is not a conference file packet in step S3246, it is further determined whether it is a PC screen unified packet (step S3249). If it is a PC screen unified packet, “PC screen unified packet reception control” is received by receiving “PC screen unified packet”. "Is executed (step S3250).

If it is not a PC screen unified packet in step S3249, it is further determined whether it is an overwrite packet (step S3251). If it is an overwrite packet, “overwrite packet reception control” is executed by receiving “overwrite packet” (step S3252). ). If it is not an overwrite packet in step S3251, it is further determined whether it is an overwrite erase packet (step S3253). If it is an overwrite erase packet, "overwrite erase packet reception control" is executed by receiving "overwrite erase packet" (step S3251). Step S3254).
If it is not an overwrite erase packet in step S3253, it is further determined whether it is an operation right packet (step S3255), and if it is an operation right packet, "operation right packet reception control" is executed by the "operation right packet" ( Step S3256). If it is not an operation right packet in step S3255, it is further determined whether it is a conference end packet (step S3257), and if it is a conference end packet, it is further determined whether the host PC is ended (step S3258).
If the host PC is not terminated here, the participation PC network address of the conference leaving PC is deleted from the conference control data by receiving the “conference end packet” from other than the host PC (step S3259). If the host PC is ended in step S3258, the display monitoring control and the overwrite control are notified and ended by receiving the “conference end packet” from the host PC (step S3260).

[PC screen unified packet reception control flow]
Next, the PC screen unified packet reception control operation will be described with reference to the flowchart of FIG. First, in order to change the display, the overwrite window is temporarily deleted (step S3261). Thereafter, it is determined whether or not the window origin coordinates are changed (step S3262). If the window origin coordinates are changed, the PC screen unification packet and the PC screen unification control data window upper left origin coordinates are compared. The APL window is moved to the packet “window upper left origin coordinates” (step S3263).
Subsequently, it is determined whether or not the width / height has been changed (step S3264). If the width / height has been changed, if the window width or the window height is different, the PC screen unified packet “window width”, “window” The window size is changed to “height” (step S3265). Subsequently, document display window information is acquired (step S3266), horizontal scroll bar window information is acquired (step S326Y), and it is determined whether the horizontal scroll box position has been changed (step S326Z, the horizontal scroll box position is changed). If so, the horizontal scroll box is moved to the PC screen uniform packet “horizontal scroll box position” (step S3267) Subsequently, it is determined whether the vertical scroll box position has been changed (step S3268). If changed, if the vertical scroll box position is different, the vertical scroll box is moved to the PC screen unification packet “vertical scroll box position” (step S3269) Subsequently, the PC screen unification control data is set (step S3269). -Up S3270), executes the "Overwrite display control", to redraw the overwrite the display change documents (step S3271).

[Overwrite packet reception control flow]
Next, the overwrite packet reception control operation will be described with reference to the flowchart of FIG. First, the number of drawing is set to 0 (step S 3281), and a line is drawn between the drawing coordinates with pen color and pen thickness (step S 3282). Thereafter, the drawing number is advanced by 1 (step S 3283), and it is determined whether the drawing number <the drawing coordinate number (step S 3284). If the drawing number <the drawing coordinate number is not satisfied, the window information (upper left origin coordinate) , Width, height) is acquired (step S3285).
Subsequently, horizontal scroll bar information (scroll box position, width) is acquired (step S3286), and vertical scroll bar information (scroll box position, width) is acquired (step S3287). Further, the drawing coordinates (screen coordinates) are converted into document coordinates (step S3288), and saved as stroke data in an overwrite file (step S3289).

[Overwrite erase packet reception control flow]
Next, the overwrite erasure packet reception control operation will be described with reference to the flowchart of FIG. First, the number of erases is set to 0 (step S3291), and the rectangular area consisting of the eraser cursor centered on the erase coordinates and the length of one side is set to a transparent color (step S3292). The overwriting of the rectangular area is erased and the background screen is displayed. Subsequently, the erase number is incremented by 1 (step S3293), and it is determined whether the erase number <the erase coordinate number (step S3294). If the erase number <the erase coordinate number, the window information of the document display area (the upper left origin) The coordinates, width, and height are acquired (step S3295).
Subsequently, horizontal scroll bar information (scroll box position, width) is acquired (step S3296), and vertical scroll bar information (scroll box position, width) is further acquired (step S3297). Subsequently, the erased coordinates are converted into document coordinates (step S3298), and the upper left origin coordinates, width, and height of the cursor scroll space rectangular area consisting of the center of the calculated coordinates and the length of one side eraser cursor are calculated. (Step S3299).
Further, stroke data is input from the overwrite file (step S3300), the document coordinate display magnification is converted (step S330Z), and it is determined whether the input stroke coordinates are within the cursor scroll space rectangular area (step S3301). If it is within the cursor scroll space rectangular area, it is further determined whether or not the stroke is divided (step S3302).

If the stroke is divided here, the stroke is divided when the middle of the stroke is erased (step S3303). Subsequently, the latter half of the divided stroke is stored in the overwrite file as a new stroke (step S3304), and the latter half of the input stroke data is deleted (step S3305). If the stroke is not divided in step S3302, the erased coordinates of the input stroke data are deleted (step S3306).
Subsequently, it is determined whether all stroke coordinates have been searched (step S3307). If all stroke coordinates are searched, the stroke data in the overwrite file is replaced with the input stroke data updated upon completion of all stroke coordinates of the input stroke data. (Step S3308). If all stroke coordinates are deleted, the stroke data is deleted from the overwrite file. Subsequently, it is determined whether all strokes have been searched (step S3309). That is, it is executed for all stroke data of the overwrite file. And it performs with respect to all the deletion coordinates (step S3310).

[Operation Right Packet Reception Control Flow]
Next, the operation right packet reception control operation will be described with reference to the flowchart of FIG. First, it is determined whether the control code is a request (step S3311). If the request is a request, the operation right (request) packet is transmitted to the host PC and an operation right request dialog box is displayed (step S3312). Subsequently, it is determined whether the operation right is permitted or rejected (step S3313). If it is permitted, the operation right PC network address of the conference control data is changed to the requested PC network address by permission of the operation right ( In step S3314, the operation right (movement) packet is transmitted to all participating PCs (step S3315).
Subsequently, the presence / absence of the operation right is determined (step S3316). If the operation right exists, if the host PC has the current operation right, the operation right transfer notification is sent to the “overwrite control” (step S3317). No right is set (step S3318). If the operation right is rejected in step S3313, the operation right (rejection) packet is transmitted to the operation right request PC by rejecting the operation right (step S3319).

If it is not the requested control code in step S3311, it is further determined whether the control code is rejected (step S3320), and if it is a reject control code, the operation right (rejection) packet is received and the operation is performed to “overwrite control”. The rejection of the right is notified (step S3321). If it is not a rejection control code in step S3320, it is further determined whether it is a movement control code (step S3322). If it is a movement control code, the operation right of the conference control data is received by receiving the operation right (movement) packet. The PC network address is changed to the requested PC network address (step S3323).
Subsequently, it is determined whether or not the operation right request PC is received (step S3324). When the operation right request PC receives the operation right (movement) packet, the operation right permission is notified to the “overwrite control” (step S3325). The operation right is set (step S3326). If it is not the operation right PC in step S3324, it is further determined whether the operation right is present (step S3327). If the operation right is present, the operation right transfer notification is sent to the “overwrite control” (step S3328). None is set (step S3329).

[Overwrite control flow]
Next, the overwrite control operation will be described with reference to the flowcharts of FIGS. First, an overwrite file is created and initialized (total number of strokes = 0) (step S3331), and the overwrite file name is set in the overwrite control data (step S3332). Subsequently, it is determined whether or not there is an operation right (step S3333). If there is no operation right, an operation right toolbar is displayed (step S3334), a transparent window having a full screen size is displayed, and operations other than the operation right toolbar are disabled. (Step S3335). If there is an operation right in step S3333, the overwrite toolbar is displayed (step S3336), the overwrite mode is set to APL (step S3337), the overwrite mode is set to APL, and the APL button of the overwrite toolbar is selected and displayed (step S3338). .
Subsequently, the overwriting control is ended by receiving a conference end packet from the host PC, or by an end notification by the “end” button of the overwriting toolbar or the operation right toolbar (step S3339). If it is not an end notification, it is determined whether it is an operation right transfer notification (step S3340), and the overwrite toolbar is deleted by the operation right transfer notification from the operation right packet reception control (step S3341). Subsequently, it is determined whether the APL overwrite mode is set (step S3342). If the APL overwrite mode is set, the overwrite window displayed in the drawing / erasing mode is deleted (step S3343).

Further, the operation right toolbar is displayed by the operation right transfer notification (step S3344), the overwrite window is displayed, and operations other than the operation right toolbar operation are invalidated (step S3345). Subsequently, only mouse data is input to the overwrite toolbar, and if the overwrite mode is “draw” or “erase”, all mouse data is input (step S3346). Mouse data is input to the operation authority toolbar, and “operation authority toolbar control” is executed (step S3346).
Subsequently, the presence / absence of the operation right is determined (step S3347). If there is no operation right, "operation right toolbar control" is executed by inputting mouse data on the operation right toolbar (step S3348). When the overwrite mode is “APL” (if there is an operation right in step S347, it is further determined whether the coordinates are overwrite toolbar coordinates (step S3349). If the coordinates are overwrite toolbar coordinates, mouse data input to the overwrite toolbar is performed. Thus, “overwrite toolbar control” is executed (step S3350).
If it is not the overwrite toolbar coordinates in step S3349, it is further determined whether the overwrite mode is drawing or erasing (step S3351). If it is the drawing overwrite mode, the overwrite mode is “ In the case of “drawing”, “overwrite drawing control” is executed (step S3352). If the overwriting mode is the erasing overwriting mode in step S351, the "overwriting erasing control" is executed when the overwriting mode is "erasing" when mouse data outside the overwriting toolbar is input (step S3353).
Subsequently, the presence / absence of the operation right is determined (step S33354). When the operation right is not present, the displayed operation right toolbar is deleted (step S3355), and the overwrite window is deleted (step S3356). If there is an operation right in step S3354, the overwriting toolbar is deleted (step S3357), and it is determined whether it is in the APL overwriting mode (step S3358). If in the APL overwriting mode, the overwriting window is deleted (step S3359). ). Then, a save confirmation dialog box is displayed (step S3360), and it is confirmed whether to save the overwrite data (step S3361). In the case of saving, the overwriting file data is copied and saved to the designated file (step S3362).

[Overwrite toolbar control flow]
Next, the overwrite toolbar control operation will be described with reference to the flowchart of FIG. Here, the mouse data when the mouse button is ON (pen touch) is executed as an argument. First, the mouse button ON coordinate button is selected and displayed, the others are displayed normally (step S3371), and mouse data up to the mouse button OFF (pen-up) is input (step S3372).
Subsequently, it is determined whether or not mouse data is input (step S3373). If the coordinates when the mouse button is OFF are not on the click button, the button function is not executed. Subsequently, it is determined whether the mouse button is OFF (step S3374). If the mouse button is OFF, it is further determined whether the coordinates are on the click button (step S3375), and the “pen” button is selected in the APL mode. If it is, the “overwrite display control” is executed, and overwriting to the currently displayed document is retrieved from the overwrite file and redrawn. Subsequently, it is determined whether the click button is a pen (step S3376), and if it is a pen, it is further determined whether the overwrite mode is APL (step S3377).
When the “erase” button is selected in the APL mode, “overwrite display control” is executed, overwriting to the currently displayed document is searched from the overwriting file, redrawing is performed (step S3378), and the overwriting mode is set to drawing. (Step S3379).

If it is not a pen click button in step S3376, it is further determined whether it is a delete click button (step S3380), and if it is an erase click button, it is further determined whether it is an APL overwrite mode (step S3381). If it is the APL overwrite mode, "overwrite display control" is executed, overwriting to the currently displayed document is retrieved from the overwrite file, redrawn (step S3382), and the overwrite mode is set to erase (step S3383). ).
If it is not the APL overwrite mode in step S3381, it is further determined whether the button is an APL click button (step S3384). If it is an APL click button, the "APL" button deletes the overwrite window, returns it to a state where normal program operation is possible (step S3385), and sets the APL overwrite mode (step S3386). If it is not the APL click button in step S3384, it is further determined whether the button is a setting click button (step S3387). If the button is a setting click button, the “setting” button displays an overwrite setting dialog box, and the pen color Then, the thickness and eraser cursor length are input (step S3388), and the overwrite setting dialog box input value is set in the overwrite control data (step S3389).
If it is not the click button for setting in step S3387, the “end” button creates a conference end packet and transmits it to all other PCs (step S3390), and notifies the network control, overwriting control, and display monitoring control of the end ( Step S3391). If it is not the click button upper coordinate in step S3375, the overwrite mode button is selected and displayed (step S3392).

[Operation Rights Toolbar Control Flow]
Next, the operation right toolbar control operation will be described with reference to the flowchart of FIG. Here, the mouse data when the mouse button is ON (pen touch) is executed as an argument. First, the mouse button ON coordinate button is selected and displayed, the others are displayed normally (step S3401), click button information is saved (step S3402), and mouse data up to mouse button OFF (pen-up) is input (step S3403). ). It is determined whether the mouse button is OFF (step S3404). If the mouse button is OFF, a click button is normally displayed (step S3405).
Subsequently, it is determined whether the coordinates are on the click button (step S3406). Here, if the coordinates when the mouse button is OFF are not on the click button, the button function is not executed. Subsequently, it is determined whether the button is an operation right click button (step S3407), and if it is an operation right click button, it is further determined whether it is a host PC (step S3408). In the case of the host PC, the host PC obtains the operation right and transmits an operation right (movement) packet to all participating PCs by the “operation right” button (step S3409).

If the PC is not the host PC in step S3408, the participating PC transmits an operation right (request) packet to the host PC and requests the operation right using the “operation right” button (step S3410). Subsequently, it waits for the network control to receive the operation right (permission) packet and notify the operation right permission (step S3411), and further waits for the network control to receive the operation right (rejection) packet and notify the operation right rejection (step S3411). Step S3412). When the operation right request is rejected, an operation right rejection dialog box is displayed (step S3413). The dialog box closes automatically after a few seconds. If the operation right request is permitted in step S411, an operation right permission dialog box is displayed (step S3414). The dialog box closes automatically after a few seconds. Subsequently, the operation right toolbar is deleted (step S3415), the overwrite window is deleted (step S3416), and the overwrite toolbar is displayed (step S3417).
Further, the overwrite mode is set to APL (step S3418), and the overwrite toolbar “APL” button is selected and displayed (step S3419). If it is ended in step S3407, a “end” button is used to create a conference end packet, transmit it to all other PCs (step S3420), and notify the network control, overwriting control, and display monitoring control of the end (step S3421). .

[Overwrite display control flow]
Next, the overwrite display control operation will be described with reference to the flowchart of FIG. First, window information (upper left origin coordinates, width, height) of the document display area is acquired (step S3431), and horizontal scroll bar information (scroll box position, width) is acquired (step S3432). Further, vertical scroll bar information (scroll box position and width) is acquired (step S3433), and the document coordinates of the document display area are calculated (step S3434). Further, a transparent window having a full screen size is displayed (step S3435), and stroke data is input in units of strokes from the overwrite file (step S3436).
The document coordinate display magnification is converted (step S343Z), then the immediately preceding coordinate is set to 0xFFFF (step S3437), and it is determined whether the stroke coordinate is the document display area (step S3438). If the stroke coordinates are not the document display area, the immediately preceding coordinates are set to 0xFFFF (step S3439). If it is the stroke coordinate of the document display area in step S3438, the stroke coordinate is converted to the screen coordinate (step S3440). Subsequently, it is determined whether or not the immediately preceding coordinate is 0xFFFF (step S3441), and if the immediately preceding coordinate is not 0xFFFF, a line is drawn between the immediately preceding coordinate and the screen coordinate with the stroke color and stroke thickness (step S3442).
Further, the screen coordinates are stored as the immediately preceding coordinates (step S3443). Subsequently, it is determined whether one stroke is completed (step S3444). If one stroke is completed, it is further determined whether the previous stroke is completed (step S3445). If all the strokes are not completed, the entire overwriting file stroke is searched and the next stroke data is input (step S3446), the document coordinate display magnification is converted (step S3447), and the process returns to step S3437.

[Overwrite drawing control flow]
Next, the overwrite drawing control operation will be described with reference to the flowchart of FIG. Here, the mouse data when the mouse button is ON (pen down) is executed as an argument. First, the mouse coordinates are stored in the drawing coordinate data, the number of drawing coordinates is set to “1” (step S3451), and the mouse data up to the mouse button OFF (pen-up) is input (step S3452). Subsequently, it is determined whether or not the mouse button is OFF (step S3453). If the mouse button is not OFF, a line between the previous coordinates and the current coordinates is drawn with the pen color and pen thickness set in the overwrite control data ( In step S3454, the input mouse coordinates are stored in the drawing coordinate data (step S3455).
If the mouse button is OFF in step S3453, an overwrite packet is created from the drawing coordinate data and transmitted to all other PCs (step S3456), and window information (upper left origin coordinates, width, height) of the document display area is acquired. (Step S3457). Further, horizontal scroll bar information (scroll box position and width) is acquired (step S3458), and vertical scroll bar information (scroll box position and width) is acquired (step S3659). Subsequently, the total number of lines of the document is calculated based on the scroll bar information, the drawing coordinates are converted into the document coordinates based on the scroll bar information and the document window information (step S3460), and the stroke data is stored in the overwrite file (step S3461).

[Overwrite deletion control flow]
Next, the overwrite erasure control operation will be described with reference to the flowchart of FIG. Here, the mouse data when the mouse button is ON (pen down) is executed as an argument. First, the mouse coordinates are stored in the erase coordinate data (step S3471), and the erase coordinate number is set to “1”. Subsequently, a rectangular area consisting of the mouse coordinate center and the length of one side and the eraser cursor length is set to a transparent color (step S3472). This erases the overwrite.
Subsequently, mouse data up to mouse button OFF (pen-up) is input (step S3473). Further, it is determined whether or not the mouse button is turned off (step S3474). If the mouse button is not turned off, the mouse coordinate is set at the center and the rectangular area consisting of the length of one side and the rubber cursor length is set to the transparent color (step S3475). The mouse coordinates are saved as erased coordinates in the erased coordinate data (step S3476). (If the mouse button is OFF in step S473, an overwrite erase packet is created based on the erase coordinate data, transmitted to all other PCs (step S3477), and window information (upper left origin coordinates, width, height) of the document display area is obtained. Obtain (step S3478).
Further, horizontal scroll bar information (scroll box position, horizontal scroll length (width)) is acquired (step S3479), and vertical scroll bar information (scroll box position, vertical scroll length (width)) is acquired (step S3480). Subsequently, the erasure coordinates are converted into document coordinates (step S3481), and the upper left origin coordinates, width, and height of the cursor scroll space rectangular area including the center of the erased cursor and the length of one side as the eraser cursor length are calculated (step S3481). The cursor document space is calculated (step S3482).

Further, stroke data is input from the overwrite file (step S3483). Subsequently, it is determined whether the input stroke coordinates are within the cursor scroll space (step S3484). Further, it is determined whether the stroke is divided (step S3485). If the stroke is divided, the stroke is divided when the middle of the stroke is deleted (step S3486).
Further, the latter half of the divided stroke is stored in the overwrite file as a new stroke (step S3487), and the latter half of the input stroke data is deleted (step S3488). If the stroke is not divided in step S3485, the erase coordinates of the input stroke data are deleted (step S3489). Subsequently, it is determined whether all stroke coordinates have been searched (step S3490). If all stroke coordinates are searched, the stroke data in the overwrite file is replaced with the input stroke data updated upon completion of all stroke coordinates of the input stroke data. . When all the stroke coordinates are deleted, the stroke data of the overwrite file is deleted (step S3491). Further, it is executed for all stroke data of the overwrite file (step S3492). Further, it is executed for all erased coordinates (step S3493).

[Overwrite edit control flow]
Next, the overwrite editing control operation will be described with reference to the flowchart of FIG. This overwriting editing program exists separately from the PC conference control program, and is used for overwriting display and editing. First, the overwrite editing program displays an “open dialog box” when started by means provided by the OS such as double-clicking on the desktop icon (step S3501).
Subsequently, it is determined whether or not to cancel (step S3502). If not cancelled, the overwrite file designated by the open dialog box is input (step S3503). Further, an overwrite file name, default pen color, pen thickness, eraser cursor length are set in the overwrite control data (step S3504), an overwrite editing toolbar is displayed (step S3505), and the overwrite editing mode is set to APL (step S3504). S3506).
Further, the overwrite editing toolbar “APL” button is selected and displayed (step S3507). Overwrite editing toolbar operation mouse data is input in the “APL” mode, and all mouse data is input in the “draw” and “erase” modes (step S3508). Subsequently, it is determined whether the coordinates are overwrite editing toolbar coordinates (step S3509). If not the overwrite editing toolbar coordinates, it is further determined whether the overwrite editing mode is drawing (step S3510).

Here, in the drawing overwrite editing mode, in the drawing mode, if the input mouse coordinates are coordinates outside the overwrite editing toolbar, “overwrite drawing control” is executed (step S3511). If the drawing edit mode is not the overwrite edit mode in step S3510, in the delete mode, if the input mouse coordinates are coordinates outside the overwrite edit toolbar, “overwrite delete control” is executed (step S3512).
If the coordinates are overwrite editing toolbar coordinates in step S3509, if the mouse coordinates are on the overwrite editing toolbar, "overwrite editing toolbar control" is executed (step S3513). Subsequently, it is determined whether or not the process has been completed (step S3514). If the process has been completed, the overwrite editing toolbar is deleted with the “end” button of the overwrite editing toolbar (step S3515). Subsequently, it is determined whether or not the overwrite editing mode is APL (step S3516). If the overwrite editing mode is not APL, if the overwrite display is being performed, the overwrite window is deleted (step S3517).
Further, an overwrite is saved or a save confirmation dialog box is displayed (step S3518). Subsequently, it is determined whether or not to save (step S3519), and if so, the overwrite data is saved to the designated overwrite file (step S3520). Here, the overwrite drawing control and the overwrite erasure control are the same as the PC conference control, and thus the description thereof is omitted.

[Overwrite editing toolbar control flow]
Next, the overwrite editing toolbar control operation will be described with reference to the flowchart of FIG. Here, the mouse data when the mouse button is ON (pen touch) is executed as an argument. First, the mouse button ON coordinate button is selectively displayed and the others are normally displayed (step S3521).
Subsequently, the click button information is stored (step S3522), and mouse data up to the mouse button OFF (pen-up) is input (step S3523). Subsequently, it is determined whether or not the mouse button is OFF (step S3524). If the mouse button is OFF, the button function is not executed if the coordinates when the mouse button is OFF are not on the click button (step S3525).
Subsequently, it is determined whether the click button is a pen (step S3526), and if it is a pen click button, it is further determined whether the overwrite editing mode is APL (step S3527), and the “pen” button in the APL mode. Is selected, “overwrite display control” is executed, overwriting to the currently displayed document is searched from the overwrite file, redrawing is performed (step S3528), and the overwrite editing mode is set to drawing (step S3529).

If it is not a pen click button in step S3526, it is determined whether it is a delete click button (step S3530). If it is an erase click button, it is determined whether the overwrite editing mode is APL (step S3531). When the overwrite editing mode is APL, when the “erase” button is selected in the APL mode, “overwrite display control” is executed, and overwriting to the currently displayed document is searched and redrawn (step) In step S3532, the overwrite editing mode is set to delete (step S3533). If the click button is not the delete click button in step S3530, it is further determined whether the click button is APL (step S3534). If the click button is APL, the "APL" button deletes the overwrite window and normal program operation is performed. (Step S3535), and the overwrite editing mode is set to APL (step S3536).
If the click button is not APL in step S3534, it is further determined whether the click button is set (step S3537). If the click button is set here, an “overwrite setting” dialog box is displayed with the “set” button, the pen color, pen thickness, and eraser cursor length are input (step S3538), and the overwrite setting dialog box input value is overwritten. Data is set (step S3539), the overwrite edit mode button is selected and displayed, and the others are displayed normally (step S3540).

It is explanatory drawing which shows the example of various display apparatuses with which this invention is applied. It is the schematic of one Embodiment which applied the document position detection apparatus which concerns on this invention to the display apparatus shown in FIG.1 (c). It is explanatory drawing of the light interception detection system as an example of a touch panel. It is the schematic which shows the structure of the display integrated coordinate input device similar to having shown in FIG.1 (c). It is explanatory drawing of another light interception detection system for touch panels. It is explanatory drawing of the other example of the system of the coordinate detection of overwriting information. It is explanatory drawing of an electromagnetic induction sensor unit. It is explanatory drawing about coordinate input control. It is explanatory drawing of the screen display information which can be acquired by I / F (API) which OS (in the case of Windows (trademark)) provides. It is explanatory drawing of the screen display information which can be acquired by I / F (API) which OS (in the case of Windows (trademark)) provides. It is explanatory drawing of the screen display information which can be acquired by I / F (API) which OS (in the case of Windows (trademark)) provides. It is explanatory drawing about the conversion method from a screen coordinate to a document coordinate. It is explanatory drawing about the conversion method from a document coordinate to a screen coordinate. It is explanatory drawing about the positional information on a scroll box. It is explanatory drawing about the display magnification conversion method when the display magnification of a document is changed. It is explanatory drawing about scroll information. It is explanatory drawing about scroll line length detection (drawing system). It is explanatory drawing about the scroll line length detection system (image extraction system) by an image extraction system. It is explanatory drawing about the scroll screen fixed display method at the time of scroll execution. It is explanatory drawing about the detection method of the present line number (current line number). It is explanatory drawing about a line coordinate detection method. It is explanatory drawing about position detection file registration. It is explanatory drawing about a position detection application example. It is explanatory drawing about the position detection control in PC5 shown in FIG. It is a flowchart which shows the processing operation of position detection start control. It is explanatory drawing which shows the input value and the state of each table in the flowchart of a position detection start control operation | movement. It is a flowchart which shows a position detection file control operation | movement. It is explanatory drawing which shows the state of the position detection registration file in the flowchart of a position detection file control operation | movement. It is a flowchart which shows document coordinate calculation control operation | movement. It is explanatory drawing which shows the state of the input value in the flowchart of document coordinate calculation control operation | movement, a display document coordinate, and a document coordinate. It is a flowchart which shows screen coordinate calculation control operation | movement. It is explanatory drawing which shows the state of the input value in the flowchart of screen coordinate calculation control operation | movement, a display document coordinate, and a document coordinate. It is a flowchart which shows display document area | region calculation control operation | movement. It is explanatory drawing which shows the state of each table and display document coordinate in the flowchart of display document area | region calculation control operation | movement. It is a flowchart which shows scroll control operation | movement. It is explanatory drawing which shows the input value and the state of each table in the flowchart of scroll control operation | movement. It is a flowchart which shows document coordinate display control operation. It is explanatory drawing which shows the input value and the state of each table in the flowchart of document coordinate display control operation | movement. It is a flowchart which shows position detection completion | finish control operation | movement. It is a flowchart which shows registration file start operation | movement. It is explanatory drawing which shows the state of the input value in the flowchart of a registration file start operation | movement, a position detection registration file, and a position detection control table. It is a flowchart which shows a current vertical line number detection operation. It is a flowchart which shows a current horizontal line number detection operation. It is a flowchart which shows a vertical line position display operation. It is explanatory drawing which shows the state of the input value in the flowchart of a vertical line position display operation. It is a flowchart which shows a horizontal line position display operation. It is explanatory drawing which shows the state of the input value in the flowchart of a horizontal line position display operation | movement. It is a flowchart which shows vertical position detection information detection operation | movement. It is explanatory drawing which shows the state of each table in the flowchart of vertical position detection information detection operation | movement. It is a flowchart which shows horizontal position detection information detection operation | movement. It is explanatory drawing which shows the state of each table in the flowchart of horizontal position detection information detection operation | movement. It is a flowchart which shows the vertical scroll line number detection operation | movement. It is a flowchart which shows horizontal scroll line number detection operation | movement. It is a flowchart which shows a vertical line coordinate detection operation. It is explanatory drawing which shows the state of the input value and vertical line coordinate table in the flowchart of a vertical line coordinate detection operation. It is a flowchart which shows a horizontal line coordinate detection operation | movement. It is explanatory drawing which shows the state of the input value in the flowchart of a horizontal line coordinate detection operation, and a horizontal line coordinate table. It is a flowchart which shows a vertical line length drawing system detection operation. It is explanatory drawing which shows the state of the position detection control table in the flowchart of a vertical line length drawing system detection operation | movement. It is a flowchart which shows horizontal line length drawing system detection operation | movement. It is explanatory drawing which shows the state of the position detection control table in the flowchart of a horizontal line length drawing system detection operation. It is a flowchart which shows a vertical line length image extraction system operation | movement. It is explanatory drawing which shows the state of the position detection control table in the flowchart of a vertical line length image extraction system detection operation | movement. It is a flowchart which shows a horizontal line length image extraction system operation | movement. It is explanatory drawing which shows the state of the position detection control table in the flowchart of a horizontal line length image extraction system detection operation | movement. 6 is a flowchart showing a document display fixing operation. It is explanatory drawing which shows the state of the position detection control table in the flowchart of document display fixing operation. It is a flowchart which shows document display fixed cancellation | release operation | movement. It is explanatory drawing which shows the state of the position detection control table in the flowchart of document display fixed cancellation | release operation | movement. It is a system configuration figure explaining various forms of a PC conference system concerning the present invention. It is a block diagram which shows schematic structure of the PC conference system concerning the 2nd Embodiment of this invention. It is explanatory drawing which shows the structure data of PC meeting control data. It is explanatory drawing concerning overwriting operation. It is explanatory drawing of overwrite stroke data. It is explanatory drawing of the meeting sponsorship start method. It is explanatory drawing of a meeting participation start method. It is explanatory drawing of the PC screen unification method. It is explanatory drawing of the processing method which displays the document of a specific position. It is explanatory drawing of the overwrite redrawing method. It is explanatory drawing of the operation right transfer method concerning 2nd Embodiment. It is explanatory drawing of the operation right transfer method concerning 2nd Embodiment. It is explanatory drawing which shows the overwrite edit operation example (1) concerning 2nd Embodiment. It is explanatory drawing which shows the overwrite edit operation example (2) concerning 2nd Embodiment. It is a figure explaining the setting of overwrite concerning 2nd Embodiment. It is a flowchart which shows the initialization control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the control operation | movement of the meeting sponsor concerning 2nd Embodiment. It is a flowchart which shows the conference participation control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the display monitoring control operation concerning 2nd Embodiment. It is a flowchart which shows the network control operation | movement concerning 2nd Embodiment. It is a flowchart which shows PC screen unified packet reception control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite packet reception control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite deletion packet reception control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the operation right packet reception control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite toolbar control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the operation right toolbar control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite display control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite drawing control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite deletion control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite edit control operation | movement concerning 2nd Embodiment. It is a flowchart which shows the overwrite edit toolbar control operation | movement concerning 2nd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Display apparatus, 3 ... Touch panel, 5 ... PC, 7 ... Coordinate calculation control apparatus, 9 ... LED, 11 ... Phototransistor, 13 ... Coordinate indicator, 15 ... PC, 16 ... Electronic pen, 17 ... Touch panel, 19 ... Tempered glass, 21 ... Liquid crystal display unit, 23 ... Electromagnetic induction sensor unit, 25 ... Resonance circuit, 27 ... Sensor coil, 29 ... Coordinate detection control device, 31 ... Transmission / reception switch, 33 ... AC power supply, 35 ... X-axis coil switching Switch ... 41 Touch panel driver 51 ... Application unit 53 ... Position detection start control unit 55 ... Position detection file control unit 57 ... Document coordinate calculation control unit 59 ... Screen coordinate calculation control unit 61 ... Display document area calculation Control part 63 ... Scroll control part 65 ... Document coordinate display control part 67 ... Position detection end control part 69 ... Position detection control table, DESCRIPTION OF SYMBOLS 1 ... Scroll information table, 73 ... Vertical line coordinate table, 75 ... Horizontal line coordinate table, 110 ... Display integrated coordinate input device, 200 ... PC screen, 201 ... Document creation program, 202 ... PC conference control program, 204 ... Initial Control unit, 205 ... display monitoring control unit, 206 ... overwrite control unit, 207 ... network control unit, 208 ... control data, 209 ... conference file, 210 ... overwrite file, 211 ... packet data, 212 ... conference start packet, 213 ... Conference file packet, 215 ... Overwrite packet, 216 ... Overwrite erase packet, 217 ... Operation right packet, 218 ... Conference end packet, 219 ... Conference control data, 220 ... PC screen unification control data, 221 ... Overwrite control data

Claims (21)

  A computer comprising: document position information detecting means for detecting the number of scroll lines and scroll line length of a displayed document displayed by an arbitrary application program operating on an operating system by moving a scroll box. An overwriting information processing apparatus in which an application for displaying a document is executed,
Display screen acquisition means for acquiring information of a display screen on which the application is executed;
Coordinate specifying means for specifying coordinates in the document in the document display area displayed on the display screen from the display screen on which the application is executed;
Input detection means for detecting overwriting in the document display area;
Storage control means for associating and storing the coordinates and the drawing,
An overwriting information processing apparatus, wherein the display screen acquisition means includes document position information detection means for detecting the number of scroll lines and the scroll line length of a display document displayed by the application program by moving a scroll box.   3. The overwriting information processing apparatus according to claim 2, wherein the document position information detecting means draws a straight line in the document area, detects the amount of movement of the drawing line by moving the scroll box, and detects the scroll line length. .   3. The overwriting information processing according to claim 2, wherein the document position information detecting means extracts a specific image of a document area, detects a moving amount of the specific image due to a scroll box movement, and detects a scroll line length. apparatus.   3. The overwriting information processing apparatus according to claim 2, wherein the document position information detecting unit acquires an image of a document display area, displays the acquired image so as to overlap the document area, and executes scroll box movement. .   3. The overwriting information processing apparatus according to claim 2, wherein the document position information detecting unit detects a scroll line length of the document and detects line coordinates of all lines of the document.   3. The overwriting information processing apparatus according to claim 2, wherein the document position information detecting means detects a current line number by moving the scroll box until the scroll box position changes.   3. The overwriting information processing apparatus according to claim 2, wherein the document position information detecting means displays a transparent window on the document and scrolls the document by moving a scroll box.   3. The overwriting according to claim 2, wherein said document position information detecting means detects scroll information for each program, stores it, automatically recognizes a program for displaying the document, and makes it unnecessary to detect scroll information. Information processing device.   3. The document position information detecting means detects and saves the number of scroll lines and scroll line coordinates of a document in advance, and makes it unnecessary to detect the number of scroll lines and line coordinates of the document when performing position detection. The overwriting information processing apparatus described.   3. The overwriting information processing apparatus according to claim 2, wherein the coordinate specifying unit has a function of specifying a drawing corresponding to a document display area on a display screen based on coordinates stored in the storage unit. . An overwriting information processing method for executing an application for displaying a document,
Acquiring information of a display screen on which the application is executed by a display screen acquiring unit;
Specifying the coordinates in the document of the document display area displayed on the display screen from the display screen on which the application is executed by the coordinate specifying means;
Detecting drawing on the document display area by input detection means;
Storing the coordinates and the drawing in association with each other by storage control means,
The step of acquiring information on the display screen includes a document position information detecting step of detecting the number of scroll lines and the scroll line length of a display document displayed by the application program by moving a scroll box. Information processing method. When executing an application that displays documents,
Acquiring information of a display screen on which the application is executed by a display screen acquiring unit;
Specifying the coordinates in the document of the document display area displayed on the display screen from the display screen on which the application is executed by the coordinate specifying means;
Detecting drawing on the document display area by input detection means;
Storing the coordinates and the drawing in association with each other by storage control means,
A computer processing method for overwriting information, wherein the step of acquiring information on the display screen includes a step of detecting document position information in which the number of scroll lines and the scroll line length of a display document displayed by the application program are detected by moving a scroll box. Program realized by. When executing an application that displays documents,
Acquiring information of a display screen on which the application is executed by a display screen acquiring unit;
Specifying the coordinates in the document of the document display area displayed on the display screen from the display screen on which the application is executed by the coordinate specifying means;
Detecting drawing on the document display area by input detection means;
Storing the coordinates and the drawing in association with each other by storage control means,
A computer processing method for overwriting information, wherein the step of acquiring information on the display screen includes a step of detecting document position information for detecting the number of scroll lines and the scroll line length of a display document displayed by the application program by moving a scroll box. A storage medium storing a program realized by the above. A plurality of PC terminals as computer devices each having a conference information display means are connected as PC conference participation terminals on the network, and one of the plurality of PC terminals is used as a PC conference operation right terminal as needed, and the PC conference In the electronic conferencing method for performing the electronic conference by outputting the display document as the conference information displayed on the operation right terminal and the overwriting information on each of the other PC terminals.
The document position information detection means of the PC terminal displays the position of the display document displayed by the arbitrary application program operating on the operating system on the meeting information display means from horizontal scroll bar information, vertical scroll bar information, and display window information. In addition to detecting, the number of scroll lines and the scroll line length of the displayed document are detected and transmitted by moving the scroll box,
In response to the received horizontal scroll bar information, vertical scroll bar information, display window information, and information detected by movement of the scroll box, the position matching means of the other PC terminal corresponds to An electronic conference method, wherein the display state of the display document as the conference information displayed on the conference information display means is set to the same display state as that of the transmission source PC terminal. A plurality of PC terminals as computer devices each having a conference information display means are connected as PC conference participation terminals on the network, and one of the plurality of PC terminals is used as a PC conference operation right terminal as needed, and the PC conference In the electronic conferencing method for performing the electronic conference by outputting the display document as the conference information displayed on the operation right terminal and the overwriting information on each of the other PC terminals.
The document position information detection unit of the PC terminal detects a change in the position and size of a work window in which an arbitrary application program operating on an operating system displays the display document on the conference information display unit, and the display document Scroll line number and scroll line length are detected by moving the scroll box and sent out.
The position matching unit of the PC terminal transmits the display state of the work window displaying the display document as the conference information displayed on the conference information display unit of the own device in correspondence with the received information. An electronic conference method, wherein the display state is the same as that of an original PC terminal. When the document position information detection unit of the PC terminal detects a change in switching of a display document as another display document as conference information displayed by an arbitrary application program operating on an operating system,
The electronic conference according to claim 15 or 16, wherein the PC conference participation terminal acquires the changed file of another display document as the conference information and redisplays the file in correspondence with the file. Method. When the document position information detecting means of the PC terminal overwrites the displayed document as conference information displayed by an arbitrary application program operating on the operating system, the coordinates on the screen for overwriting operations such as overwriting characters And converting the coordinates on the screen into document coordinates, detecting the number of scroll lines and the scroll line length of the displayed document by moving the scroll box, and transmitting to the PC conference participation terminal, and Save the document coordinates on your machine,
The overwriting information processing apparatus draws the overwriting contents on the display document as the meeting information displayed on the meeting information display means by the coordinates on the screen, and the same overwriting on the document material is performed. 18. The method of claim 15, wherein the same display state is made corresponding to the information detected by the movement of the scroll box, and the coordinates on the screen are converted into document coordinates and stored. The electronic conference method according to claim 1. When the overwriting content of the displayed document displayed on the meeting information display means is erased in whole or in part, the overwriting information processing apparatus acquires the coordinates on the screen of the overwriting erasing operation, and Transmitted to the PC conference participation terminal, and deletes the deleted document coordinates from the document coordinates stored in the own device,
The document position information detecting means of the PC terminal erases the overwritten character by the screen coordinate, makes the overwritten content on the document material the same, and calculates the document coordinate of the overwritten character erased based on the screen coordinate 19. The electronic conference method according to claim 18, wherein the deleted document coordinates are deleted from the stored document coordinates.   When changing the display document, the display area document space being displayed is calculated, the saved document coordinates are converted to screen coordinates according to the calculation results, and redrawn to make the overwriting contents on the conference material the same. The electronic conferencing method according to claim 18 or 19, characterized by the above.   The overwriting information processing apparatus displays the file of the display document that has been overwritten, if necessary, and redraws the overwrite information from the stored overwrite information to the position at the time of overwriting, and then adds the overwrite, 21. The electronic conference method according to claim 18, wherein an editing process such as deletion is possible.

Images