JP6237359B2 - Display processing apparatus, display processing method, and display processing program - Google Patents

Description

  This case relates to a display processing device, a display processing method, and a display processing program.

  Along with advances in communication technology, for example, by sending screen display data from a personal computer (hereinafter referred to as PC) in the office to the smartphone of a salesperson or maintenance person in the field, the PC is displayed on the screen of the smartphone. Systems that display the same content as are widely used. By using this system, it is possible to share information resources such as materials between the office and the site, so that work efficiency is improved.

  Regarding sharing of information displayed on the screen, for example, Patent Document 1 describes that an image transmitted from an information processing device is displayed on the screen of a terminal device. Japanese Patent Application Laid-Open No. H10-228561 describes that screen display contents are shared by communication between terminal devices.

JP 2012-198675 A JP 2006-323607 A

  For example, when editing documents and the like between the office and the site, the smartphone screen is smaller than the PC screen, so the edit area in the PC screen can be specified on the smartphone screen by specifying it with the mouse. The displayed editing area is enlarged.

  However, a wireless communication network to which a smartphone is connected has a low communication speed (for example, 1 to 5 (Mbps) and a large delay time (for example, a round trip time of 50 to 100 (msec) in the case of 3G (3rd Generation)), for example. For this reason, if designation of the editing area on the PC side is frequently changed from top to bottom and left and right, a large amount of display data is transmitted, and the smartphone takes a lot of time to receive the display data. .

  Therefore, each time the smartphone receives display data, the screen is updated, so that the material displayed on the screen fluctuates up and down and left and right, which makes it difficult for the smartphone user to grasp the editing area. Such a problem is not limited to smartphones, but also exists in other types of terminal devices.

  Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide a display processing device, a display processing method, and a display processing program that efficiently update the screen of another device.

  The display processing device described in the present specification transmits display data of a designated area designated in the first screen to another device having a second screen smaller than the first screen, so that the second screen is displayed. The first screen is divided and divided into a plurality of areas based on at least two types of display contents of text, image, and background, and the designated area is changed. Thus, when one or more areas included in the designated area among the plurality of areas change to one or more other areas, the display data of the designated area after the change is transmitted to the other device. Part.

  The display processing method described in the present specification transmits display data of a designated area designated in the first screen of the first device to a second device having a second screen smaller than the first screen, In the method of displaying on the second screen, the step of dividing the first screen into a plurality of regions based on a type of display content divided into at least two types of text, image, and background; When one or more areas included in the designated area are changed to one or more other areas among the plurality of areas by changing the designated area, the display data of the designated area after the change is The computer executes the step of transmitting to the second device.

  The display processing program described in this specification transmits display data of a designated area designated in the first screen of the first device to a second device having a second screen smaller than the first screen, In the program to be displayed on the second screen, the first screen is divided into a plurality of regions based on at least two types of display contents among text, image, and background, and the designated region When one or more areas included in the designated area among the plurality of areas are changed to one or more other areas, the display data of the designated area after the change is changed to the second The processing to be transmitted to the apparatus is executed by the computer.

  The screens of other devices can be updated efficiently.

It is a block diagram which shows an example of a screen sharing system. It is a block diagram which shows an example of a display processing apparatus. It is a block diagram which shows an example of a terminal device. It is a block diagram which shows an example of each function of a display processing apparatus and a terminal device. It is a flowchart of the display processing program of a display processing apparatus. It is a flowchart of the display processing program of a terminal device. It is a flowchart which shows the division | segmentation process of a screen. It is a figure which shows the example of a division | segmentation of a screen. It is a graph which shows an example of distribution of the number of colors contained in a unit field. It is a table | surface which shows an example of area | region division information. It is a flowchart which shows the transmission process of display data. It is a figure which shows an example of the movement of a designated area | region, and display data. It is a figure which shows the other example of the movement of a designated area | region, and display data.

  FIG. 1 is a configuration diagram illustrating an example of a screen sharing system. The screen sharing system includes a display processing device (first device) 1 and a terminal device (second device, other device) 2 connected to each other via a network NW.

  The display processing device 1 is, for example, a computer device (that is, an information processing device) such as a desktop PC, and is installed in an office. The display processing device 1 has a screen (first screen) 1a provided on the display. The size (width W1 × height H1) of the screen 1a is, for example, 3840 × 2400 (pixels).

  The terminal device 2 is a wireless communication terminal such as a smartphone, for example, and is possessed by a sales staff or maintenance staff at the site. The terminal device 2 has a screen (second screen) 2 a that is smaller than the screen 1 a of the display processing device 1. The size (width W2 × height H2) of the screen 2a is, for example, 720 × 1280 (pixels).

  As an example, a document is displayed on the screen 1 a of the display processing device 1. The document includes, for example, texts T1 and T2, an image (such as a photograph or drawing) G, and a background (margin portion) M. In the following example, the operation of the display processing device 1 will be described with reference to the document of the screen 1a shown in FIG.

  The display processing device 1 transmits the display data DT of the rectangular designated region L designated in the screen 1a to the terminal device 2 via the network NW. The display data DT is stored in, for example, an IP (Internet Protocol) packet and transmitted.

  The designated area L is defined, for example, by the position (reference position) S of the pointer P that the user operates using the mouse 1b or the keyboard 1c. That is, the user freely operates the pointer P using the mouse 1b or the keyboard 1c, and accordingly, the designated area L moves within the screen 1a. In the present embodiment, the position S of the pointer P is located at the center of the designated area L, but is not limited thereto, and may be located at the corner of the designated area L. Further, the shape of the designated region L is not limited to a rectangle, and may be, for example, a circle.

  The terminal device 2 displays the received display data DT on the screen 2a. That is, the display processing device 1 displays the display data DT on the screen 2a of the terminal device 2 by transmitting the display data DT of the designated area L. Here, the size of the designated area L is determined based on the size of the screen 2 a of the terminal device 2.

  In this way, the display processing device 1 shares the display content of the designated area L in the screen 1a with the terminal device 2. Therefore, by using the screen sharing system, information resources such as documents can be shared between the office and the site, so that work efficiency is improved.

  The network NW includes, for example, an in-office LAN (Local Area Network) to which the display processing device 1 is connected and an outdoor wireless communication network to which the terminal device 2 is connected. The communication speed of the wireless communication network is lower than the communication speed in the LAN, and the delay time is also larger than the delay time of the LAN.

  For this reason, if the display processing device 1 transmits the display data DT of the designated area L in small increments every time the designated area L moves, the terminal device is changed when the designated area L is frequently changed vertically and horizontally. 2 displays a large amount of display data DT. At this time, the terminal device 2 requires a long time for the reception process of the display data DT.

  Therefore, whenever the terminal device 2 receives the display data DT, the document displayed on the screen 2a fluctuates vertically and horizontally by updating the screen 2a, so that the user of the terminal device 2 can grasp the designated area L. It becomes difficult.

  Therefore, the display processing apparatus 1 according to the present embodiment divides the screen 1a into a plurality of attribute areas based on the types of the texts T1 and T2, the image G, and the background M, and the attribute areas included in the designated area L are the other attribute areas. When the attribute area is changed, the display data of the designated area L is transmitted. Thereby, the screen 2a of the terminal device 2 is updated efficiently. Below, the structure of the display processing apparatus 1 and the terminal device 2 of a present Example is described.

  FIG. 2 is a configuration diagram illustrating an example of the display processing device 1. The display processing apparatus 1 includes a CPU (Central Processing Unit) 10, a ROM (Read Only Memory) 11, a RAM (Random Access Memory) 12, an HDD (Hard Disk Drive) 13, a communication processing unit 14, and a portable storage medium drive 15. , An input processing unit 16, an output processing unit 17, and the like.

  The CPU 10 is arithmetic processing means, and operates according to a display processing program. The CPU 10 is connected to the units 11 to 17 via the bus 18. The display processing apparatus 1 is not limited to one that operates by software, and hardware such as an application specific integrated circuit may be used instead of the CPU 10.

  The RAM 12 is used as a working memory for the CPU 10. The ROM 11 and the HDD 13 are used as storage means for storing a display processing program for operating the CPU 10. The communication processing unit 14 is, for example, a network card, and is a communication unit that communicates with the terminal device 2 via the network NW.

  The portable storage medium drive 15 is a device that writes information to and reads information from the portable storage medium 150. Examples of the portable storage medium 150 include a USB memory (USB: Universal Serial Bus), a CD-R (Compact Disc Recordable), and a memory card. The display processing program may be stored in the portable storage medium 150.

  The display processing apparatus 1 further includes an input device 160 for performing an information input operation, and a display 170 having a screen 1a. The input device 160 is input means such as a keyboard 1c and a mouse 1b, and input information is output to the CPU 10 via the input processing unit 16. The display 170 is a display unit such as a liquid crystal display, and display data to be displayed is output from the CPU 10 to the display 170 via the output processing unit 17. In addition, it can replace with the input device 160 and the display 170, and can also use devices, such as a touchscreen provided with these functions.

  The CPU 10 executes a program stored in the ROM 11 or the HDD 13 or a program read from the portable storage medium 150 by the portable storage medium drive 15. This program includes not only the OS (Operating System) but also the above display processing program. The program may be downloaded via the communication processing unit 14.

  FIG. 3 is a configuration diagram illustrating an example of the terminal device 2. The terminal device 2 includes a CPU 20, a ROM (Read Only Memory) 21, a RAM (Random Access Memory) 22, a communication processing unit 24, a touch panel 25, a display panel 26, and the like.

  The CPU 20 is arithmetic processing means, and operates according to a display processing program. The CPU 20 is connected to each part 21, 22, 24 to 26 via the bus 28. The terminal device 2 is not limited to one that operates by software, and hardware such as an application specific integrated circuit may be used instead of the CPU 20.

  The RAM 22 is used as a working memory for the CPU 20. The ROM 21 is used as storage means for storing a display processing program for operating the CPU 20. The communication processing unit 24 includes, for example, an antenna and an RF signal processing circuit, and is a communication unit that communicates with the display processing device 1 via the network NW.

  The touch panel 25 is an information input unit, and the input information is output to the CPU 20. The display panel 26 is a display means such as a liquid crystal panel and has a screen 2a. Display data displayed on the screen 2 a is output from the CPU 20 to the display panel 26. The touch panel 25 and the display panel 26 are integrated by being overlapped.

  The CPU 20 executes a program stored in the ROM 21 or the like. This program includes not only the OS but also the above display processing program. The program may be downloaded via the communication processing unit 24.

  When the CPUs 10 and 20 execute the respective display processing programs, a plurality of functions are formed. FIG. 4 is a configuration diagram illustrating an example of each function of the display processing device 1 and the terminal device 2. FIG. 4 shows an example of each function formed in the CPUs 10 and 20 and information stored in the HDD 13 and RAM 22.

  In the display processing device 1, the CPU 10 includes a session establishment unit 100, a screen control unit 101, a division unit 102, a display update determination unit 103, an intermediate area generation unit 104, and a display data generation unit 105. The HDD 13 stores document data 130, screen display data 131, area division information 132, designated area information 133, and size information 134.

  In the terminal device 2, the CPU 20 includes a session establishment unit 200 and a screen control unit 201. The RAM 22 stores size information 134 and display data DT. The function of each part is described below.

  The session establishment unit 100 of the display processing device 1 transmits a session establishment request message to the terminal device 2 via the communication processing unit 14. When receiving the session establishment request message via the communication processing unit 24, the session establishing unit 200 of the terminal device 2 transmits a response message to the display processing device 1 via the communication processing unit 24. As a result, a session using VPN (Virtual Private Network) or the like is established between the display processing device 1 and the terminal device 2, and the display data DT can be transmitted from the display processing device 1 to the terminal device 2. .

  In addition, the session establishing unit 200 of the terminal device 2 reads the size information 134 indicating the size of the screen 2 a from the RAM 22 and transmits it to the display processing device 1 via the communication processing unit 24. When the session establishing unit 100 of the display processing device 1 receives the size information 134 via the communication processing unit 14, it stores it in the HDD 13.

  The screen control unit 101 of the display processing device 1 performs display control of the screen 1a. The screen control unit 101 acquires the scroll amount of the screen 1 a generated by the operation of the input device 160 from the input processing unit 16. The screen control unit 101 reads the document data 130 from the HDD 13, extracts data within the display range of the screen 1 a based on the scroll amount from the document data 130, and writes the data in the HDD 13 as the screen display data 131. That is, the screen display data 131 is display data of a portion displayed on the screen 1a of the display 170 in the entire document.

  The screen control unit 101 writes the screen display data 131 to the HDD 13 and then notifies the dividing unit 102 of the completion of the writing. When receiving the notification from the screen control unit 101, the dividing unit 102 reads the screen display data 131 from the HDD 13.

  The dividing unit 102 refers to the screen display data 131, thereby dividing the screen 1a into a plurality of attribute areas (areas) based on the types of display contents divided into the texts T1, T2, the image G, and the background M. To divide. Although a specific example of the division method will be described later, as an example, the division unit 102 divides the screen 1a into a plurality of attribute areas based on the color distribution in the screen 1a. Thereby, the dividing unit 102 can clearly divide the screen 1a into the attribute areas of the texts T1 and T2, the image G, and the background M.

  The dividing unit 102 writes area division information 132 indicating each attribute area obtained by dividing the screen 1 a to the HDD 13. The area division information 132 indicates the ID (identifier) and position range of each attribute area, as will be described later. In this embodiment, an example of division based on the three types of display contents of text T1, T2, image G, and background M is given, but the present invention is not limited to this, and the text T1, T2, image G, and background are not limited thereto. The screen 1a may be divided based on two types of M.

  The display update determination unit 103, the intermediate area generation unit 104, and the display data generation unit 105 constitute a display data DT transmission unit 10a. The transmission unit 10a changes one or more attribute areas included in the specified area L to one or more other attribute areas from among the plurality of attribute areas when the specified area L is changed by the operation of the input device 160. Then, the display data DT of the designated area L after the change is transmitted to the terminal device 2.

  More specifically, when the position (reference position) S1 of the pointer P that defines the designated area L exceeds the boundary separating the plurality of attribute areas, the transmitting unit 10a displays the display data of the designated area L after the change. DT is transmitted to the terminal device 2. That is, the transmission unit 10a detects that the configuration of the attribute area included in the designated area L is changed when the position S of the pointer P exceeds the boundary of the attribute area.

  The display update determination unit 103 refers to the region division information 132 to determine whether there is a change in the attribute region including the position S of the pointer P acquired from the input processing unit 16. Thereby, it is determined whether or not the position S of the pointer P exceeds the boundary of the attribute area.

  When the attribute area including the position S of the pointer P changes, the display update determination unit 103 instructs the display data generation unit 105 to generate the display data DT together with the position S of the pointer P. The display update determination unit 103 determines the configuration of the attribute area included in the designated area L based on the size information 134, and instructs the display data generation unit 105 to generate the display data DT according to the determination result. May be.

  Upon receiving the display data DT generation instruction, the display data generation unit 105 reads the size information 134, the screen display data 131, and the area division information 132 from the HDD 13 to generate display data DT for the specified area L after the change. To do. The display data DT is generated by, for example, extracting (clipping) a portion overlapping with the designated region L from each attribute region and combining the extracted portions. The generated display data DT is transmitted to the terminal device 2 via the communication processing unit 14.

  In the terminal device 2, the screen control unit 201 receives the display data DT via the communication processing unit 24 and temporarily stores it in the RAM 22. The screen control unit 201 reads the display data DT from the RAM 22 and outputs it to the display panel 26 at the display update timing of the display panel 26. As a result, the display data DT is displayed on the display panel 26.

  In addition, when there is no overlapping portion between the designated area L before the change and the designated area L after the change, the transmitting unit 10a designates the designation before the change before transmitting the display data DT of the designated area L after the change. The display data DT of the intermediate area existing between the area L and the changed designated area L is transmitted to the terminal device 2. When the display update determination unit 103 outputs a display data DT generation instruction, the intermediate region generation unit 104 determines whether or not there is an overlapping portion between the specified region L before the change and the specified region L after the change.

  At this time, the intermediate area generation unit 104 reads the size information 134 and the specified area information 133 from the HDD 13 and acquires the position S of the pointer P from the display update determination unit 103. The designated area information 133 indicates the position range of the designated area L of the display data DT transmitted last time. The intermediate area generation unit 104 updates the designated area information 133 every time the display data DT of the designated area L is transmitted.

  The intermediate area generation unit 104 acquires the position range of the specified area L before the update from the specified area information 133 and acquires the position range of the specified area L after the change from the size information 134 and the position of the pointer P. The intermediate area generation unit 104 determines whether or not there is an overlapping portion by comparing the position range of the designated area L before update with the position range of the designated area L after change.

  When there is no overlapping portion, the intermediate area generation unit 104 instructs the display data generation unit 105 to generate display data DT for the intermediate area existing between the designated area L before the change and the designated area L after the change. In the present embodiment, the intermediate area generation unit 104 determines the position range of the intermediate area based on the midpoint of the position S of each pointer P that defines the designated area L before and after the change, but is not limited thereto. The intermediate region generation unit 104 notifies the display data generation unit 105 of the position range of the intermediate region together with an instruction to generate display data DT for the intermediate region.

  Upon receiving an instruction to generate the display data DT for the intermediate area, the display data generation unit 105 generates the display data DT for the intermediate area, and before the transmission of the display data DT for the designated area L after the change, the communication processing unit 14 Is transmitted to the terminal device 2 via. Here, the generation of the display data DT for the intermediate area is performed based on the screen display data 131 and the area division information 132 in the same manner as the generation of the display data DT for the designated area L. In the terminal device 2, when receiving the display data DT in the intermediate area, the screen control unit 201 temporarily stores it in the RAM 22 and outputs it to the display panel 26.

  For this reason, the display data DT in the intermediate area is displayed on the screen 2a of the terminal device 2 before the display data DT in the designated area L after the change. Therefore, by complementing between the designated areas L before and after the update, the user of the terminal device 2 can easily grasp the designated area L after the change.

  Next, a display processing method executed by each of the CPUs 10 and 20 according to each display processing program will be described. FIG. 5 is a flowchart of the display processing program of the display processing device 1.

  First, the session establishment unit 100 transmits a session establishment request message to the terminal device 2 (step St1). Next, the session establishing unit 100 determines whether or not a response message has been received from the terminal device 2 (step St2). Here, for example, the session establishing unit 100 determines whether or not a response message is received within a certain time after the transmission of the establishment request message.

  When the response message is not received (No in Step St2), the session establishing unit 100 ends the process. When the response message is received (Yes in step St2), the session establishing unit 100 establishes a session with the terminal device 2 using VPN or the like (step St3). Thereby, the display data DT can be transmitted from the display processing device 1 to the terminal device 2.

  Next, the session establishing unit 100 receives the size information 134 of the screen 2a from the terminal device 2 (step St4). The received size information 134 is written in the HDD 13 and used to determine the size of the designated area L.

  Next, the division process of the screen 1a is performed (step St5), and the display data DT transmission process (step St6) is performed. The screen 1a dividing process and the display data DT transmitting process will be described later.

  Next, the session establishing unit 100 determines whether or not the session can be ended based on the operation of the input device 160 (step St7). If the session is not terminated (No in step St7), the process in step St5 is performed again. When the session is ended (Yes in step St7), the operation of the display processing program is ended. In this way, the display processing program of the display processing device 1 is executed.

  FIG. 6 is a flowchart of the display processing program of the terminal device 2. First, the session establishment unit 200 determines whether or not a session establishment request message has been received from the display processing device 1 (step St31). If the establishment request message has not been received (No in step St31), the reception session establishment unit 200 ends the process.

  When the establishment request message is received (Yes in step St31), the session establishment unit 200 transmits a response message to the display processing device 1 (step St32). Next, the session establishing unit 200 transmits the size information 134 of the screen 2a to the display processing device 1 (step St33).

  Next, the screen control unit 201 determines whether or not the display data DT has been received from the display processing device 1 (step St34). When the display data DT has not been received (No in step St34), the process in step St36 described later is performed. When the display data DT is received (Yes in step St34), the screen control unit 201 temporarily stores the display data DT in the RAM 22 and then displays it on the screen 2a (step St35).

  Next, the session establishment unit 200 determines whether or not the session can be terminated based on the notification from the session establishment unit 100 of the display processing device 1 (step St36). If the session is not terminated (No in step St36), the process in step St34 is performed again. When the session is ended (Yes in step St36), the operation of the display processing program is ended. In this way, the display processing program of the terminal device 2 is executed.

  Next, the division process (step St5) of the screen 1a executed in the display processing device 1 will be described. FIG. 7 is a flowchart showing the division process of the screen 1a.

  First, the screen control unit 101 acquires the scroll amount of the screen 1a generated by the operation of the input device 160 from the input processing unit 16, and determines whether or not the screen 1a is scrolled based on the scroll amount (step St11). When the screen 1a is scrolled (Yes in step St11), the screen control unit 101 extracts the screen display data 131 displayed on the screen 1a from the document data 130 in the HDD 13 based on the scroll amount (step St12). . That is, when the screen 1a is scrolled, the display content of the screen 1a is changed, so that the screen 1a is divided in order to update the region division information 132.

  If the screen 1a is not scrolled (No in step St11), the screen control unit 101 determines whether or not the area division information 132 has been stored in the HDD 13 (step St18). If the area division information 132 has been stored (Yes in step St18), the screen control unit 101 ends the process. On the other hand, when the area division information 132 has not been stored (No in step St18), the process in step St12 is performed. That is, even when the screen 1a is not scrolled, the area division information 132 does not exist immediately after the display processing program is started, so that the division process of the screen 1a is performed.

  Next, the dividing unit 102 divides the screen 1a into a plurality of unit areas by referring to the extracted screen display data 131 (step St13). FIG. 8 shows an example of screen division. FIG. 8A shows a state in which the screen 1a is divided into a plurality of unit areas U.

  The dividing unit 102 generates a plurality of unit regions U by dividing the screen 1a into a lattice shape. Each unit region U has, for example, a square shape with the same size, but is not limited thereto.

  Next, the dividing unit 102 counts the colors included in each unit region U (step St14). The number of colors is acquired from the color components of the screen display data 131, for example.

  Next, the dividing unit 102 determines the display contents (text, image, background) of each unit area U based on the number of colors (step St15). That is, the dividing unit 102 determines the type (attribute) of the unit region U according to the color distribution in the screen 1a.

  FIG. 9 is a graph illustrating an example of the distribution of the number of colors included in the unit region U. In FIG. 9, the horizontal axis represents the number of colors, and the vertical axis represents the number of unit areas U including the number of colors.

  The dividing unit 102 classifies the unit area U into a group G1 having 1 color, a group G2 having 2 to 10 colors, and a group G3 having 11 or more colors. Here, the unit area U belonging to the group G1 is located within the range of the background (margin portion) M in the document (see FIG. 1) displayed on the screen 1a, and the unit area U belonging to the group G2 is the screen 1a. Is located within the range of the texts T1 and T2 in the document displayed. The unit area U belonging to the group G3 is located within the range of the image G in the document displayed on the screen 1a.

  Next, the dividing unit 102 generates the attribute region by combining the adjacent unit regions U with the same display content type (step St16). FIG. 8B shows an example when the screen 1a shown in FIG. 1 is divided into a plurality of attribute areas.

  In this example, the screen 1a is divided into four attribute areas A1 to A4. The attribute area A1 corresponds to the text T1 in the document, and the attribute area A2 corresponds to the image G in the document. The attribute area A3 corresponds to the text T2 in the document, and the attribute area A4 corresponds to the background M in the document.

  Moreover, in this example, although attribute area A1-A4 has a rectangular shape, it is not limited to this, For example, you may have a circular shape. The dividing unit 102 assigns, for example, 1 to 4 (see “ID1” to “ID4”) as IDs (identifiers) to the attribute areas A1 to A4.

  Thus, the dividing unit 102 divides the screen 1a into a plurality of attribute areas A1 to A4 based on the color distribution in the screen 1a. Thereby, the dividing unit 102 can clearly divide the screen 1a into the attribute areas A1 to A4 of the texts T1 and T2, the image G, and the background M.

  Further, the dividing unit 102 determines the type of display content for each unit region U based on the number of colors, and the type of display content is the same, and the adjacent unit regions U are joined together to connect the attribute regions A1 to A1. A4 is generated. Therefore, the dividing unit 102 can divide the screen 1a with high accuracy. Note that the division of the screen 1a is not limited to the distribution of the number of colors, and may be performed based on the distribution of line segment elements, for example.

  Next, the dividing unit 102 records the generation results (division results) of the attribute areas A1 to A4 on the HDD 13 as area division information 132 (step St17). In this way, the screen 1a is divided.

  FIG. 10 shows an example of the area division information 132. The area division information 132 includes “ID”, “type”, “reference coordinates”, “width”, “height”, and “determination flag”.

  “ID” indicates an identifier of each of the attribute areas A1 to A4 as described above. The “reference coordinates” indicate the corner positions of the attribute areas A1 to A4 (in this example, the upper left corner in front view). “Type” indicates the type (text, image, background) of the display content of each attribute area A1 to A4.

  “Width” and “Height” indicate the horizontal and vertical lengths of the attribute areas A1 to A4. FIG. 8B shows, as an example, the reference coordinates P1, the width Wa1, and the height Ha1 of the attribute area A1.

  The “determination flag” indicates whether or not each attribute area A1 to A4 includes the position S of the pointer P. When the “determination flag” indicates “1”, the attribute areas A1 to A4 include the position S of the pointer P, and when the “determination flag” indicates “0”, the attribute areas A1 to A4 include the pointer P. Position S is not included. In the example of FIG. 10, the “determination flag” of the attribute area A2 (ID2) indicates “1”, and the “determination flags” of the other attribute areas A1, A3, A4 (ID1, 3, 4) are “0”. As shown, the position S of the pointer P is included in the attribute area A2.

  The “determination flag” is used and updated by the display update determination unit 103 in the display data DT transmission process. The display update determination unit 103 detects the position ranges of the attribute areas A1 to A4 based on the reference coordinates, width, and height, and compares them with the position S of the pointer P acquired from the input processing unit 16. Thereby, the display update determination unit 103 determines the attribute areas A1 to A4 including the position S of the pointer P (that is, the attribute areas A1 to A4 where the pointer P is located).

  When the determined attribute areas A1 to A4 do not match the attribute areas A1 to A4 whose “determination flag” indicates “1”, the display update determination unit 103 regards that the specified area L has changed, and displays data The generation unit 105 is instructed to generate display data DT. Then, the display update determination unit 103 updates the “determination flag” of the determined attribute areas A1 to A4 to “1”, and updates the “determination flag” of the other attribute areas A1 to A4 to “0”. If the determined attribute areas A1 to A4 match the attribute areas A1 to A4 whose “determination flag” indicates “1”, the generation instruction of the display data DT is not performed.

  Next, the display data DT transmission process will be described. FIG. 11 is a flowchart showing a display data DT transmission process.

  First, the display update determination unit 103 acquires the position S of the pointer P from the input processing unit 16 (step St21). Next, the display update determination unit 103 determines whether or not the position S of the pointer P has been changed (step St22). Here, the display update determination unit 103 holds the position S of the pointer P acquired last time, and compares the position S of the pointer P acquired last time with the position S of the newly acquired pointer P, thereby Determine presence or absence.

  If there is no change in the position S of the pointer P (No in step St22), the display update determination unit 103 ends the process. When the position S of the pointer P is changed (Yes in step St22), the display update determination unit 103 determines whether or not there is a change in the attribute areas A1 to A4 including the position S of the pointer P (step St23). That is, the display update determination unit 103 determines whether or not the position S of the pointer P has exceeded the boundary separating the plurality of attribute areas A1 to A4. For this determination, as described above, the position S of the pointer P acquired in step St21 and the “determination flag” of the area division information 132 are used.

  When there is no change in the attribute areas A1 to A4 including the position S of the pointer P (No in step St23), the display update determination unit 103 ends the process. When the attribute areas A1 to A4 including the position S of the pointer P change (Yes in step St23), the intermediate area generation unit 104 changes the designated area L after the change based on the position S of the pointer P and the size information 134. Is determined (step St24).

  Next, the intermediate area generation unit 104 reads the designated area information 133 from the HDD 13, and determines whether or not there is an overlapping portion between the designated area L before the change and the designated area L after the change (step St25). . When there is an overlapping portion (Yes in step St25), the display data generation unit 105 generates display data DT of the designated area L after the change from the screen display data 131 based on the position S of the pointer P and the size information 134. Then, the data is transmitted to the terminal device 2 via the communication processing unit 14 (Step St28). Hereinafter, an example in which there is an overlapping portion between the designated area L before update and the designated area L after change will be described.

  FIG. 12 shows an example of movement of the designated area L and display data DT. FIG. 12A shows a state where the pointer P has moved from the position S1 to the position S2.

  In this example, the position S1 of the pointer P before the movement is included in the attribute area A2 of the image G, and the position S2 of the pointer P after the movement is included in the attribute area A1 of the text T1. That is, the pointer P has moved beyond the boundary B separating the attribute area A2 and the attribute area A1. Therefore, the display update determination unit 103 determines that there is a change in the attribute areas A1 to A4 including the position S of the pointer P in the process of step St23.

  In addition, there is an overlapping portion between the designated area L1 before change defined by the position S1 of the pointer P before movement and the designated area L2 after change defined by the position S2 of the pointer P after movement. To do. Therefore, the display update determination unit 103 determines that there is an overlapping portion between the designated area L before the change and the designated area L after the change in the process of step St25. Therefore, the display data generation unit 105 generates the display data DT for the designated area L after the change without generating the display data DT for the intermediate area, and transmits the display data DT to the terminal device 2.

  Further, as described above, the size (width W3 × height H3) of the designated region L is set based on the size information 134 of the screen 2a of the terminal device 2. For example, assuming that the position S1 of the pointer P before movement is the coordinate (Xs, Ys), the position range (upper left corner to lower right corner) of the designated area L1 before change is (Xs−W3 / 2, Ys + H3 / 2) to (Xs + W3 / 2, Ys−H3 / 2).

  The display update determination unit 103 calculates the position ranges of the designated areas L1 and L2 before and after the change, and detects changes in the configuration of the attribute areas A1 to A4 included in the designated areas L1 and L2 from the area division information 132. Depending on the detection result, it may be determined whether or not the display data DT of the designated area L after the change can be generated. In the example of FIG. 12A, the designated area L1 before the change includes attribute areas A1 to A3, and the designated area L2 after the change includes attribute areas A1, A2, and A4. That is, the configuration of the attribute areas A1 to A4 included in the designated area L is changed by changing the designated area L. For this reason, the display update determination unit 103 instructs the display data generation unit 105 to generate the display data DT of the designated area L2 after the change.

  In this way, even when the designated area L is changed, the display processing device 1 only displays the display data DT of the designated area L2 after the change only when the configuration of the attribute areas A1 to A4 included in the designated area L changes. Transmit to device 2. For this reason, since the transmission frequency of the display data DT accompanying the change of the designation | designated area | region L is reduced, the time which the reception process of the display data DT in the terminal device 2 requires is reduced.

  FIGS. 12B and 12C show the display data DT of the designated areas L1 and L2 before and after the movement, respectively. That is, FIG.12 (b) and FIG.12 (c) show the screen 2a of the terminal device 2 before and after an update, respectively. Comparing FIG. 12B and FIG. 12C, the configuration of the display content on the screen 2a is changed before and after the change of the designated area L.

  Thus, since the screen 2a of the terminal device 2 is updated only when the configuration of the attribute areas A1 to A4 included in the designated area L is changed, fluctuations are prevented and the user can grasp the designated area L. Becomes easier.

  In this example, the display data DT of the attribute area A2 of the image G is common to the screen 2a before update and the screen 2a after update. For this reason, the display processing device 1 may omit transmission of the display data DT in the attribute area A2. In this case, the screen control unit 201 of the terminal device 2 holds the display data DT of the entire attribute area A2 in the RAM 22 in advance and outputs it to the updated screen 2a. Thereby, the display processing apparatus 1 can reduce the transmission amount of the display data DT.

  Referring to FIG. 11 again, if there is no overlapping portion between the designated area L before update and the designated area L after change (No in step St25), the intermediate area generation unit 104 determines that the designated area L before change is An intermediate area existing between the designated areas L after the change is determined (step St26). More specifically, the intermediate area generation unit 104 determines the position range of the intermediate area based on the size information 134 and the midpoint of the position S of the pointer P that defines each designated area L before and after the change. The intermediate region generation unit 104 notifies the display data generation unit 105 of the position range of the intermediate region together with an instruction to generate display data DT for the intermediate region.

  Next, the display data generation unit 105 reads the screen display data 131 from the HDD 13 and generates the display data DT for the intermediate area before generating and transmitting the display data DT for the specified area L after the change (step St28). It transmits to the terminal device 2 through the communication processing unit 14 (step St27). Next, the display data generation unit 105 performs the process of step St28 and ends the process. In this way, the display data DT is transmitted.

  FIG. 13 shows another example of the movement of the designated area L and the display data DT. FIG. 13A shows a state where the pointer P has moved from position S1 to position S3.

  In this example, the position S1 of the pointer P before the movement is included in the attribute area A2 of the image G, and the position S3 of the pointer P after the movement is included in the attribute area A1 of the text T1. That is, the pointer P has moved beyond the boundary B separating the attribute area A2 and the attribute area A1. Therefore, the display update determination unit 103 determines that there is a change in the attribute areas A1 to A4 including the position S of the pointer P in the process of step St23.

  In addition, there is an overlapping portion between the designated area L1 before change defined by the position S1 of the pointer P before movement and the designated area L3 after change defined by the position S3 of the pointer P after movement. do not do. Therefore, the intermediate area generation unit 104 calculates the position of the middle point Sm of the pointer P before the movement and the position S3 of the pointer P after the movement, and uses the middle point Sm as a reference based on the size information 134. The position range of the intermediate area Lm as the position is calculated. Here, since the size of the intermediate area Lm is the same as that of the designated area L, the position range of the intermediate area Lm is calculated by the same method as the calculation example of the position range of the designated area L1 described above.

  The display data generation unit 105 generates display data DT for the intermediate area Lm and transmits it to the terminal device 2 before generating and transmitting the display data DT for the designated area L3 after the change.

  FIGS. 13B and 13C show the display data DT of the intermediate area Lm and the designated area L3 after the movement, respectively. That is, FIG.13 (b) and FIG.13 (c) show the screen 2a of the terminal device 2 after the 1st time and the 2nd time update, respectively. Further, the display data DT of the designated area L1 before the movement, that is, the screen 2a of the terminal device 2 before the update is as shown in FIG. That is, the display data DT of FIGS. 12B, 13B, and 13C are displayed in this order on the screen 2a of the terminal device 2.

  Comparing FIG. 12B and FIG. 13C, the display data DT of the designated area L1 before movement and the display data DT of the designated area L3 after movement are common to only the attribute area A1 of the text T1. For this reason, if the display data DT (FIG. 13B) of the intermediate area Lm is not displayed on the screen 2a of the terminal apparatus 2, the user of the terminal apparatus 2 sets the destination of the specified area L1 (specified area L3). It is difficult to grasp.

  On the other hand, comparing FIG. 12B and FIG. 13B, the display data DT of the designated area L1 and the display data DT of the intermediate area Lm before the movement are not only the attribute area A1 of the text T1, but also the image. The attribute area A2 of G is also common. Further, comparing FIG. 13B and FIG. 13C, the display data DT of the intermediate area Lm and the display data DT of the designated area L3 after the movement are not only the attribute area A1 of the text T1 but also the background M. The attribute area A4 is also common.

  Therefore, after the display data DT of the designated area L1 before the movement is displayed on the screen 2a of the terminal device 2, the display data DT of the intermediate area Lm is displayed before the display data DT of the designated area L3 after the movement is displayed. By displaying, the user of the terminal device 2 can easily grasp the destination of the designated area L1. That is, the display processing device 1 complements the display data DT of the designated area L1 before movement and the display data DT of the designated area L3 after movement, thereby smoothly updating the screen 2a of the terminal device 2.

  As described so far, the display processing device 1 according to the embodiment converts the display data DT of the designated area L designated in the first screen 1a to another device having a second screen 2a smaller than the first screen 1a ( By transmitting it to the terminal device 2, it is displayed on the second screen 2 a. The dividing unit 102 divides the first screen 1a into a plurality of areas (attribute areas) A1 to A4 based on at least two types of display contents among text, image, and background. When the designated area L is changed, the transmitting unit 10a changes one or more areas included in the designated area L to one or more other areas among the plurality of areas A1 to A4. The display data DT of the designated area L is transmitted to the other device 2.

  According to the above configuration, even when the designated region L is changed, the transmission unit 10a displays the display data DT of the designated region L after the change only when the configuration of the region in the designated region L is changed to the configuration of another region. Is transmitted to the other device 2. For this reason, since the transmission frequency of the display data DT accompanying the change of the designation | designated area | region L is reduced, the time which the receiving process of the display data DT in the other apparatus 2 requires is reduced.

  As a result, the second screen 2a of the other device 2 is updated only when the configuration of the area in the designated area L is changed to the configuration of the other area. Is easier to grasp. Therefore, according to the display processing device 1 according to the embodiment, the screen 2a of the other device 2 can be updated efficiently.

  In addition, the display processing method according to the embodiment uses the display data DT of the designated area L designated in the first screen 1a of the first device (display processing device) 1 as the second screen 2a smaller than the first screen 1a. It is a method of displaying on the 2nd screen 2a by transmitting to the 2nd apparatus (terminal device) 2 which has. In the display processing method according to the embodiment, the computer executes the following steps.

Process (1): The 1st screen 1a is divided | segmented into several area | region (attribute area | region) A1-A4 based on the classification of the display content divided into at least 2 types among a text, an image, and a background.
Step (2): When the designated area L is changed, when one or more areas included in the designated area L among the plurality of areas A1 to A4 are changed to one or more other areas, the changed area The display data DT of the designated area L is transmitted to the second device 2.

  Since the display processing method according to the embodiment includes the same configuration as that of the display processing device 1 described above, the same effects as those described above can be achieved.

  Further, the display processing program according to the embodiment displays the display data DT of the designated area L designated in the first screen 1a of the first device (display processing device) 1 as the second screen 2a smaller than the first screen 1a. The program is displayed on the second screen 2a by being transmitted to the second device (terminal device) 2 having the second device. The display processing program according to the embodiment causes the computer to execute the following processing.

Process (1): The first screen 1a is divided into a plurality of areas (attribute areas) A1 to A4 based on at least two types of display contents among text, image, and background.
Process (2): When the designated area L is changed, when one or more areas included in the designated area L among the plurality of areas A1 to A4 are changed to one or more other areas, the changed area The display data DT of the designated area L is transmitted to the second device 2.

  Since the display processing program according to the embodiment includes the same configuration as that of the display processing device 1 described above, the same effects as those described above can be obtained.

  The above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the processing apparatus should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium (except for a carrier wave).

  When the program is distributed, for example, it is sold in the form of a portable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

  The above-described embodiment is an example of a preferred embodiment of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

In addition, the following additional notes are disclosed regarding the above description.
(Supplementary note 1) In the display processing device that displays the display data of the designated area designated in the first screen to the other device having the second screen smaller than the first screen, and displays it on the second screen.
A dividing unit that divides the first screen into a plurality of regions based on a type of display content divided into at least two of a text, an image, and a background;
When one or more areas included in the designated area are changed to one or more other areas among the plurality of areas by changing the designated area, display data of the designated area after the change is changed. A display processing apparatus comprising: a transmission unit configured to transmit to the other apparatus.
(Supplementary Note 2) When the reference position defining the designated area exceeds a boundary separating the plurality of areas, the transmitting unit transmits the display data of the designated area after the change to the other device. The display processing apparatus according to Supplementary Note 1, wherein the display processing apparatus is characterized.
(Supplementary Note 3) When there is no overlapping portion between the designated area before the change and the designated area after the change, the transmission unit transmits the display data of the designated area after the change, The display processing apparatus according to appendix 1 or 2, wherein display data of an intermediate area existing between the designated area before the change and the designated area after the change is transmitted to the other apparatus.
(Supplementary note 4) The display according to any one of Supplementary notes 1 to 3, wherein the dividing unit divides the first screen into the plurality of regions based on a color distribution in the first screen. Processing equipment.
(Supplementary Note 5) The dividing unit is
Dividing the first screen into a plurality of unit areas;
According to the number of colors included in each of the unit areas, determine the type of display content for each unit area,
The display processing apparatus according to appendix 4, wherein the types of display contents are the same, and each of the plurality of areas is generated by combining adjacent unit areas.
(Supplementary Note 6) Display data of a designated area designated in the first screen of the first device is displayed on the second screen by transmitting it to a second device having a second screen smaller than the first screen. In the display processing method,
Dividing the first screen into a plurality of regions based on at least two types of display content among text, image, and background; and
When one or more areas included in the designated area are changed to one or more other areas among the plurality of areas by changing the designated area, display data of the designated area after the change is changed. A display processing method, wherein the computer executes the step of transmitting to the second device.
(Supplementary Note 7) In the step of transmitting the display data of the designated area, when the reference position that defines the designated area exceeds the boundary separating the plurality of areas, the display data of the designated area after the change is The display processing method according to appendix 6, wherein the display processing method is transmitted to another device.
(Supplementary note 8) In the step of transmitting the display data of the designated area, when there is no overlapping portion between the designated area before the change and the designated area after the change, the display of the designated area after the change Appendix 6 or 7, wherein display data of an intermediate area existing between the designated area before the change and the designated area after the change is transmitted to the other device before transmitting the data. Display processing method.
(Supplementary note 9) In the step of dividing the first screen into the plurality of regions, the first screen is divided into the plurality of regions based on a color distribution in the first screen. The display processing method according to any one of 6 to 8.
(Supplementary Note 10) In the step of dividing the first screen into the plurality of regions,
Dividing the first screen into a plurality of unit areas;
According to the number of colors included in each of the unit areas, determine the type of display content for each unit area,
The display processing method according to appendix 9, wherein each of the plurality of regions is generated by combining the adjacent unit regions with the same display content type.
(Appendix 11) Display data of a designated area designated in the first screen of the first device is displayed on the second screen by transmitting it to a second device having a second screen smaller than the first screen. In the display processing program,
The first screen is divided into a plurality of areas based on at least two types of display contents among text, image, and background,
When one or more areas included in the designated area are changed to one or more other areas among the plurality of areas by changing the designated area, display data of the designated area after the change is changed. A display processing program for causing a computer to execute processing to be transmitted to the second device.
(Supplementary Note 12) In the process of transmitting the display data of the designated area, when the reference position that defines the designated area exceeds the boundary separating the plurality of areas, the display data of the designated area after the change is The display processing program according to appendix 11, wherein the display processing program is transmitted to another device.
(Additional remark 13) In the process which transmits the display data of the said designated area | region, when there is no overlapping part between the said designated area | region before this change and the said designated area after this change, the display of the said designated area | region after this change The additional data 11 or 12, wherein display data of an intermediate area existing between the designated area before the change and the designated area after the change is transmitted to the other device before the data is transmitted. Display processing program.
(Supplementary Note 14) In the process of dividing the first screen into the plurality of regions, the first screen is divided into the plurality of regions based on a color distribution in the first screen. The display processing program according to any one of 11 to 13.
(Supplementary Note 15) In the process of dividing the first screen into the plurality of regions,
Dividing the first screen into a plurality of unit areas;
According to the number of colors included in each of the unit areas, determine the type of display content for each unit area,
15. The display processing program according to appendix 14, wherein each of the plurality of areas is generated by combining the adjacent unit areas with the same display content type.

1 Display processing device (first device)
1a screen (first screen)
2 Terminal devices (second device, other devices)
2a screen (second screen)
10,20 CPU
10a Transmission unit 102 Dividing unit L Designated area P Pointer S Position (reference position)
DT display data A1-A4 Attribute area U Unit area

Claims (7)

In the display processing device for displaying on the second screen by transmitting the display data of the designated area designated in the first screen to another device having a second screen smaller than the first screen,
A dividing unit that divides the first screen into a plurality of regions based on a type of display content divided into at least two of a text, an image, and a background;
When one or more areas included in the designated area are changed to one or more other areas among the plurality of areas by changing the designated area, display data of the designated area after the change is changed. A display processing apparatus comprising: a transmission unit configured to transmit to the other apparatus.   The transmission unit, when a reference position defining the designated area exceeds a boundary separating the plurality of areas, transmits the display data of the designated area after the change to the other device. Item 4. The display processing device according to Item 1.   When there is no overlapping part between the designated area before the change and the designated area after the change, the transmission unit transmits the display data of the designated area after the change before transmitting the display data of the designated area after the change. The display processing apparatus according to claim 1, wherein display data of an intermediate area existing between the designated area and the designated area after the change is transmitted to the other apparatus.   The display processing apparatus according to claim 1, wherein the division unit divides the first screen into the plurality of regions based on a color distribution in the first screen. The dividing unit is
Dividing the first screen into a plurality of unit areas;
According to the number of colors included in each of the unit areas, determine the type of display content for each unit area,
The display processing apparatus according to claim 4, wherein the types of display contents are the same, and each of the plurality of areas is generated by combining adjacent unit areas. In a display processing method of displaying on a second screen by transmitting display data of a designated area designated in the first screen of the first device to a second device having a second screen smaller than the first screen. ,
Dividing the first screen into a plurality of regions based on at least two types of display content among text, image, and background; and
When one or more areas included in the designated area are changed to one or more other areas among the plurality of areas by changing the designated area, display data of the designated area after the change is changed. A display processing method, wherein the computer executes the step of transmitting to the second device. In a display processing program for displaying on a second screen by transmitting display data of a designated area designated in the first screen of the first device to a second device having a second screen smaller than the first screen. ,
The first screen is divided into a plurality of areas based on at least two types of display contents among text, image, and background,
When one or more areas included in the designated area are changed to one or more other areas among the plurality of areas by changing the designated area, display data of the designated area after the change is changed. A display processing program for causing a computer to execute processing to be transmitted to the second device.

Images