JP4811042B2 - Client device and program - Google Patents

Description

  The present invention is configured such that a client device that makes a service request to a server device that operates an application program and a server device that operates an application program based on the service request from the client device can be connected to each other via a network. Server-based computing system.

In recent years, the adoption of server-based computing by companies has been rapidly progressing.
Server-based computing is a system in which a client device and a server device can be connected to each other via a network. When a client device makes a service request to the server device, the server device receives a request from the client device. Data management and application programs are operated based on service requests (see, for example, Patent Document 1 and Non-Patent Document 1).

  The server device creates display data for the client and sends it to the client device. In a client device (also called a thin client device), operation information such as a user's key and mouse is sent to a server device via a network, and display data for a client created on the server device side is received and displayed. . The client device only needs to include an input device such as a network device, a display device, and a mouse and a keyboard. Since no data is stored in the client device, storage such as a hard disk is not required, and application programs are not executed. Therefore, the processing load required for the client device is light.

By introducing server-based computing, companies can manage resources such as data and application programs on the server device side, which has the advantage of reducing operational costs. In addition, the client device has an advantage that it can be procured at a lower cost than a conventional personal computer. Also in terms of security, it is possible to reduce the risk of unauthorized access to information by restricting access to data in the server device and collecting access logs. It is possible to deal with viruses in a batch with the server device.
JP 2005-228227 A http: // www. keyman. or. jp / search / 30000031_1. html (server-based computing)

  As described above, when server-based computing is executed, a client device that makes a service request to the server device that operates the application program, and a server that operates the application program based on the service request from the client device The apparatus is configured to be connectable to each other via a network.

  When the client device is a mobile terminal device, it is a client device that places importance on mobility, so that the display screen cannot be made large. For example, a 4-inch HVGA (640 × 240) is much smaller than a standard desktop personal computer display size of 15 inches, XGA (1024 × 768). Therefore, every time the display data of the client device is changed in response to the operation of the client device, it is necessary to transmit the display data from the server device to the client device. In particular, when the display screen of the client device is small, there is a problem that access to the server device 101 frequently occurs and the power consumption of the mobile terminal device 104 is increased. Furthermore, when the mobile terminal device is battery-driven, it can be used only for the time corresponding to the installed battery capacity. Also, a large battery pack cannot be used to maintain mobility. In other words, performing server-based computing on a mobile terminal device means using a radio unit that occupies most of the power consumption in the mobile terminal device, which greatly limits the usage time of the mobile terminal device. There was a problem.

  Therefore, the present invention has been made in view of the above problems, and in server-based computing, the frequency of communication between the server device and the client device is reduced, and the power consumption of the client device having mobility is achieved. It is an object of the present invention to provide a server-based computing system capable of displaying an enlarged image with reduced image quality.

The present invention employs the following configuration in order to solve the above problems.
That is, according to one aspect of the present invention, the client apparatus of the present invention is a client apparatus that makes a service request to a server apparatus that operates an application program via a network, and display means for displaying an image A frame memory area for temporarily storing the image for displaying the image on the display means, and an enlargement instruction input means for inputting an instruction for enlarging and displaying the image displayed on the display means Determining means for determining whether the enlargement instruction input by the enlargement instruction input means is greater than or equal to a predetermined value; and determining by the determination means that the enlargement instruction is greater than or equal to a predetermined value. In this case, the server enlargement image request for transmitting the server enlargement image enlarged in response to the enlargement instruction to the server device is required. Means, server enlarged image receiving means for receiving the server enlarged image transmitted from the server device, and memory control means for storing the server enlarged image received by the server enlarged image receiving means in the frame memory area. When the determination means determines that the enlargement instruction is not greater than or equal to a predetermined value, the client image enlargement means obtains a client enlargement image corresponding to the enlargement instruction from the image stored in the frame memory area. And display control means for controlling to display the server enlarged image stored in the frame memory area or the client enlarged image enlarged by the client image enlarging means on the display means.

  According to the present invention, in server-based computing, it is possible to reduce the frequency of communication between the server device and the client device, reduce the power consumption of the client device, and display an enlarged image.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing an outline of a network configuration of server-based computing.

In FIG. 1, a server apparatus 101 has a function of executing an application program, a function of managing data, and the like.
The client terminal device 102 is connected to the server device 101 via the network 109. The wireless access point 103 is also connected to the server apparatus 101 via the network 109. In addition, mobile terminal devices such as the mobile terminal device 104 and the smartphone 105 are connected to the server device 101 via the network 9 by performing wireless communication with the access point 103. In the following description, the client terminal device 102, the mobile terminal device 104, and the smartphone 105 are collectively referred to as a client device.

  The client terminal device 102, the mobile terminal device 104, or the smartphone 105 constructs a server-based computing system with the server device 101.

  The mobile terminal device 104 is connected to the server device 101 via a wireless network (for example, WiFi) and the access point 103. A mobile terminal device such as the mobile terminal device 104 is designed with low power consumption in consideration of battery life.

FIG. 2 is a diagram illustrating an example of a breakdown of power consumption of the mobile terminal device 104.
The mobile terminal device 104 consumes power mainly in the LSI unit, the display unit, and the radio unit as shown in FIG. In the example shown in FIG. 2, the wireless unit is operating 100%. If the non-communication period with the server apparatus 101 can be long, the wireless unit can be powered down. However, in general, communication is frequently performed.

  As shown in FIG. 2, the power consumption of the wireless unit occupies 65% of the power consumed by the mobile terminal device. Since the mobile terminal device 104 is battery-driven, the mobile terminal device 104 can only use time according to the installed battery capacity, and performing server-based computing in the mobile terminal device 104 means that the power consumption in the mobile terminal device 104 is reduced. The radio unit occupies the majority, and this limits the usage time of the mobile terminal device 104.

  The server-based computing system to which the present invention is applied has a frame buffer in the mobile terminal device 104. For example, for enlargement up to 2 times corresponding to the enlargement instruction, the data in the frame buffer is interpolated. Enlarged display is performed, and the enlargement of 2 times or more is processed on the server apparatus 101 side. As a result, communication associated with frequent enlargement processing can be reduced, and power consumption in the mobile terminal device 104 can be reduced.

FIG. 3 is a diagram showing an outline of the configuration of the mobile terminal device 104 to which the present invention is applied.
In FIG. 3, the mobile terminal device 104 includes an LSI 200, an LCD 210, a program storage memory 212, and a work memory 213.

  The LSI 200 is configured by SOC (System On Chip). The internal configuration of the LSI 200 will be described. The CPU 201 controls communication with each unit via an internal bus (Bus) 204a. A memory controller (Mem. Cont.) 204 controls the internal bus 204b, a work memory 213 such as SDRAM, and a program storage memory 212 such as FROM. The LCDC (LCD controller) 202 uses the built-in memory 203 such as a RAM built in the LSI 200 to transfer display data to the LCD 210 (display unit). Further, it is configured by a UART (Universal Asynchronous Receiver) 206, a power management block (Power Mgmt) 207, a real-time counter (RTC) 205, a USB function (USBF) 208, and the like. The wireless communication unit 209 enables communication by WiFi and is connected to the UART 206.

FIG. 4 is a diagram showing an internal configuration of the LCDC.
In FIG. 4, the LCDC 202 is connected to the internal bus 204 a of the LSI 200 via a bus interface (BUS I / F) 301. The register (Reg.) 302 stores various setting information for the LCDC 202. A memory controller (Memory Cont.) 303 controls a built-in memory (memory for storing display data) 203. The memory controller 303 includes a DMAC (direct memory access controller) 303a. The DMAC 303 a manages data transfer from the built-in memory 203 to the LCD 210.

  The display data read from the internal memory 203 by the DMAC 303a is processed at a desired enlargement ratio by a resizer (Resizer) 305 having an enlargement / reduction function.

  Then, the FIFO 306 synchronizes, and the display data is transferred to the LCD 210 (display unit) via the LCD I / F 311 to display an image. The LCD 210 of the mobile terminal device 104 has an HSVGA (800 × 300) display area, while the built-in memory 203 of the mobile terminal device 104 has an HSVGA (800 × 300) size frame buffer. The server apparatus 101 also executes an application with a frame buffer corresponding to the display size of HSVGA (800 × 300).

FIG. 5 is a diagram showing an outline of the configuration of the server device.
In FIG. 5, the server device 101 includes a CPU (Central Processing Unit) 701, a program for executing processing executed in the server device 101, and a control program for controlling and executing each function of the server device 101. A memory 702 such as a ROM or a RAM in which an image is stored, an input unit 703 for inputting various data and signals, a display unit 704 for displaying images and other information, and recording various information including images. A storage unit 705 for driving, a recording medium driving unit 706 for driving an external recording medium, and a network connection unit 707 for connecting to a network 109 such as a LAN are connected to the bus 700, and the CPU 701 controls each of these units. ing.

Next, a first embodiment to which the present invention is applied will be described.
FIG. 6 is a flowchart showing the flow of image enlargement processing in the first embodiment to which the present invention is applied.

  6, step 400 to step 406 are flowcharts showing the flow of image enlargement processing of the mobile terminal device 104, and step 410 to step 412 are processing of the server device 101 in response to the image enlargement processing of the mobile terminal device 104. It is a flowchart to show.

  When the enlargement operation of the image displayed on the LCD 210 is performed on the mobile terminal device 104, first, in step S400, the CPU 201 of the mobile terminal device 104 presses the key for instructing the input operation of enlargement magnification or enlargement. The enlarging operation is detected by counting. In step 401, the CPU 201 analyzes the contents of the enlargement operation to obtain a value corresponding to the enlargement operation, and the value is equal to or greater than a predetermined value (here, the value corresponding to the enlargement rate is double). It is determined whether or not the value is an instruction for enlargement. Enlarging the image is because it is difficult to recognize the displayed image. That is, the large enlargement ratio means that the image is to be viewed more clearly.

  If it is determined that an enlargement ratio of 2 times or more has been requested (step 401: Yes), in step 402, the CPU 210 notifies the enlargement position and enlargement ratio to the server apparatus 101 and requests enlarged display data. Note that the case where it is determined that an enlargement ratio of less than twice has been requested (step 401: No) will be described later.

  In step 410, when the server apparatus 101 receives a display data request from the mobile terminal apparatus 104, the server apparatus 101 performs enlargement of a desired portion at an enlargement rate according to the request in step 411. The enlargement in the server apparatus 101 is not a bitmap enlargement but a vector enlargement, and the enlargement is performed more smoothly. As a vector enlargement method, an enlargement ratio changing function, which is one of the functions possessed by an application, is used, or an enlargement function of an OS (operation system) on the server apparatus 101 side is used to enlarge the whole. Good. Here, since it is the character information that is desired to be preferentially and smoothly enlarged, vector enlargement in which the character information is enlarged by vector processing is selected as the enlargement method. In step 412, the server apparatus 101 compresses the expanded data and transmits the compressed data to the mobile terminal apparatus 104.

  On the mobile terminal device 104 side, in step 403, the mobile terminal device 104 receives data transmitted from the server device 101, and in step 404, the CPU 201 updates the contents of the frame buffer (FB) with the received data. Here, the entire frame buffer is a partially enlarged image.

  In step 405, the CPU 201 designates the display position in the frame buffer and designates the enlargement ratio to the resizer 305. The memory controller 303 selectively acquires data from the frame buffer of the built-in memory 203 according to the display position specified above. Here, since the image in the frame buffer is a partially enlarged image, the entire frame buffer is selected as the display position, and the enlargement ratio to the resizer 305 is 1 (equal magnification). That is, the contents of the frame buffer itself are transferred to the LCD 210, and an enlarged image is displayed in step 406.

FIG. 7 is a diagram illustrating a state of the frame buffer and the display unit of the mobile terminal device when the screen enlargement process is executed in the server device.
7A shows the display area 501 before enlargement displayed on the LCD 210 of the mobile terminal device 104, and FIG. 7B shows the state of the frame buffer 500 of the mobile terminal device 104 in the situation of FIG. Is shown.

  7C shows that when the display area 501 is displayed as shown in FIG. 7A, the part to be enlarged and displayed is the enlarged area 502 from now on. This enlarged area 502 is enlarged by the server apparatus 101 and held in the frame buffer 500 of the mobile terminal apparatus 104. FIG. 7D shows the state of the frame buffer 500 after the enlarged area 502 is enlarged, and FIG. 7E shows the display area 501 displayed on the LED 210 in the enlarged state.

  Returning to the description of FIG. 6, when an enlargement rate of less than 2 times is requested in step 401 (step 401: No), for example, when an enlargement rate of 1.8 times is requested, the client terminal device 104 determines that the server Access to the device 101 is not performed. That is, in step 405, the CPU 201 of the client terminal device 104 designates an area to be partially enlarged as a display position and designates an enlargement ratio to the resizer 305 as 1.8. Then, the memory controller 303 selectively acquires the data of the designated display position from the frame buffer 500 and enlarges it by 1.8 times by the resizer 305. The enlargement of the image is executed by the resizer 305 performing interpolation using a linear interpolation method or a bicubic method. In step 406, the result is displayed on the LCD 210 via the LCD I / F 307.

FIG. 8 is a diagram illustrating a state of the frame buffer and the display unit of the mobile terminal device when the screen enlargement process is executed in the mobile terminal device.
8, (A) shows the display area 501 before enlargement displayed on the LCD 210 of the mobile terminal device 104, as in (A) of FIG. 7, and (B) shows the display area 501 in FIG. Similarly, the state of the frame buffer 500 of the mobile terminal device 104 in the situation of (A) is shown.

  8C shows that when the display area 501 is displayed as shown in FIG. 8A, the part to be enlarged and displayed is the enlarged area 502 from now on. Since the expansion area 502 is expanded by the mobile terminal device 104, the contents of the frame buffer 500 of the mobile terminal device 104 are not changed.

  8D shows the state of the frame buffer 500 after the enlargement area 502 is enlarged, and FIG. 8E shows the display area 501 displayed on the LED 210 in the state after enlargement. That is, when the user gives an instruction to partially enlarge the enlargement area 502 in the display screen (display area 501) of FIG. 8C, the mobile terminal device 104 displays the frame buffer 500 in the mobile terminal device 104. As shown in FIG. 8D, the contents of the frame buffer 500 are not changed, but the display area is designated in the area 503 by the CPU 201, and an enlarged image is displayed as shown in FIG. 8E. .

  In the description of FIG. 6, when the server apparatus 101 enlarges the image, the image in the frame buffer is the same as the display image (enlargement ratio 1 to the resizer 305). May be received as an image in a larger area than the part to be enlarged, and stored in a frame buffer, and the enlargement ratio (a value greater than 1 and less than 2) of the resizer 305 may be set.

Next, a second embodiment to which the present invention is applied will be described.
FIG. 9 is a flowchart showing the flow of image enlargement processing in the second embodiment to which the present invention is applied.

  As a form of use of partial enlargement as in the first embodiment described above, there are many cases where it is desired to see an image enlarged by one dimension. For example, when an image is being browsed and a fine character appears in the image, the portion is enlarged only when the portion is read, and when the reading is completed, the original magnification is restored and the view is continued. The second embodiment is an image enlargement process in such a case, and is a modification of the first embodiment.

  The second embodiment shown in FIG. 9 is different from the first embodiment shown in FIG. 6 in that step 704 is executed instead of step 404, and the other steps are the same as those in FIG. It is the same. That is, in the first embodiment described with reference to FIG. 6, the frame buffer is updated in step 404, but in the second embodiment, the mobile terminal device 104 is changed to the server device 101 in step 403. When the data transmitted from is received, the frame buffer is not directly updated, and in step 704, the subframe buffer dedicated for enlargement provided in the internal memory 203 is updated. This subframe buffer is used only when partial enlarged data is received from the server apparatus 101. That is, a subframe buffer is used for partial enlargement using the server apparatus 101, and a normal frame buffer is used otherwise. When the normal enlargement processing is returned from the partial enlargement, the server apparatus 101 is not accessed and the frame buffer holding the data before the partial enlargement is used.

  By executing the first and second embodiments, it is possible to reduce communication frequency and reduce power consumption in server computing using the mobile terminal device 104 with a narrow display screen.

Next, a third embodiment to which the present invention is applied will be described.
FIG. 10 is a flowchart showing the flow of image enlargement processing in the third embodiment to which the present invention is applied.

The third embodiment to which the present invention is applied is also a modification of the first embodiment, similarly to the second embodiment.
The third embodiment shown in FIG. 10 differs from the first embodiment shown in FIG. 6 in that step 801 is executed instead of step 401, and the other steps are the same as those in FIG. It is the same. That is, in the first embodiment described with reference to FIG. 6, whether the enlargement process is executed by the server apparatus 101 or the mobile terminal apparatus 104 is determined based on the enlargement ratio (step 401 in FIG. 6). In the third embodiment, determination is made by image processing in step 801. Here, the determination by image processing means that when an image in the enlarged region designated by the user is converted from the spatial domain to the frequency domain using, for example, discrete cosine transform, a fine image is included. Since many high-frequency components are detected, it is determined that vector processing is necessary when a certain amount of high-frequency components is included, and it is determined whether or not the server apparatus 101 is to execute the enlargement process, or partial enlargement A determination is made as to whether or not to cause the server apparatus 101 to execute vector processing on the assumption that a fine image is included if the density difference with the adjacent pixels is accumulated over a certain area and the amount exceeds a certain value. . This determination is performed by dedicated hardware (not shown) connected to the memory controller 303 in the LCDC 202 or the CPU 201.

  In this way, by causing the mobile terminal device 104 to recognize an image, in server computing using the mobile terminal device 104 with a narrow display screen, it is possible to naturally perform an enlargement operation while suppressing total power consumption.

Next, fourth to sixth embodiments to which the present invention is applied will be described.
In the first and second embodiments described above, whether the enlargement process is executed by the server device 101 or the mobile terminal device 104 is determined based on the enlargement rate. In a form, it determines by specific key operation.

FIG. 11 is a diagram illustrating an overview of a mobile terminal device.
In FIG. 11, the mobile terminal device 104 includes an LCD 210 and a keyboard 904. The keyboard 904 includes a cross cursor key including an up cursor key 902a, a right cursor key 902b, a down cursor key 902c, and a left cursor key 902d, and a pointer 901 that is normally operated with the right hand at the center of the cross cursor key. The keyboard 904 includes a left click key 901a and a right click key 901b. When the pointer 901 is operated with the right hand, the right click and the left click operate the left click key 901a and the right click key 901b with the left hand. The keyboard 904 has a slide bar 910 for general use at a position where it can be easily operated with the right hand. The slide bar 910 has a function of scrolling up and down. For example, when the slide bar 910 is operated with the left hand while pressing the shift key 905, the display screen is reduced by the upward slide operation, and the display screen is enlarged by the downward slide operation. It becomes.

First, a fourth embodiment to which the present invention is applied will be described.
In the fourth embodiment, whether the enlargement process is executed by the server device 101 or the mobile terminal device 104 is determined based on the slide speed of the slide key 905.

  Specifically, it is determined whether the enlargement process is executed by the server device 101 or the mobile terminal device 104 at a speed at which the slide key 910 is slid while the shift key 905 is pressed. When the speed is less than a predetermined slide speed (normal slide operation), the enlargement process is performed in the mobile terminal device 104. When the speed is higher than the predetermined speed (quick slide operation), the enlargement process is performed in the server apparatus 101.

FIG. 12 is a flowchart showing a flow of image enlargement processing in the fourth embodiment to which the present invention is applied.
When an enlargement operation is performed by operating a slide key 910 provided in the mobile terminal device 104, first, in step S1200, the CPU 201 of the mobile terminal device 104 detects an enlargement operation by the slide key 910. In step 1201, the CPU 201 determines whether or not the speed at which the slide key 910 is slid is faster than a predetermined speed. If it is determined to be fast (step 1201: early), the enlargement process is executed in the server apparatus 101 in step 1204. On the other hand, if it is determined that it is late (step 1201: late), the enlargement process is executed in the mobile terminal device 104 in step 1202. As a human operation, when the user wants to see the screen in detail, the slide key 910 is operated quickly, so that the enlargement process in the server apparatus 101 is performed. Here, “server processing” in step 1204 corresponds to steps 402, 410 to 412, and 403 to 406 in FIG. 6, and “terminal processing” in step 1202 directly proceeds from step 401 in FIG. This corresponds to steps 405 and 406.

As described above, when the user wants to see the screen in detail, the server apparatus 101 can perform an enlargement process in which the detailed portion can be seen by performing a natural operation of sliding quickly.
Next, a fifth embodiment to which the present invention is applied will be described.

In the fifth embodiment, whether the enlargement process is executed by the server apparatus 101 or the mobile terminal apparatus 104 is determined based on the continuous operation degree of the cross cursor key.
When the up cursor key 902a is pressed while pressing the shift key 905, the enlargement operation is performed, and when the down cursor key 902c is pressed while the shift key 905 is pressed, the reduction operation is performed. When the enlargement operation (pressing the up cursor key 902a while pressing the shift key 905) is continuously executed, the enlargement process is performed in the server apparatus 101. For example, when the enlargement operation is performed within 2 seconds, the enlargement process is performed on the server apparatus 101 side.

FIG. 13 is a flowchart showing the flow of image enlargement processing in the fifth embodiment to which the present invention is applied.
When an enlargement operation is performed by operating the up cursor key 902a included in the mobile terminal device 104, first, in step S1200, the CPU 201 of the mobile terminal device 104 detects the enlargement operation by the up cursor key 902a. In step 1301, it is determined whether the timer TM is zero. The timer TM is a subtraction timer and is a timer that subtracts a set value to zero. The timer TM does not become 0 or less. In addition, since the timer TM has already been reset to 0 in an initial setting unit (not shown), the timer TM is 0 when it is first called (step 1301: Yes) (because it is not subtracted below 0). In step 1202, enlargement processing is executed by the mobile terminal device 104. Next, in step 1303, 2 seconds are set in the timer TM. The timer TM starts subtraction as soon as the value is set.

  Then, when the next enlargement operation is performed (step 1200), in step 1301, it is determined whether the timer TM is 0, in other words, whether 2 seconds have passed since the previous operation. Here, if the elapse is within 2 seconds, TM is not 0 (step 1301: No), and therefore, in step 1204, the enlargement process is executed by the server apparatus 101. As a human operation, when the user wants to look at the screen in detail, the enlargement operation is continued, so that the enlargement process in the server apparatus 101 is performed.

As described above, when the user wants to see the screen in detail, the server apparatus 101 can perform an enlargement process in which the detailed portion is visible by performing a natural operation of hitting repeatedly.
Next, a sixth embodiment to which the present invention is applied will be described.

In the sixth embodiment, whether the enlargement process is executed by the server device 101 or the mobile terminal device 104 is determined by the pressing force of the cross cursor key.
In the sixth embodiment, as in the fifth embodiment, when the up cursor key 902a is pressed while the shift key 905 is pressed, the enlargement operation is performed, and when the down cursor key 902c is pressed while the shift key 905 is pressed, the reduction operation is performed. . Further, the up cursor key 902a has a function of detecting a pressing force. For example, the contact point of the upper cursor key 902a is configured by a resistance film, and the amount of pressing can be detected by detecting the resistance of the contact point. When the enlargement operation is performed by strongly pressing the up cursor key 902a, the enlargement process in the server apparatus 101 is performed.

FIG. 14 is a flowchart showing the flow of image enlargement processing in the sixth embodiment to which the present invention is applied.
When an enlargement operation is performed by operating the up cursor key 902a included in the mobile terminal device 104, first, in step S1200, the CPU 201 of the mobile terminal device 104 detects the enlargement operation by the up cursor key 902a. In step 1401, the CPU 201 determines whether or not the force for pressing the up cursor key 902a is larger than a predetermined value. If it is determined that the value is large (step 1401: large), the enlargement process is executed in the server apparatus 101 in step 1204. On the other hand, if it is determined that it is small (step 1401: small), the enlargement process is executed in the mobile terminal device 104 in step 1202.

As described above, when the user wants to see the screen in detail, the server apparatus 101 can perform an enlargement process in which the detailed portion can be seen by performing a natural operation of pressing the screen strongly.
Next, a seventh embodiment to which the present invention is applied will be described.

  The seventh embodiment is basically the same as the first embodiment described with reference to FIG. 6, and enlargement processing is executed by the server apparatus 101 or enlarged by the mobile terminal apparatus 104 depending on the enlargement ratio. Execute the process. In the seventh embodiment, in addition to the first embodiment, when changing from the enlargement process in the mobile terminal device 104 to the enlargement process in the server device 101, a message is sent to the user. Output. When the enlargement process is performed in the server apparatus 101, there is a possibility that the power consumption and the operation speed decrease due to communication may occur. Therefore, by intentionally notifying the user, the convenience is improved.

FIG. 15 is a flowchart showing the flow of image enlargement processing in the seventh embodiment to which the present invention is applied.
In FIG. 15, when the enlargement operation of the image displayed on the LCD 210 is performed on the mobile terminal device 104, first, in step S1300, the CPU 201 of the mobile terminal device 104 detects the enlargement operation. In step 1301, the CPU 201 determines whether or not the enlargement ratio of the enlargement operation is an enlargement of a predetermined magnification (here, twice) or more.

  If it is determined that an enlargement ratio of less than 2 times is requested (step 1301: No), the mobile terminal device 104 executes enlargement processing in step 1303. On the other hand, when it is determined that an enlargement ratio of 2 times or more is requested (step 1301: Yes), in step 1302, it is determined whether the enlargement ratio at the previous enlargement operation is less than double. That is, it is determined whether or not the change has been made so as to cross the enlargement ratio of 2 times this time (whether or not the previous time is less than 2 times and this time is 2 times or more).

  If it is determined that the enlargement rate of the previous enlargement process is also twice or more (step 1302: No), the server apparatus 101 executes the enlargement process in step 1306. On the other hand, if it is determined that the previous time is less than twice and the current time is more than twice (step 1302: Yes), in step 1304, for example, “Can the enlargement process be continued?” A message is displayed to confirm whether or not the enlargement process can proceed.

  Next, in step 1305, the server apparatus 101 determines whether or not the enlargement process can be performed. If it is determined that there is an instruction to proceed (step 1305: Yes), step 1306 is performed. The server apparatus 101 executes the enlargement process. On the other hand, if it is determined that there is no instruction to proceed (No in step 1305), neither the server apparatus 101 nor the mobile terminal apparatus 104 executes the enlargement process.

Next, an eighth embodiment to which the present invention is applied will be described.
FIG. 16 is a flowchart showing a flow of image enlargement processing in the eighth embodiment to which the present invention is applied.

The eighth embodiment described with reference to FIG. 16 is a modification of the seventh embodiment described with reference to FIG.
In the eighth embodiment shown in FIG. 16, the difference from the seventh embodiment shown in FIG. 15 is that the server apparatus 101 is allowed to proceed with the enlargement process based on the determination in step 1305. If it is determined that there is not (step 1305: No), neither the server device 101 nor the mobile terminal device 104 executes the enlargement process, that is, instead of stopping the enlargement process, the server device is determined by the determination in step 1405. If it is determined in 101 that there is no instruction that the enlargement process may proceed (step 1405: No), the enlargement process is executed in the mobile terminal device 104 in step 1403. The other steps are as follows: This is the same as FIG.

Next, a ninth embodiment to which the present invention is applied will be described.
FIG. 17 is a flowchart showing the flow of image enlargement processing in the ninth embodiment to which the present invention is applied.

The ninth embodiment described with reference to FIG. 17 is also a modification of the seventh embodiment described with reference to FIG.
In the ninth embodiment shown in FIG. 17, the difference from the seventh embodiment shown in FIG. 15 is that the enlargement rate of the previous enlargement process is less than twice and this time is more than twice. If it is determined (step 1302: Yes), a message indicating whether or not the enlargement process may be advanced is output to the user in step 1304, and whether or not the enlargement process may be advanced in the server apparatus 101 in step 1303. While the enlargement process is executed by the server apparatus 101 or the enlargement process is stopped by determining the instruction of, the enlargement rate of the previous enlargement process is less than twice, and this time is twice. If it is determined that this is the case (step 1302: Yes), in step 1504, some message is output to the user for a certain period, for example, 0.5 seconds, and then step 15 5, it is determined whether or not the enlargement operation is continued. If it is determined that the enlargement operation is continued (step 1505: Yes), the server apparatus 101 executes the enlargement process in step 1306, If it is determined that there is no instruction to proceed (step 1505: No), neither the server device 101 nor the mobile terminal device 104 executes the enlargement process, and the other steps are the same as in FIG. is there.

Next, a tenth embodiment to which the present invention is applied will be described.
FIG. 18 is a flowchart showing the flow of image enlargement processing in the tenth embodiment to which the present invention is applied.

  The tenth embodiment described with reference to FIG. 18 is a modification in which the first embodiment described with reference to FIG. 6 and the third embodiment described with reference to FIG. 10 are combined. . That is, whether to execute the enlargement process in the server apparatus 101 or the mobile terminal apparatus 104 is determined based on both the difference in enlargement ratio and the image recognition.

  Specifically, when an enlargement operation of an image displayed on the LCD 210 is performed on the mobile terminal device 104, first, in step S400, the CPU 201 of the mobile terminal device 104 detects the enlargement operation. In step 1601, image recognition is performed, and in step 1602, a determination is made by image processing as to whether the spatial frequency of the recognized image is high or low, that is, whether the image is fine or coarse. Here, the determination based on the image processing is determination as to whether or not the server apparatus 101 is to execute vector processing because vector processing is necessary when a certain amount of high frequency is included as described above. .

  If it is determined in step 1602 that the spatial frequency is high (the image is fine) (step 1602: high), in step 1604, the CPU 201 determines that the enlargement ratio of the enlargement operation is greater than or equal to a predetermined magnification (here, twice). It is determined whether or not the image is enlarged. If it is determined that an enlargement ratio of 2 times or more has been requested (step 1604: Yes), the process proceeds to step 402, the enlargement process is executed by the server apparatus 101, and an enlargement ratio of less than 2 times is requested. If it is determined (step 1604: No), the process proceeds to step 405 and the server apparatus 101 is not accessed.

  On the other hand, if it is determined in step 1602 that the spatial frequency is low (the image is rough) (step 1602: low), in step 1603, the CPU 201 determines that the enlargement ratio of the enlargement operation is larger than the predetermined magnification ( It is determined whether or not the enlargement is three times or more. If it is determined that an enlargement ratio of 3 times or more has been requested (step 1603: Yes), the process proceeds to step 402, the enlargement process is executed by the server apparatus 101, and an enlargement ratio of less than 3 times is requested. If it is determined (step 1603: No), the process proceeds to step 405 and the server apparatus 101 is not accessed.

Steps 402 to 406 and 410 to 412 are the same as those in the first embodiment described with reference to FIG.
The embodiments of the present invention have been described above with reference to the drawings.

In the present embodiment, a value corresponding to the enlargement magnification 2 is used as a predetermined value. However, the value corresponding to an arbitrary magnification is not limited to the enlargement magnification 2.
The embodiment of the present invention described above can be realized by firmware or software on a hardware, a DSP (Digital Signal Processor) board or a CPU board as one function of the server apparatus or the client apparatus.

  The embodiments of the present invention have been described above with reference to the drawings. However, the server device or the client device to which the present invention is applied is limited to the above-described embodiments as long as the function is executed. Needless to say, a single device, a system composed of a plurality of devices, an integrated device, or a system that performs processing via a network such as a LAN or WAN may be used. .

  It can also be realized by a system including a CPU, a ROM or RAM memory connected to a bus, an input device, an output device, an external recording device, a medium driving device, and a network connection device. That is, a ROM or RAM memory, an external recording device, or a portable recording medium in which a software program for realizing the system of the above-described embodiment is recorded is supplied to the server device or client device, and the server device or client is supplied. Needless to say, this can also be achieved by the computer of the apparatus reading and executing the program.

  In this case, the program itself read from the portable recording medium or the like realizes the novel function of the present invention, and the portable recording medium or the like on which the program is recorded constitutes the present invention.

  Examples of portable recording media for supplying the program include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, DVD-RAMs, magnetic tapes, and nonvolatile memory cards. Various recording media recorded via a network connection device (in other words, a communication line) such as a ROM card, electronic mail or personal computer communication can be used.

  The computer (information processing apparatus) executes the program read out on the memory, thereby realizing the functions of the above-described embodiment, and an OS running on the computer based on the instructions of the program. Performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

  Furthermore, a program read from a portable recording medium or a program (data) provided by a program (data) provider is stored in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. After being written, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are also realized by the processing. obtain.

  That is, the present invention is not limited to the embodiment described above, and can take various configurations or shapes without departing from the gist of the present invention.

It is a figure which shows the outline of the network structure of server-based computing. It is a figure which shows an example of the breakdown of the power consumption of a mobile terminal device. It is a figure which shows the outline of a structure of the mobile terminal device to which this invention is applied. It is a figure which shows the internal structure of LCDC. It is a figure which shows the outline of a structure of a server apparatus. It is a flowchart which shows the flow of the image expansion process in 1st Embodiment to which this invention is applied. It is a figure which shows the state of the frame buffer and display part of a mobile terminal device when a screen expansion process is performed in the server apparatus. It is a figure which shows the state of the frame buffer of a mobile terminal device when a screen expansion process is performed with a mobile terminal device, and a display part. It is a flowchart which shows the flow of the image expansion process in 2nd Embodiment to which this invention is applied. It is a flowchart which shows the flow of the image expansion process in 3rd Embodiment to which this invention is applied. It is a figure which shows the external appearance of a mobile terminal device. It is a flowchart which shows the flow of the image expansion process in 4th Embodiment to which this invention is applied. It is a flowchart which shows the flow of the image expansion process in 5th Embodiment to which this invention is applied. It is a flowchart which shows the flow of the image expansion process in 6th Embodiment to which this invention is applied. It is a flowchart which shows the flow of the image expansion process in 7th Embodiment to which this invention is applied. It is a flowchart which shows the flow of the image expansion process in 8th Embodiment to which this invention is applied. It is a flowchart which shows the flow of the image expansion process in 9th Embodiment to which this invention is applied. It is a flowchart which shows the flow of the image expansion process in 10th Embodiment to which this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Server apparatus 102 Client terminal apparatus 103 Wireless access point 104 Mobile terminal apparatus 105 Smartphone 109 Network 200 LSI
201 CPU
202 LCD controller (LCDC)
203 Internal memory (RAM)
204 Memory Controller (Mem. Cont.)
204a Internal bus (Bus)
204b Internal bus 205 Real-time counter (RTC)
206 Asynchronous serial communication (UART)
207 Power Management Block (Power Mgmt)
208 USB function (USBF)
209 Wireless communication unit (WiFi)
210 LCD
212 Program storage memory (FROM)
213 Work memory (SDRAM)
301 Bus interface (BUS I / F)
302 Register (Reg.)
303 Memory Controller (Memory Cont.)
303a DMAC (Direct Memory Access Controller)
305 Resizer
306 FIFO
307 LCD I / F
500 frame buffer (mobile terminal side)
501 Display area 502 Enlarged area 503 Area 701 CPU
702 Memory 703 Input unit 704 Display unit 705 Storage unit 706 Recording medium drive unit 707 Network connection unit 700 Bus 901 Pointer 901a Left click key 901b Right click key 902a Up cursor key 902b Right cursor key 902c Down cursor key 902d Left cursor key 904 Keyboard 905 Shift key 910 Slide bar

Claims (8)

A client device that makes a service request via a network to a server device that operates an application program,
Display means for displaying an image;
A frame memory area for temporarily storing the image for displaying the image on the display means;
An enlargement instruction input means for inputting an instruction for enlarging and displaying the image displayed on the display means;
Determining means for determining whether the enlargement instruction input by the enlargement instruction input means is a predetermined value or more;
Server enlargement image requesting means for requesting the server device to transmit an enlarged server enlarged image corresponding to the enlargement instruction when the determining means determines that the enlargement instruction is greater than or equal to a predetermined value. When,
Server enlarged image receiving means for receiving the server enlarged image transmitted from the server device;
Memory control means for storing the server enlarged image received by the server enlarged image receiving means in the frame memory area;
Client image enlargement means for obtaining a client enlarged image corresponding to the enlargement instruction by using the image stored in the frame memory area when the determination means determines that the enlargement instruction is not greater than a predetermined value; ,
Display control means for controlling to display the server enlarged image stored in the frame memory area or the client enlarged image enlarged by the client image enlarging means on the display means;
A client device comprising:   The client apparatus according to claim 1, wherein the client image enlargement unit enlarges the image by bitmap interpolation using a linear interpolation method or a bicubic method.   The client apparatus according to claim 1, wherein the enlargement instruction indicates a magnification for enlarging the image. A server enlarged image frame memory area for temporarily storing the server enlarged image for displaying the server enlarged image on the display means;
Further comprising
The memory control means stores the server enlarged image received by the server enlarged image receiving means in the server enlarged image frame memory area, and the display control means stores the server enlarged image frame memory area. Controlling to display a server enlarged image on the display means;
The client device according to claim 1, wherein the client device is a device. The server enlarged image requesting unit requests the server device to transmit a server enlarged image enlarged to a magnification corresponding to the enlargement instruction when the enlargement instruction key provided in the enlargement instruction input unit is weakly pressed. The client image enlarging means calculates a client enlarged image obtained by enlarging the image stored in the frame memory area at a magnification corresponding to the enlargement instruction when the enlargement instruction key provided in the enlargement instruction input means is strongly pressed. To
The client device according to any one of claims 1 to 4 any one, characterized in that. The server enlarged image requesting means transmits a server enlarged image enlarged to a magnification corresponding to the enlargement instruction to the server device when an enlargement instruction key provided in the enlargement instruction input means is continuously pressed late. The client image enlargement means enlarges the image stored in the frame memory area at a magnification corresponding to the enlargement instruction when the enlargement instruction key provided in the enlargement instruction input means is pressed quickly and continuously. The enlarged client enlarged image,
The client device according to any one of claims 1 to 4 any one, characterized in that. The server enlarged image request means requests the server device to transmit a server enlarged image enlarged to a magnification corresponding to the instruction value when the enlargement instruction slide bar provided in the enlargement instruction input means is operated slowly. the client image enlarging means, the enlargement instruction when the input means enlargement instruction slide bar is provided which is operated quickly, the client enlarged image of the image stored in the frame memory area enlarged at a magnification corresponding to the enlargement instruction To calculate,
The client device according to any one of claims 1 to 4, characterized in that. A client device computer that makes a service request to a server device that runs an application program via a network.
Display means for displaying images,
An enlargement instruction input means for inputting an instruction for enlarging and displaying the image displayed on the display means;
Determining means for determining whether or not an enlargement instruction input by the enlargement instruction input means is a predetermined value or more;
Server enlargement image requesting means for requesting the server device to transmit an enlarged server enlarged image corresponding to the enlargement instruction when the determining means determines that the enlargement instruction is greater than or equal to a predetermined value. ,
Server enlarged image receiving means for receiving the server enlarged image transmitted from the server device;
Memory control means for storing the server enlarged image received by the server enlarged image receiving means in the frame memory area;
Client image enlargement means for obtaining a client enlarged image corresponding to the enlargement instruction from the image stored in the frame memory area when the determination means determines that the enlargement instruction is not equal to or greater than a predetermined value;
Display control means for controlling to display the server enlarged image stored in the frame memory area or the client enlarged image enlarged by the client image enlarging means on the display means;
Program to function as.