JP5156616B2 - Image display program, terminal device, image display method, and image display system - Google Patents

Description

  The present invention relates to a technical field of an image display program, a terminal device, an image display method, and an image display system that download and display image data through a network or the like.

  In recent years, portable terminals such as portable wireless telephones have become widespread, and users can easily view moving images such as Web pages and broadcast programs with a portable terminal even when away from home.

When browsing these, since the display screen of the mobile terminal is smaller than the screen of a personal computer or TV, the mobile terminal displays image data etc. compressed based on the size of the display screen on the display screen. Yes. For example, Patent Document 1 discloses the following portable terminal technology. When the received original image data is larger than the number of pixels of the display unit, the mobile terminal displays the original image data by reducing the number of pixels to the number of pixels of the display unit. Then, when a position is designated on the image by the pointing device, corresponding coordinates in the original image data are calculated, and based on the corresponding coordinates, an image area equal to the number of pixels of the display unit is designated in the original image data. Is transmitted to the image data distribution apparatus. Thus, when image data corresponding to the image area is received, the image data is displayed on the display unit as a partially enlarged image without changing the number of pixels of the image data.
JP 2003-91410 A

  By the way, with the spread of broadband Internet connection such as optical fiber, high-resolution images such as still images of several megapixels to gigapixels such as detailed map data and moving images such as high-definition television broadcasts are released on the web, It can be browsed by a personal computer or the like.

  However, in the case of a portable terminal such as a portable radio telephone, there are many functional restrictions such as communication speed and storage capacity in addition to the size of the display screen. It was difficult to smoothly view high-resolution images, and it was difficult to say that the user's operability was sufficiently improved. Even when the user browses on a personal computer, for example, if the communication status is bad and the response is even slow, the user cannot wait for a response to the request, and makes successive requests such as clicking the screen from one to the next. Tend to. In this case, the load due to the communication overhead is increased, and the response to the user's operation is excessively slow. Therefore, it cannot be said that the user's operability is sufficiently improved.

  The present invention has been made in view of such problems, and an example of the problem is to provide an image display program, a terminal device, an image display method, and an image display system that improve user operability. For the purpose.

  In order to solve the above-mentioned problem, the invention described in claim 1 is directed to a computer, original image information receiving means for receiving original image information relating to the original image, and first and first of the original image based on the original image information. Area selection means for accepting selection of a second partial area, and a third part for calculating a third partial area including the first and second partial areas based on attribute information of the first and second partial areas Based on attribute information of at least one of the attribute information of the area calculation means, the first and second partial areas, and the attribute information of the third partial area, the partial area image data relating to the second partial area or Reception selection means for selecting which partial area image data related to the third partial area is received, image data reception means for receiving the selected partial area image data, and the received part Characterized in that to function as a display means for displaying an area image data.

  According to the present invention, when the partial area image data related to the third partial area is received, the image data can be received in a form in which the partial areas scheduled to be received are merged. The time can be shortened, the response to the user's operation is improved, and the operability of the user can be improved.

  According to a second aspect of the present invention, in the image display program according to the first aspect, the attribute information of the partial area in the reception selection means is the size of the partial area. In this case, the reception time can be easily estimated based on the size of the partial area.

  According to a third aspect of the present invention, in the image display program according to the second aspect, the attribute information of the partial area in the reception selection means includes the size of the first and second partial areas, and the third It is a value obtained by comparing the sizes of the partial areas.

  In this case, partial area image data relating to the merged partial areas is received by a quantitative index such as how much the first partial area and the second partial area are included in the three partial areas. It is possible to judge more accurately whether or not it is faster.

  According to a fourth aspect of the present invention, in the image display program according to the third aspect, of the plurality of selected partial areas, the partial area where the compared value is an extreme value with respect to the second partial area Is the first partial region.

  In this case, the first partial area becomes an optimal fusion candidate with respect to the second partial area requested by the latest user, and image data can be received more efficiently.

  According to a fifth aspect of the present invention, in the image display program according to the first aspect, the attribute information of the partial area in the reception selection means is an inter-area distance between the first partial area and the second partial area. It is.

  In this case, the fusion efficiency can be easily calculated from the distance between the regions.

  According to a sixth aspect of the present invention, in the image display program according to any one of the first to fifth aspects of the present invention, the reception selection means is arranged in the second partial area based on the communication state of the reception. Whether to receive the image data or the partial area image data related to the third partial area is selected.

  In this case, by making a determination based on the communication state of the mobile communication network or the like, it is possible to efficiently receive the image data according to the communication state, and to improve the user operability.

  According to a seventh aspect of the present invention, in the image display program according to any one of the first to sixth aspects, the shape of the third partial region is a rectangle, and the side of the rectangle is the The minimum circumscribed rectangle is in contact with the first partial region and the second partial region, and minimizes the size of the third partial region.

  In this case, the third partial region can be calculated efficiently.

  The invention according to claim 8 is an original image information receiving means for receiving original image information relating to the original image, and an area selection for receiving selection of the first and second partial areas of the original image based on the original image information. Means, a third partial area calculating means for calculating a third partial area including the first and second partial areas based on attribute information of the first and second partial areas, and the first and second partial areas Based on the attribute information of the second partial area and the attribute information of at least one of the attribute information of the third partial area, the partial area image data related to the second partial area or the third partial area Receiving selection means for selecting which partial area image data is to be received; image data receiving means for receiving the selected partial area image data; and display means for displaying the received partial area image data. And said that there were pictures.

  According to the present invention, when the partial area image data related to the third partial area is received, the image data can be received in a form in which the partial areas scheduled to be received are merged. It is possible to provide a mobile terminal that can shorten the time and improve the operability of the user by improving the response to the user's operation.

  The invention according to claim 9 is an original image information receiving step for receiving original image information relating to the original image, and an area selection for receiving selection of the first and second partial areas of the original image based on the original image information. A third partial region calculating step of calculating a third partial region including the first and second partial regions based on attribute information of the first and second partial regions; Based on the attribute information of the second partial area and the attribute information of at least one of the attribute information of the third partial area, the image data related to the second partial area or the image data related to the third partial area A receiving selection step for selecting which of the image data to be received, an image data receiving step for receiving the selected image data, and a display step for displaying the received image data And wherein the door.

  According to the present invention, when the partial area image data related to the third partial area is received, the image data can be received in a form in which the partial areas scheduled to be received are merged. It is possible to provide an image display method that can shorten the time and improve the user operability by improving the response to the user operation.

Invention according to claim 10, and a terminal device for displaying an image data in the image display system and a server device that distributes image data in response to a request of the terminal device, the terminal device, the original image Original image information receiving means for receiving the original image information, area selection means for accepting selection of the first and second partial areas of the original image based on the original image information, and the first and second parts Based on attribute information of the region, third partial region calculation means for calculating a third partial region including the first and second partial regions, attribute information of the first and second partial regions, and Based on at least one attribute information of the attribute information of the third partial area, either the partial area image data related to the second partial area or the partial area image data related to the third partial area is received. Comprising a reception selection means for selecting either the image data receiving means for receiving the partial region image data selected, a display means for displaying the partial region image data received, and the server device, the terminal device Request receiving means for receiving a request from the terminal , original image data receiving means for receiving original image data relating to the original image in response to a request from the terminal device , and an original image for creating original image information relating to the original image an information creating means, the original and the image information original image information transmitting means for transmitting to the terminal device, the terminal device partial area image data generating means for generating the partial-area image data from the original image data in response to a request And image data transmitting means for transmitting the partial area image data to the terminal device .

  According to the present invention, the first and second partial areas of the original image are selected based on the original image information relating to the original image, and the first and second partial areas are selected based on the attribute information of the first and second partial areas. 3rd partial area including 2 partial areas is calculated, and based on attribute information of at least one of attribute information of the 1st and 2nd partial areas and attribute information of the 3rd partial area, the 2nd part The partial area image data related to the third partial area is received by selecting whether to receive the partial area image data related to the area or the partial area image data related to the third partial area and receiving the image data. In this case, image data can be received in a form in which partial areas to be received are fused, so the number of receptions can be reduced to shorten the overall reception time, the response to user operations is improved, and user operability is improved. The figure It is possible.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  First, a schematic configuration and function of an image display system according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic diagram showing a schematic configuration example of an image display system according to the first embodiment of the present invention.

  As shown in FIG. 1, the image display system 1 includes a mobile terminal 10 that displays an image, and an image server 20 that distributes image data in response to a request from the mobile terminal 10.

  The mobile terminal 10 is connected to the mobile communication network 2, the image server 20 is connected to a network 3 such as the Internet, and the mobile communication network 2 and the network 3 perform protocol conversion, language conversion for describing contents, and the like. Connected by a gateway or the like for performing. The image server 20 is connected to a Web server 30 that stores data such as HTML documents and images via a network 3 such as the Internet. Then, the mobile terminal 10 transmits to the image server 20 request information such as an image to be browsed, that is, a URL (Uniform Resource Locator) of the original image. The image server 20 receives original image data relating to the original image from the Web server 30 corresponding to the URL, and creates original image information relating to the original image from the original image data, or uses the original image data for the portable terminal 10. The original image information and the image data are transmitted to the mobile terminal 10. For example, the URL of the image server 20 as the access destination is incorporated in the software on the mobile terminal 10 side, and the URL of the Web server 30 as the original image data destination can be directly input on the mobile terminal 10 side, When a location where the URL of the Web server 30 is embedded is clicked, request information is transmitted from the portable terminal 10 to the image server 20. Then, after acquiring the original image data from the Web server 30, the image server 20 can provide the original image data that has been subjected to predetermined processing to the mobile terminal 10. Thus, the mobile terminal 10 acquires information from the Web server 30 or the like via the image server 20 based on the installed software.

  Next, the configuration and function of the mobile terminal 10 will be described with reference to the drawings. FIG. 2 is a block diagram illustrating an example of a schematic configuration of the mobile terminal 10.

  As shown in FIG. 2, the mobile terminal 10 transmits and receives to and from the display unit 11 that displays an image, text, and the like, the operation unit 12 that is used to operate the pointer displayed on the display unit 11, the image server 20, and the like. A wireless communication unit 13 that performs voice input / output, a voice input / output unit 14 that inputs / outputs voice for a call, a storage unit 15 that stores data transmitted from the image server 20, and various controls of the mobile terminal 10. And a control unit 16, which are connected by a system bus 17.

  The display unit 11 is configured by a liquid crystal display element, an EL (Electro Luminescence) element, or the like. As shown in FIG. 3, a pointer 11 a indicating a position in the displayed image is displayed on the display unit 11. The display unit 11 functions as an example of a display unit that displays received image data and the like.

  The operation unit 12 includes various keys such as numeric keys, cross keys, and function keys. The operation unit 12 moves the pointer 11a displayed on the display unit 11, rotates the image, reduces or enlarges the image, and determines which partial area of the displayed image the detailed image is to be downloaded. Or The operation unit 12 functions as an example of an area selection unit that receives selection of the first and second partial areas of the original image based on the original image information.

  The wireless communication unit 13 functions as an example of an original image information receiving unit that receives original image information relating to an original image and an image data receiving unit that receives selected partial area image data.

  The storage unit 15 includes a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, and stores image data downloaded from the image server 20, image attribute information, image partial area attribute information, and the like. . In the storage unit 15, a memory space corresponding to an image with a certain resolution, a stack for temporarily storing information, and the like are formed.

  The control unit 16 includes a CPU (Central Processing Unit) and the like, and controls the above-described units and executes various types of information processing. Then, the control unit 16 receives the first and second partial areas based on the original image information, the area selection means for accepting selection of the first and second partial areas of the original image, and the first and second partial areas. At least one attribute of the third partial region calculation means for calculating the third partial region including the second partial region, the attribute information of the first and second partial regions, and the attribute information of the third partial region Based on the information, it functions as an example of a reception selection unit that selects which partial area image data related to the first and second partial areas or partial area image data related to the third partial area is received.

  Next, as shown in FIG. 1, the image server 20, which is an example of a server device, includes a control unit 21 that controls the entire image server 20 and a database 22 that stores image data and the like.

  The control unit 21 includes a CPU, a transmission / reception unit, and the like, and communicates with the mobile terminal 10 and the Web server 30, controls the database 22, and reduces various resolutions of image data downloaded from the Web server 30. Process. Further, the database 22 stores image data downloaded from the Web server 30, data obtained by image processing of this data, requests from the portable terminal 10, and the like. Then, the control unit 21 receives request from the mobile terminal 10, original image data receiving means for receiving original image data related to the original image in response to the request from the mobile terminal 10, and original image information. Original image information transmitting means for transmitting to the mobile terminal 10, partial area image data generating means for generating partial area image data from the original image data 25 in response to a request from the mobile terminal 10, and partial area image data to the mobile terminal 10 It functions as an example of image data transmission means for transmission.

  Next, an example of image data used in the present embodiment will be described with reference to FIGS. 4 and 5.

  FIG. 4 is a schematic diagram showing an example of original image data existing in the database of the Web server 30. FIG. 5 is a schematic diagram illustrating an example of image data with different resolutions in the database of the image server 20.

  The Web server 30 has original image data indicated by the URL of the mobile terminal 10, for example, original image data 35 as shown in FIG. 4. The original image data 35 is image data having a size of several mega to giga pixels, and cannot be displayed on the mobile terminal 10 as it is.

  As shown in FIG. 5, the original image data 25 downloaded from the Web server 30 to the image server 20 is stored in the database 22. And the control part 21 of the image server 20 produces the image data 25a, 25b, 25c, etc. of various resolutions, as shown in FIG.

  Here, as the original image information related to the original image, information on the number of pixels (resolution), the ratio between the width and the height, and the like regarding the original image data 25 and the created image data with various resolutions can be cited. Further, information related to the original image displayed on the display unit 11 of the mobile terminal 10, that is, image data 25 a of an image compressed so as to understand the entire state of the original image, a part of the image data 25 b and 25 c, and the original image A part of the data 25 is mentioned. In this way, the control unit 21 functions as an example of original image information creation means for creating original image information relating to an original image.

  Note that when the resolution is decreased to reduce the number of pixels, the original image data 25 is once divided into a plurality of regions to form divided images. Then, by parallel processing using a multi-core CPU or the like, the resolution of each of these divided images is reduced to obtain a low resolution divided image, and then these are integrated to obtain a low resolution integrated image. As described above, since the image to be compressed and reduced in resolution is first divided into divided images, and the divided images are compressed by parallel processing using a plurality of resolution reduction means, it is possible to achieve high-speed image compression. . In addition, when an image of a certain megapixel image is browsed on the mobile terminal 10 in response to a user request, an image with reduced resolution can be transmitted at high speed.

  Next, the operation of this embodiment will be described with reference to FIGS.

  FIG. 6 is a flowchart illustrating an example of the algorithm of the present embodiment. FIG. 7 is a schematic diagram illustrating an example of a region displayed on the display unit 11 and a cached region in the storage unit 15. 8 (A) to (C), FIG. 9 (D) to (F), FIG. 10 (G), and FIG. 10 (H) are examples of partial areas and stacks to be downloaded based on the flowchart of FIG. It is the shown schematic diagram. FIGS. 11A and 11B are schematic views showing an example of the fusion of partial areas by the algorithm of the present embodiment.

  First, based on a user input, the mobile terminal 10 transmits a URL where an image of a certain site exists to the image server 20. At this time, the control unit 16 of the mobile terminal 10 also transmits information related to the function of the mobile terminal 10 such as the screen size and resolution of the display unit 11 in the mobile terminal 10 to the image server 20. The control unit 21 of the image server 20 accesses the Web server 30 corresponding to the requested URL, downloads the original image data 35 as shown in FIG. 4, and stores it in the database 22 as the original image data 25. Note that the URL of the image server 20 that is the transmission destination of the request information is incorporated in an image display program executed by the control unit 16.

  When the data amount of the original image data 25 is large with respect to the display function or the like of the mobile terminal 10, the control unit 21 creates low-resolution image data of the original image data 25 as shown in FIG. For example, the control unit 21 reduces the original image data 25 to create low-resolution image data 25a in which the number of pixels is reduced so as to fit on the screen of the display unit 11, and as the original image information, the number of pixels of the original image, etc. To the portable terminal 10 together with the information.

  The control unit 16 of the portable terminal 10 receives the original image information such as the image data 25a from the image server 20, and displays the image data 25a on the display unit 11 as shown in FIG.

  Next, as illustrated in FIG. 6, the control unit 16 of the mobile terminal 10 receives an input of a candidate partial area to be downloaded from the image server 20 from the operation unit 12 (step S <b> 1). For example, the user of the mobile terminal 10 operates the operation unit 12 and moves the pointer 11a while viewing the image in which the image data 25a is displayed on the display unit 11 as shown in FIG. Select what you want to see and confirm.

  When the desired place is determined, as shown in FIG. 7, the control unit 16 calculates a partial area P in the memory space 15a corresponding to an image having a resolution higher than the resolution of the currently displayed image. The partial area P is a display area SA to be displayed on the display unit 11 and a double buffering area to be prefetched and cached, that is, an area indicated by oblique lines around the display area SA. The memory space 15a is a space on the memory that is formed inside the storage unit 15 and stores an image to be downloaded. Further, the image shown in FIG. 7 shows a high-resolution image (for example, image data 25b) to be downloaded from the image server 20 from now on.

  Here, the display area SA is rectangular, and is defined by, for example, the position u1 closest to the origin of the memory space 15a, and the width and height. The partial area P is also rectangular, and is defined by, for example, the position u2 closest to the origin of the memory space 15a, the width w2, and the height h2. The origin is the upper left in the drawing, the x axis is the right direction, and the y axis is the downward direction. Further, it is assumed that each rectangular side of the display area SA or the partial area P is parallel to the x axis or the y axis.

  The position u2 is a width W and a height H obtained from the clicked position in the image data 25a, the ratio between the resolution of the image data 25a and the resolution of the image data 25b, and the ratio of the resolution, width, and height of the image data 25b. Etc. can be calculated. As described above, the control unit 16 functions as an example of an area selection unit that receives selection of a partial area of an original image based on original image information such as resolution.

  Here, the partial area of the original image may be a partial area of the original image data 25, or may be a partial area of the low-resolution image data 25a, 25b, 25c of the original image data 25.

  Note that the resolution of the high-resolution image downloaded from the image server 20, the position u1, the position u2, the width w2, the height h2, the size of the partial area, and the like are examples of partial area attribute information. .

  Further, the resolution of a high-resolution image to be downloaded from the image server 20 is specified in advance by the image server 20, and this level may be specified, or an arbitrary resolution may be specified from the mobile terminal 10. You may request.

  Next, as shown in FIG. 8A, a case is first assumed in which the user makes a selection after selecting the partial area Pa, the partial area Pb, and the partial area Pc.

  The control unit 16 determines whether the candidate partial area to be downloaded is an insufficient area in the memory space 15a (step S2). That is, the control unit 16 determines whether or not the partial area has already been downloaded and stored in the memory space 15a. Specifically, when the partial area Pa is selected, it is determined whether or not the image data of the partial area Pa is already in the memory space 15a. If it is not an insufficient region (step S2; NO), the control unit 16 ends the process and displays the image data of the partial region on the display unit 11 as it is. On the other hand, if the area is insufficient (step S2; YES), the control unit 16 determines whether or not the stack 15b is empty (step S3). Here, the stack 15 b is an area on the memory of the stack formed inside the storage unit 15.

  If the stack 15b is empty (step S3; YES), the control unit 16 adds a part of the attribute information of the partial area to the stack 15b (step S6). Specifically, in FIG. 8A, the partial area Pa indicated by the broken line is not yet in the memory space 15a and is an insufficient area, and the stack 15b is empty. Part of the attribute information is stacked on the stack 15b. The information accumulated on the stack 15b may be information that can identify the partial area Pa and can be distinguished from other partial areas, and may be information on the coordinate point ua, for example. In FIG. 8, FIG. 9, and FIG. 10, the information stacked on the stack 15b is the subscript of the partial area P such as “a”, “b”, or “(da)” in the case of the fusion area. Etc.

  Next, the control unit 16 puts the partial area Pa in a download waiting state (step S7). And the process similar to partial area Pa is performed also about partial area | region Pb, and the control part 16 adds to the stack | stuck 15b also the information which can specify partial area | region Pb (step S1-step S6). As shown in FIG. 8A, the stack 15b is in a state in which information that can specify the partial area Pb is stacked on information that can specify the partial area Pa. However, the processing for the partial region Pb has gone through steps S4 and S5, but steps S4 and S5 will be described in detail later.

Next, as shown in FIG. 8A, when the partial area Pc is selected by the user after the partial areas Pa and Pb, the control unit 16 receives an input of the partial area Pc from the operation unit 12 ( In step S1), it is determined whether the partial area Pc is an insufficient area (step S2), and it is determined whether the stack 15b is empty (step S3). Since the stack 15b is not empty (step S3; NO), the control unit 16 sets a minimum circumscribed rectangle (MBR: Minimum Boundary) for the partial area Pb and the partial area Pa that can be specified by the information in the stack 15b. Rectangle) and the fusion efficiency η are calculated (step S4). Then, it is determined whether or not the maximum fusion efficiency η _max among the fusion efficiencies η is greater than a threshold value (step S5).

Here, the minimum circumscribed rectangle (MBR) and fusion efficiency η relating to step S4 and the maximum fusion efficiency η _max relating to step S5 will be described in detail with reference to FIG.

  FIGS. 11A and 11B are schematic views showing an example of the fusion of partial areas by the algorithm of the present embodiment.

Assume that the user selects partial areas in the order of partial area r ₁ , partial area r ₂ , partial area r ₃ , and partial area q. The stack 15b is loaded with information that can specify each partial region in the order of the partial region r ₁ , the partial region r ₂ , and the partial region r ₃ . Then, the partial region q is selected, and a circumscribed rectangle is obtained for the partial region q in the order of the partial region r ₃ , the partial region r ₂ , and the partial region r ₁ . Here, the partial region r ₁ , the partial region r ₂ , and the partial region r ₃ correspond to the first partial region, and the partial region q corresponds to the second partial region.

Next, a formula for obtaining the circumscribed rectangle of the partial regions q and subregion r _i. The subscript _i of the partial region ri is, for example, i = 1, 2, 3.

First, as shown in FIG. 11A, the range of the partial region q is defined by a point U [q] closest to the origin and a point V [q] farthest from the origin, and the partial region r _i range is defined by the most points closer to the origin U [r _i] most point furthest from the origin V [r _i]. The coordinates of the point U [q] (U [q ] x, U [q] y), the coordinates of the point V [q] (V [q ] x, V [q] y), the point U _{[r i]} the coordinates representing the _{(U [r i] x,} U [r i] y), the coordinate of the point _{V [r i] (V [} r i] x, V [r i] y). Here, the coordinates of the point U [r _i] coordinates and the point V [r _i] of, corresponds to the attribute information of the first partial region, the coordinate of the point U coordinates and point [q] V [q] , Corresponding to the attribute information of the second partial area.

  The minimum circumscribed rectangle can be obtained by the following equation.

MBR (q, r _i )
= Rect [min (U [q ] x, U [r i] x), min (U [q] y, U [r i] y),
_{max (V [q] x,} V [r i] x), max (V [q] y, V [r i] y)] ··· (1)
Here, Rect [•] is a function that represents a rectangle by a point closest to the origin and a point farthest from the origin. This minimum circumscribed rectangle MBR (q, r _i ) is an example of the third partial region. That the control unit 16 obtains the minimum circumscribed rectangle MBR (q, r _i ) by the expression (1) includes the first and second partial areas based on the attribute information of the first and second partial areas. It is an example of the 3rd partial area calculation means which calculates a 3rd partial area. Thus, the shape of the third partial region R is rectangular, and the side of the rectangle is in contact with the first partial region r _i and a second partial region q, the magnitude of the third partial region R Is the smallest bounding rectangle that minimizes

Next, the fusion efficiency η is
_{η (q, r i) =} S (q∪r i) / S (MBR (q, r i)) ··· (2)
It becomes. Here, S (•) represents the area of the region.

The fusion efficiency η is a quantitative indicator of how much the partial region q and the partial region r _i are included with respect to the minimum circumscribed rectangle MBR. For example, the fusion efficiency eta, minimum bounding rectangle MBR (q, _{r i)} the size of the size of the partial region q and the partial region _{r i} (e.g., S (q∪r _i)) closer to a value greater Any index can be used. It is also the reciprocal of the difference between _{S (MBR (q, r i} ) and S (q∪r _i). Thus, the fusion efficiency eta, in an example of the attribute information of the first and second partial regions Based on a certain S (q∪r _i ) and S (MBR (q, r _i )), which is an example of attribute information of the third partial region, and the fusion efficiency η is a part of the reception selection means. as attribute information of the region, the first and second partial regions and the size S (q∪r _i), the size _{S (MBR (q, r i} )) of the third partial area and of the value obtained by comparing the It is an example.

As described above, when each MBR and the fusion efficiency η are calculated in step S4, next, in step S5, the control unit 16 merges the partial region q with another partial region in the stack 15b. Each efficiency η _i is calculated. The control unit 16 then determines the maximum fusion efficiency η _max
= Max [η (q, r ₁ ), η (q, r ₂ ), ..., η (q, r _i ), ..., η (q, r _n )] (3)
Ask for.

In the case of FIG. 11, η _max = η (q, r ₃ ). Of the plurality of selected partial regions r ₁ , r ₂ , r ₃ , the partial region in which the fusion efficiency η is the extreme value η _max with respect to the second partial region q is the first candidate for fusion. the sub-region _{r 3.} Each fusion efficiency η _i may be _limited to a fusion efficiency larger than the threshold θ, and the maximum fusion efficiency η _max may be obtained.

Here, when r _i = r and η _max , if η _max > θ (step S5; YES), then the download candidate partial area is a partial area R = MBR (q, r). On the other hand, if η _max ≦ θ (step S5; NO), the partial area of the download candidate becomes the partial area q. For example, in the case of FIG. 11, if η _max = η (q, r ₃ ) and η (q, r ₃ )> θ, then the download candidate partial area is the partial area R
Rect [U [R] _x , U [R] _y , V [R] _x , V [R] _y ]
= MBR (q, r ₃ )
= Rect [U [q] _x , U [r ₃ ] _y , V [r ₃ ] _x , V [q] _y ]
It becomes.

Thus, the attribute information of the first partial region r _i and a second partial region q, and, based on the attribute information of the third partial region R, the partial-area image data or the according to the second partial region q 3 of the partial area image data relating to the partial area R is selected. Further, based on a value obtained by comparing the size S (q∪r _i ) of the first and second partial regions with the size S (MBR (q, r _i )) of the third partial region, Which partial area image data relating to the partial area q or partial area image data relating to the third partial area R is selected.

  The threshold θ is determined by the communication state of traffic in the mobile communication network 2 and the network 3, the function of the mobile terminal 10, and the volume of image data to be downloaded. For example, when the communication state is good and the communication traffic is not congested, the threshold value θ is set lower and the download is performed in the fusion area. In addition, the threshold value θ may be set higher as the size of the image in the fusion area increases with respect to the capacity of the storage unit 15 of the mobile terminal 10. Further, the threshold value θ may be changed according to the type of line such as a mobile phone line or a broadband line. As described above, the control unit 16 dynamically changes the threshold θ in accordance with the communication state, the volume of image data to be downloaded, and the like.

Control unit 16 Thus, as an example of a reception selection means, based on the communication state of the reception, image according to the image data or the third partial region R of the first partial region r _i and a second partial region q Select which data to receive.

As described above, the control unit 16 calculates each MBR and each fusion efficiency η in step S4, and calculates the maximum fusion efficiency η _max in step S5.

  Specifically, first, the control unit 16 extracts information that can identify the partial area Pb from the stack 15b as shown in FIG. 8A, obtains attribute information of the partial area Pb, and displays the information shown in FIG. 8B. As shown, MBR (Pc, Pb) fused with the partial region Pc and the partial region Pb and the fusion efficiency η are calculated. Then, the control unit 16 extracts information that can specify the partial area Pa from the stack 15b, obtains attribute information of the partial area Pa, and combines the MBR (Pc, Pa) that is merged with the partial area Pc and the partial area Pa, and the fusion efficiency η. And calculate. In the case of FIG. 8B, the fusion efficiency η of MBR (Pc, Pb) is about ½ and the fusion efficiency η of MBR (Pc, Pa) is about ３ according to equation (2).

When the maximum fusion efficiency η _max is obtained in step S5 from the equation (3), the control unit 16 then determines that the maximum fusion efficiency η _max is smaller than the threshold θ (step S5; NO) in the stack 15b. Information is added (step S6). Specifically, assuming that the threshold θ is 0.7, the fusion efficiency η of MBR (Pc, Pb) and MBR (Pc, Pa) is less than or equal to the threshold θ, and therefore the control unit 16 determines that FIG. As shown, information that can specify the partial area Pc is added to the stack 15b. Then, the partial area Pc is put in a state of waiting for download with the highest priority (step S7).

  Next, as shown in FIG. 8C, the control unit 16 extracts information that can identify the partial region Pc from the stack 15b, and transmits the partial region image data related to the partial region Pc to the image server via the wireless communication unit 13. Make a request. Then, the control unit 16 starts downloading the partial area image data via the wireless communication unit 13 and stores the received data in the storage unit 15.

When the maximum fusion efficiency η _max is larger than the threshold (step S5; YES), the control unit 16 sets a circumscribed rectangle obtained by merging the partial region and the partial region Pc having the maximum fusion efficiency η _max as a download candidate. A partial area is set (step S8), and the stack 15b is updated (step S9). Then, the minimum circumscribed rectangle is put in a download waiting state with the highest priority (step S7). Specific examples of these will be described with reference to the following examples of FIGS. 9D to 9F.

  Next, based on FIG. 9D to FIG. 9F, how to merge partial areas and how to download partial area image data related to the fused areas will be described.

  As shown in FIG. 9D, when the user newly selects a partial area Pd after downloading of the partial area image data related to the partial area Pc is started, the control unit 16 performs the partial area in step S1. An input of Pd is received, and in step S2, it is determined whether or not the partial area Pd is an insufficient area. Since the partial area Pd is an insufficient area, the control unit 16 determines whether or not the stack 15b is empty in step S3. Since the stack 15b is not empty, the control unit 16 extracts information that can identify the partial area Pb from the stack 15b in step S4, and merges the partial area Pd and the partial area Pb with MBR (Pd, Pb) and the fusion efficiency. η is calculated. And the control part 16 takes out the information which can specify the partial area Pa from the stack | stuck 15b, and calculates MBR (Pc, Pa) which united the partial area Pd and the partial area Pa, and fusion efficiency (eta).

In the case shown in FIG. 9E, the fusion efficiency η of MBR (Pd, Pb) is about ½, and the fusion efficiency η of MBR (Pd, Pa) is about 1. Therefore, in step S5, the fusion efficiency η of MBR (Pd, Pa) is the maximum fusion efficiency η _max . When the threshold value θ is 0.7, the fusion efficiency η of MBR (Pd, Pa) is larger than the threshold value θ, so that the control unit 16 in step S8, the minimum circumscribed rectangle MBR obtained by fusing the partial region Pd and the partial region Pa. Let (Pd, Pa) be a partial area of a download candidate. Then, in step S9, the control unit 16 updates the stack 15b as shown in FIG. 9E, and adds the minimum circumscribed rectangle MBR (Pd, Pa) that is the fusion region on the information that can specify the partial region Pd. ) Is stored.

  In step S7, the control unit 16 puts the minimum circumscribed rectangle MBR (Pd, Pa) in a state of waiting for the highest priority download.

  Next, as illustrated in FIG. 9F, when the download of the partial area image data related to the partial area Pc is completed, the control unit 16 obtains information that can specify the minimum circumscribed rectangle MBR (Pa, Pd) from the stack 15b. Then, the image server 20 is requested to request partial area image data relating to the minimum circumscribed rectangle MBR (Pa, Pd). Then, the control unit 16 starts downloading the partial area image data, and stores the received data in the storage unit 15. During downloading, since there is no data in the display area SA, the control unit 16 may display on the display unit 11 that the latest selected portion is being downloaded.

  Next, a state in which the display area SA is moved (scrolled) by the operation unit 12 after downloading will be described with reference to FIGS. 10 (G) and 10 (H).

  As shown in FIG. 10G, after the download of the partial area image data relating to the minimum circumscribed rectangle MBR (Pd, Pa) is completed and the image data of the display area SA is displayed on the display unit 11, This is a case where the user operates the operation unit 12 and the control unit 16 moves the display area SA so as to view an image above the display unit 11 (negative direction in the y-axis).

  In step S1, the control unit 16 receives an input of the partial area Pe, and determines whether or not the partial area Pe is an insufficient area in step S2. Since the partial area Pe is an insufficient area, the control unit 16 determines whether or not the stack 15b is empty in step S3. Since the stack 15b is empty as shown in FIG. 10G (step S3; YES), as shown in FIG. 10H, the control unit 16 assigns the partial area Pe to the stack 15b in step S6. Information that can be identified is loaded. Then, the partial area Pe is put in a state of waiting for download with the highest priority (step S7), and the process is terminated. When the download of the partial area image data related to the partial area Pe is completed, if there is no new request, the control unit 16 displays the image data of the part of the display area SA instructed on the display unit 11.

  Note that when the image displayed on the display unit 11 is rotated, the control unit 16 uses the image data in the double buffering area to compensate for image data that is insufficient due to the rotation. Further, the image data exceeding the capacity of the storage unit 15 is erased or overwritten from the oldest.

Thus, according to the present embodiment, the mobile terminal 10 downloads (receives) the original image information related to the original image data 35 from the image server 20, and based on the original image information, the first partial region r of the original image. select _i and a second partial region q, first and second partial regions r _i, based on the attribute information of q, the first and second partial regions r _i, a third partial region including the q R is calculated, and based on the attribute information of the first and second partial regions r _i and q and the attribute information of the third partial region R, the partial region image data or the third partial region image data relating to the second partial region q By selecting which of the partial area image data related to the partial area R is to be downloaded, downloading the selected partial area image data from the image server 20 and displaying the portion of the display area SA on the display unit 11, Third part When downloading the partial area image data related to the area R, the image data can be downloaded in a form in which the partial areas scheduled to be received are merged, so that the number of downloads can be reduced and the entire download time can be shortened. The response is improved and the operability of the user can be improved.

  In particular, when there is a time lag until the download is actually started when the control unit 16 issues a download command, it is effective to merge the partial areas to be downloaded as in this embodiment.

Further, according to this embodiment, if those who have downloaded the partial area image data of the first subregion r _i and a second partial region q separately fast, partial-area image of the third partial region R Whether the data is downloaded earlier can be accurately determined according to the attribute information of the partial area, and the image data can be downloaded efficiently.

  Further, according to the present embodiment, the display function of the mobile terminal 10 is limited, and even if an image with a large number of pixels cannot be browsed at once, the image can be browsed partially and efficiently, or the mobile terminal 10 Thus, it is possible to view the image at any time while enlarging / reducing or rotating the image, thereby improving user operability.

  If the attribute information of the partial area in the reception selection means is the size of the partial area, the download time can be easily estimated based on the size of the partial area.

If the attribute information of the partial areas r _i and q in the reception selection means is a value obtained by comparing the size of the first and second partial areas r _i and q with the size of the third partial area R, For example, the partial region image data relating to the merged partial region is expressed by a quantitative index such as how much the partial region q and the partial region r _i are included in the minimum circumscribed rectangle MBR such as the fusion efficiency η. The control unit 16 can more accurately determine whether the download is earlier.

In addition, among the plurality of selected partial areas r _i , when the partial area where the compared value is the extreme value with respect to the second partial area q is the first partial area r ₃ , the latest request from the user is requested. for the second partial region q is first subregion r ₃ are candidates for fusion to be downloaded more efficiently image data.

  When the control unit 16 selects whether to receive the image data related to the second partial region q or the partial region image data related to the third partial region R based on the received communication state, the mobile communication network 2 and network 3 based on the communication state such as traffic, the image data can be downloaded efficiently according to the communication state, and the operability of the user can be improved. For example, when the communication state is good, the threshold value θ is set low, and the user moves the display area SA by downloading the image data related to the third partial area R, which is a slightly larger fusion area. However, the number of new download requests can be reduced.

The shape of the third partial region R is rectangular, and the side of the rectangle is in contact with the first partial region r _i and a second partial region q, the magnitude of the third partial region R Min When the minimum circumscribed rectangle is, the control unit 16 can efficiently calculate the third partial region R.

  Further, in the image display system 1, the mobile terminal 10 calculates the third partial region R, and the efficiency is such that it is determined whether it is faster to download the partial region image data related to the third partial region R. Information processing is often performed to download image data, and the image server 20 performs time-consuming information processing such as image data compression, and the portable terminal 10 and the image server 20 share roles. Therefore, the image display system 1 can efficiently transmit high-resolution image data from the image server 20 to the mobile terminal 10, and can improve the operability of the user.

Incidentally, in the fusion efficiency eta, instead of S (q∪r _i) of formula (2) may be used _{S (q) + S (r} i). Furthermore, fusion efficiency η of _{(S (MBR (q, r} i) -S (q) -S (r i)) / S (MBR (q, r i) may be defined as). Also these This is a quantitative indicator of how much the partial region q and the partial region r _i are included in the minimum circumscribed rectangle MBR.

  The size of the partial region may be a combination of the area and the resolution level. For example, with respect to the partial area requested by the user, image data of an adjacent resolution level is also cached in advance to improve user operability.

Instead of the fusion efficiency η, the inter-region distance between the first partial region q and the second partial region r _i may be used as attribute information of the partial regions r _i and q in the reception selection unit. Here, region distance between the partial regions q and the partial region r _i is the distance or the like of the center between the partial regions distance and the point U [q] and the point U [r _i], first and second This is an example of attribute information of the partial area. In this case, if the inter-region distance between the partial region q and the partial region r _i is smaller than a certain reference value, the minimum circumscribed rectangle MBR (q, r _i ) is set as a download candidate. The fusion efficiency can be easily calculated based on the distance between the regions.

Further, instead of the fusion efficiency η, the size of the minimum circumscribed rectangle MBR (q, r _i ) may be used. The size such as the area of the minimum circumscribed rectangle MBR (q, r _i ) is an example of attribute information of the third partial region. In this case, if the area of MBR (q, r _i ) is smaller than a certain reference area, the minimum circumscribed rectangle MBR (q, r _i ) is set as a download candidate.

  Furthermore, a panel touch in which the display unit 11 and the operation unit 12 are integrated may be used. The area to be merged may be a partial area of image data having a resolution lower than that of the currently displayed image data, or may be a partial area having a different resolution.

  Furthermore, the present invention is not limited to the above embodiments. Each of the above embodiments is an exemplification, and any configuration that has substantially the same configuration as the technical idea described in the claims of the present invention and has the same operational effects can be used. It is included in the technical scope of the present invention.

1 is a schematic diagram illustrating a schematic configuration example of an image display system according to a first embodiment of the present invention. It is a block diagram which shows an example of schematic structure of the portable terminal of FIG. It is a schematic diagram which shows an example of the display part of FIG. It is a schematic diagram which shows an example of the original image data which exists in the database of the web server of FIG. It is a schematic diagram which shows an example of the image data from which the resolution differs in the database of the image server of FIG. It is a flowchart which shows an example of the algorithm of this embodiment. FIG. 3 is a schematic diagram illustrating an example of a region displayed on the display unit in FIG. 2 and a cached region in the storage unit in FIG. 2. (A) to (C) are schematic diagrams showing an example of a partial area to be downloaded and a state of a stack based on the flowchart of FIG. 6. (D) to (F) are schematic diagrams showing an example of a partial area to be downloaded and a state of a stack based on the flowchart of FIG. 6. (G) to (H) are schematic diagrams showing an example of a partial area to be downloaded and a state of a stack based on the flowchart of FIG. 6. (A) And (B) is a schematic diagram which shows an example of the fusion of the partial area | region by the algorithm of this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image display system 10 ... Portable terminal 11 ... Display part,
DESCRIPTION OF SYMBOLS 12 ... Operation part 13 ... Wireless communication part 15 ... Memory | storage part 16 ... Control part 20 ... Server 21 ... Control part 25, 35 ... Original image data (original image)
r _i · · · first partial region q · · · second partial region R · · · third partial region

Claims (10)

Computer
Original image information receiving means for receiving original image information relating to the original image;
Area selection means for receiving selection of first and second partial areas of the original image based on the original image information;
Third partial region calculation means for calculating a third partial region including the first and second partial regions based on the attribute information of the first and second partial regions;
Based on attribute information of at least one of the attribute information of the first and second partial regions and the attribute information of the third partial region, the partial region image data relating to the second partial region or the third partial region Receiving selection means for selecting which of the partial area image data related to the partial area is received;
Image data receiving means for receiving the selected partial area image data; and
An image display program that functions as display means for displaying the received partial area image data. The image display program according to claim 1,
The image display program characterized in that the attribute information of the partial area in the reception selection means is the size of the partial area. The image display program according to claim 2,
The image display program characterized in that the attribute information of the partial area in the reception selection means is a value obtained by comparing the size of the first and second partial areas with the size of the third partial area. In the image display program according to claim 3,
An image display program characterized in that, among the plurality of selected partial areas, a partial area where the compared value is an extreme value with respect to the second partial area is defined as a first partial area. The image display program according to claim 1,
The image display program characterized in that the attribute information of the partial area in the reception selection means is an inter-area distance between the first partial area and the second partial area. In the image display program according to any one of claims 1 to 5,
The reception selection means selects whether to receive image data related to the second partial area or partial area image data related to the third partial area based on the communication state of the reception. Image display program. The image display program according to any one of claims 1 to 6,
The shape of the third partial area is a rectangle, and a minimum circumscribing where the side of the rectangle is in contact with the first partial area and the second partial area and the size of the third partial area is minimized. An image display program characterized by being rectangular. Original image information receiving means for receiving original image information relating to the original image;
Area selection means for receiving selection of first and second partial areas of the original image based on the original image information;
Third partial region calculation means for calculating a third partial region including the first and second partial regions based on the attribute information of the first and second partial regions;
Based on attribute information of at least one of the attribute information of the first and second partial regions and the attribute information of the third partial region, the partial region image data relating to the second partial region or the third partial region Receiving selection means for selecting which of the partial area image data related to the partial area is received;
Image data receiving means for receiving the selected partial area image data;
Display means for displaying the received partial area image data;
A terminal device comprising: An original image information receiving step for receiving original image information relating to the original image;
An area selection step for accepting selection of first and second partial areas of the original image based on the original image information;
A third partial region calculating step of calculating a third partial region including the first and second partial regions based on the attribute information of the first and second partial regions;
Based on the attribute information of the first and second partial areas and the attribute information of at least one of the attribute information of the third partial area, the image data relating to the second partial area or the third partial area A reception selection step for selecting which of the image data to be received;
An image data receiving step for receiving the selected image data;
A display step for displaying the received image data;
An image display method characterized by comprising: A terminal device for displaying image data;
A server device that distributes image data in response to a request from the terminal device ;
In an image display system comprising:
The terminal device
Original image information receiving means for receiving original image information relating to the original image;
Area selection means for receiving selection of first and second partial areas of the original image based on the original image information;
Third partial region calculation means for calculating a third partial region including the first and second partial regions based on the attribute information of the first and second partial regions;
Based on attribute information of at least one of the attribute information of the first and second partial regions and the attribute information of the third partial region, the partial region image data relating to the second partial region or the third partial region Receiving selection means for selecting which of the partial area image data related to the partial area is received;
Image data receiving means for receiving the selected partial area image data;
Display means for displaying the received partial area image data;
With
The server device
Request receiving means for receiving a request from the terminal device ;
Original image data receiving means for receiving original image data relating to the original image in response to a request from the terminal device ;
Original image information creating means for creating original image information relating to the original image;
Original image information transmitting means for transmitting the original image information to the terminal device ;
Partial area image data creating means for creating the partial area image data from the original image data in response to a request from the terminal device ;
Image data transmitting means for transmitting the partial area image data to the terminal device ;
An image display system comprising:

Images