JP5463726B2 - Image generating apparatus, server apparatus, image generating method, and program - Google Patents

Description

The present invention relates to an image generation device, a server device, an image generation method, and a program, and more particularly, to an image generation device, a server device, an image generation method, and a program that are suitable when the display screen size is small.

  In business documents, more efficient information transmission is achieved by using graphs and the like effectively. Conventionally, such business documents have been created and viewed mainly by desktop or notebook personal computers.

  Such business documents are often distributed and used as electronic data. In such a case, convenience is improved by converting graphs or character strings provided as image data into reconfigurable numerical data or relationships between data (for example, patent documents). 1).

  On the other hand, the performance of smaller terminal devices than general personal computers such as mobile communication terminals such as mobile phones and smartphones, PDAs (Personal Digital Assistants), netbooks, etc. Such a small device can handle business documents in the same manner as a personal computer.

JP-A-9-134404

  Although the information processing capability of the small terminal device as described above is improved, the size of the display screen is inevitably smaller than that of a personal computer or the like due to physical size restrictions. In such a case, even if the image data is easily converted and processed, there remains a problem with respect to the visibility of the display screen. That is, when a graph or the like is displayed in a size that is easy to visually recognize, the size of the entire image becomes larger than the area of the display screen, and the entire image may not be visually recognized unless a scroll operation or the like is performed. In addition, the image size may be reduced in accordance with the display screen. In this case, however, there is a problem that characters attached to the graph are difficult to read.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image generation device, a server device, an image generation method, and a program capable of realizing good image generation in accordance with a display environment. And

In order to achieve the above object, an image generation apparatus according to a first aspect of the present invention includes:
An image generation device for generating a display image including at least a raster image,
Character extraction means for recognizing characters included in the raster image and extracting a character image indicating the recognized characters;
Character coordinate calculation means for calculating coordinates on the raster image of the character image extracted by the character extraction means;
Image deformation means for deforming the raster image in a non-uniform aspect ratio in accordance with the screen size of the screen for displaying the display image;
Based on such aspect unequal multiples of deformation rate deformation of the raster image by the image deforming unit, the character size correcting means for correcting the display size of the character that said character extracting means has extracted the vertical and horizontal magnification,
On the basis of the vertical and horizontal non magnification of deformation rate of the raster image, so that the position relation of the character and the raster image is maintained, and the character coordinate correcting means for correcting the coordinates of the character coordinate calculation means has calculated,
Image combining means for combining at least the character image of the character size corrected by the character size correcting means with the coordinates corrected by the character coordinate correcting means on the raster image deformed by the image deforming means;
Output control means for outputting a display image synthesized by the image synthesis means;
It is characterized by providing.
In the image generating apparatus,
The character size correcting means corrects the display size of the character to the same size in the vertical and horizontal directions based on the lower one of the vertical and horizontal deformation rates,
The character coordinate correction means corrects the coordinates based on the deformation rate with the higher deformation rate in either the vertical or horizontal direction,
It is characterized by that.

The image generation apparatus includes:
The character extraction means preferably further comprises character conversion means for converting characters extracted from the raster image into text data.
The image synthesizing unit may synthesize the text data converted by the character converting unit with the deformed raster image.

In the image generating apparatus,
The image deformation means deforms at least the raster image excluding the character extracted by the character extraction means.

The image generation apparatus includes:
A line segment extracting means for recognizing a line segment included in the raster image and extracting a line segment image indicating the recognized line segment;
Line segment coordinate calculating means for calculating coordinates on the raster image of the line segment image extracted by the line segment extracting means;
A line segment size correcting unit that corrects a display size of the line segment extracted by the line segment extracting unit based on a deformation rate applied to the deformation of the raster image by the image deforming unit;
It is desirable that the apparatus further comprises line segment coordinate correction means for correcting the coordinates calculated by the line segment coordinate calculation means based on the deformation rate of the raster image.
The image synthesizing unit synthesizes a line segment image corrected by the line segment size correcting unit with the coordinates corrected by the line segment coordinate correcting unit, on the raster image deformed by the image deforming unit. Is desirable.

In the image generating apparatus,
The image deformation means deforms at least the raster image excluding the line segment extracted by the line segment extraction means.

The image generation apparatus includes:
Preferably, the line segment extraction unit further includes a line segment conversion unit that converts a line segment image extracted from the raster image into a vector image.
The image synthesizing unit preferably synthesizes the vector image converted by the line segment converting unit with the deformed raster image.

In the image generating apparatus,
The raster image deformed by the image deforming means is preferably a graph, and in this case,
The character extracted by the character extraction means may be a character string attached to the graph.
In the image generating apparatus,
The output control unit outputs the display image synthesized by the image synthesis unit to a display unit included in the image generation apparatus.
In the image generating apparatus,
The output control means outputs to an external device having a display unit for displaying a display image synthesized by the image synthesizing means.

In order to achieve the above object, a server device according to the second aspect of the present invention provides:
A server device of a thin client system that performs processing based on an input operation from a thin client device and transfers a display screen generated by the processing to the thin client device,
Character extraction means for recognizing a character included in a raster image generated by the processing and extracting a character image indicating the recognized character;
Character coordinate calculation means for calculating coordinates on the raster image of the character image extracted by the character extraction means;
Image deformation means for deforming the raster image in a non-equal size in accordance with the screen size of the thin client device;
Based on such aspect unequal multiples of deformation rate deformation of the raster image by the image deforming unit, the character size correcting means for correcting the display size of the character that said character extracting means has extracted the vertical and horizontal magnification,
On the basis of the vertical and horizontal non magnification of deformation rate of the raster image, so that the position relation of the character and the raster image is maintained, and the character coordinate correcting means for correcting the coordinates of the character coordinate calculation means has calculated,
Image composition for generating the display screen by synthesizing at least the character image of the character size corrected by the character size correction unit with the coordinates corrected by the character coordinate correction unit to the raster image deformed by the image deformation unit. Means,
It is characterized by providing.

In order to achieve the above object, an image generation method according to a third aspect of the present invention includes:
An image generation method for generating a display image including at least a raster image,
Recognizing characters included in the raster image and extracting a character image indicating the recognized characters;
Calculating coordinates on the raster image of the extracted character image;
Transforming the raster image into a non-equal aspect ratio in accordance with the screen size of the screen for displaying the display image;
A step of based on said take aspect unequal multiples of deformation rate deformation of the raster image, corrects the display size of the character that the extracted vertically and horizontally magnification,
On the basis of the vertical and horizontal non magnification of deformation rate of the raster image, so that the position relation of the character and the raster image is maintained, and correcting the coordinates of the character images the calculated,
Combining at least the corrected character size character image with the corrected coordinates on the deformed raster image;
Outputting the combined display image;
It is characterized by including.
In order to achieve the above object, a program according to the fourth aspect of the present invention is:
A computer for controlling a device for generating a display image including at least a raster image;
A function of recognizing characters included in the raster image and extracting a character image indicating the recognized characters;
A function of calculating coordinates on the raster image of the extracted character image;
A function of deforming the raster image in a non-equal size in accordance with the screen size of the screen for displaying the display image;
A function based on the deformation rate of the vertical and horizontal non-magnification according to a modification of the raster image, corrects the display size of the character that the extracted vertically and horizontally magnification,
Based on the vertical and horizontal non magnification of deformation index of the raster image, so that the position relation of the character and the raster image is maintained, a function for correcting the coordinates of the character images the calculated,
A function of combining at least the corrected character size character image with the corrected coordinates on the deformed raster image;
A function of outputting the synthesized display image;
It is characterized by realizing.

  According to the present invention, it is possible to display an image with high visibility even when the display screen size is narrow.

It is a block diagram which shows the hardware constitutions of the mobile communication terminal concerning Embodiment 1 of this invention. It is a functional block diagram which shows the function implement | achieved by the control part shown in FIG. It is a flowchart for demonstrating the "display optimization process" concerning Embodiment 1 of this invention. FIG. 4 is a flowchart for explaining “resizing processing (1)” executed in “display optimization processing” shown in FIG. 3; FIG. FIG. 4 is a flowchart for explaining a “resizing process (2)” executed in the “display optimization process” shown in FIG. 3. FIG. It is a figure which shows the example of a display of the "context menu" displayed by the "display optimization process" shown in FIG. FIG. 4 is a diagram illustrating an example of an image processed in the “display optimization process” illustrated in FIG. 3, (a) illustrates an example of an entire graph image, and (b) illustrates a main part graph constituting the entire graph image. An example of an image is shown, (c) shows an example of a line segment image constituting the whole graph image, and (d) shows an example of a character image constituting the whole graph image. FIG. 5 is a diagram for explaining processing by “resizing processing (1)” shown in FIG. 4, where (a) shows an example of a positional relationship between a graph image before resizing processing and a display screen, and (b) shows The example of the positional relationship of the graph image in the middle of a process and a display screen is shown, (c) shows the example of a display after a resizing process. FIG. 6 is a diagram for explaining processing by “resizing processing (2)” shown in FIG. 5, where (a) shows an example of the positional relationship between the graph image before the resizing processing and the display screen, and (b) shows An example of the positional relationship between the graph image after the normal reduction and the display screen is shown. (C) shows an example of the positional relationship between the graph image after the non-equal reduction and the display screen. The example of a display after a process is shown. It is a figure which shows the structure of the thin client system concerning Embodiment 2 of this invention. It is a block diagram which shows the hardware constitutions of the server shown in FIG. It is a functional block diagram which shows the function implement | achieved by the control part shown in FIG.

(Embodiment 1)
Embodiments according to the present invention will be described below with reference to the drawings. In this embodiment, the case where the image generation apparatus concerning this invention is implement | achieved by the mobile communication terminal is illustrated. That is, it is assumed that the mobile communication terminal 100 according to the present embodiment can display and output an image including at least a raster image in addition to the call / communication function that is a basic function. In the present embodiment, it is assumed that an image including a graph used in a business document or the like is displayed on the mobile communication terminal 100.

  A configuration of the mobile communication terminal 100 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a hardware configuration of mobile communication terminal 100. As illustrated, the mobile communication terminal 100 includes a control unit 110, a communication unit 120, a voice processing unit 130, a storage unit 140, an operation unit 150, an image memory 160, a display unit 170, and the like.

  The control unit 110 includes, for example, a CPU (Central Processing Unit), a RAM (Random Access Memory) serving as a work area, and the like, and each unit of the mobile communication terminal 100 by executing a predetermined operation program. To control. That is, each component of the mobile communication terminal 100 is controlled by the control unit 110, and information transmission between the components is performed via the control unit 110.

  The communication unit 120 has a configuration related to wireless access when the mobile communication terminal 100 performs communication. For example, a code division multiple access (CDMA) method, a global systems for mobile communications (GSM) method, and the like. Wireless communication is performed with an antenna 121 corresponding to the communication method, and wireless communication is performed with a nearby base station.

  The audio processing unit 130 is constituted by, for example, a codec circuit for audio data, and performs processing related to audio input / output of the mobile communication terminal 100. That is, the reception operation of converting the digital audio data received by the communication unit 120 into an analog audio signal and outputting it from the speaker 131, or the user's utterance input from the microphone 132 is converted into digital audio data to the communication unit 120. Perform the transmission operation to send out.

  The storage unit 140 includes a storage device such as a flash memory, for example, and stores various data necessary for realizing the present invention in addition to storing an operation program executed by the control unit 110.

  The operation unit 150 includes, for example, buttons and keys configured on the outer surface of the housing, and is operated by the user of the mobile communication terminal 100. The operation unit 150 includes an input circuit connected to each button, key, and the like, generates an input signal corresponding to a user operation, and inputs the input signal to the control unit 110.

  The image memory 160 is composed of, for example, a semiconductor storage device such as a RAM, and expands image data of an image to be displayed.

  The display unit 170 is a display output device configured by, for example, a liquid crystal display device, and displays and outputs various screens configured by images such as raster images and vector images, texts, and the like under the control of the control unit 110. .

  When the display unit 170 has a touch panel function, the display unit 170 also functions as the operation unit 150. In the present embodiment, it is assumed that the display unit 170 has a touch panel function, and the configuration of the operation unit 150 is also mounted on the display unit 170.

  The above is the main configuration of the mobile communication terminal 100 according to the present embodiment, but these are the configurations necessary for realizing the present invention, and for realizing the main functions and additional functions as the mobile communication terminal. It is assumed that other configurations necessary for the above are appropriately provided.

  As described above, in the present embodiment, it is assumed that a business document including a graph or the like is displayed on the mobile communication terminal 100. Here, when the graph is a raster image, the entire graph may not be displayed with a screen size that can be implemented as the display unit 170 of the mobile communication terminal 100. In this embodiment, in such a case, display control is performed so that the entire graph fits within the display screen and the legibility of the characters attached to the graph is maintained.

  In order to perform such an operation, in the present embodiment, the control unit 110 executes the operation program stored in the storage unit 140, thereby configuring a function as illustrated in FIG. That is, the control unit 110 functions as an operation determination unit 111, an image recognition unit 112, an image extraction unit 113, an image size correction unit 114, a coordinate correction unit 115, an image composition unit 116, a display control unit 117, and the like.

  The operation discriminating unit 111 discriminates the operation content desired by the user of the mobile communication terminal 100 based on the input signal from the operation unit 150, and gives an instruction necessary for processing according to the operation content.

  When the operation content determined by the operation determination unit 111 instructs to resize the display image so that it fits within the display screen, the image recognition unit 112 determines the type, size, aspect ratio, and the like of the target image. Recognize Here, it is assumed that these recognition operations are performed by using a known image recognition technique, and the operation determination unit 111 includes a graph in the displayed raster image based on such a recognition operation. Determine whether or not.

  Further, when the resizing target is a raster image indicating a graph, the image recognition unit 112 recognizes an element constituting the graph (hereinafter referred to as “graph element”) and its coordinates. Here, for example, line segments indicating axis lines and scale lines constituting the graph and characters (labels, etc.) attached to the graph are recognized, and their coordinates are recognized. The coordinates in this case are assumed to be coordinates indicating positions in the entire raster image.

  The image extraction unit 113 performs image extraction in units of elements recognized by the image recognition unit 112. Here, an image (line segment image) indicating a line segment constituting the graph and an image (character image) indicating a character attached to the graph are extracted from the target raster image. It is assumed that a known image extraction technique is used for image extraction.

  The image extraction unit 113 converts the extracted character image into text data by using a known character recognition technique. Furthermore, the extracted line segment image is converted from a raster image to a vector image by using a known image conversion technique.

  The image size correction unit 114 corrects (resizes) the raster image showing the graph and the size of the extracted line segment image and character image in accordance with the screen size of the display unit 170. Here, for example, based on the aspect ratio of the graph image and the aspect ratio of the screen, to fit the entire graph within the screen size, it is determined whether the aspect of the graph image should be adjusted to the screen size. The size of the graph image is corrected by compressing / decompressing the graph image with the deformation rate when the screen size is adjusted in the direction.

  The image size correcting unit 114 also performs size correction on characters attached to the graph and line segments constituting the graph. Since characters are converted into text data by the image extraction unit 113, the character size is corrected by specifying the font size of the text data so as to be an appropriate character size obtained from the screen size and the deformation rate of the graph image. To do. For the line segment, the line segment size is corrected by specifying the thickness of the vector image so as to have an appropriate thickness obtained from the screen size and the deformation rate of the graph image.

  Based on the coordinates of the graph elements in the raster image before resizing recognized by the image recognition unit 112 and the deformation rate at the time of resizing, the coordinate correction unit 115 is configured to display the graph elements (characters and lines) to be arranged on the graph image after resizing. The coordinates of each graph element are corrected by calculating the (min) coordinates.

  The image composition unit 116 synthesizes the character image converted into the text data and the line segment converted into the vector image into the graph image (raster image) resized by the image size correction unit 114, so that the screen of the display unit 170 is displayed. Generate a graph image that fits the size. In this case, the image composition unit 116 performs composition so that characters and line segments are arranged at the coordinates corrected by the coordinate correction unit 115.

  The display control unit 117 sends an image signal indicating the image combined by the image combining unit 116 to the display unit 170 and displays the image based on the image signal in accordance with the screen size of the display unit 170. To control.

  Processing executed by the functional configuration shown in FIG. 2 will be described. Here, “display optimization processing” executed when screen display is performed on the mobile communication terminal 100 having the above-described configuration will be described with reference to the flowchart shown in FIG. This “display optimization process” is started when a display operation is performed on the display unit 170 (for example, when the mobile communication terminal 100 is powered on, returned from the sleep mode, etc.).

  After the processing is started by turning on the mobile communication terminal 100 or the like, the user of the mobile communication terminal 100 operates the operation unit 150 to call a menu screen for selecting an operation for the displayed image. Is performed, a signal corresponding to this operation is input to the control unit 110. When determining that the menu call operation has been performed based on the input signal (step S11: Yes), the operation determination unit 111 instructs the image recognition unit 112 to perform a recognition operation on the currently displayed image. To do. The displayed image to be resized may be designated by the user by operating the operation unit 150 or may be recognized by a known recognition technique.

  The image recognition unit 112 performs an operation of recognizing the currently displayed image and determines whether the image can be resized (step S12). In the present embodiment, an image showing a graph whose full size display size is larger than the screen size of the display unit 170 is an image that can be resized. For this reason, the image recognition unit 112 recognizes the screen size of the display unit 170 based on hardware specification information managed by an OS or the like that controls the operation of the mobile communication terminal 100, and performs known image recognition. Using technology, it is determined whether the image includes a graph.

  When the display image is an image that can be resized (step S12: Yes), the image recognition unit 112 notifies the operation determination unit 111 to that effect. Based on the notification from the image recognition unit 112, the operation determination unit 111 generates a context menu screen including an operation item for an image that can be resized, and displays it on the display unit 170 through a control operation by the display control unit 117. Displayed (step S13).

  Here, for example, a context menu as shown in FIG. In the present embodiment, as shown in the figure, for example, options such as “fit graph in drawing (float)”, “fit screen in drawing (compression)”, “return to original display” are selected from the context menu. Displayed as possible. Here, “fit the graph in the drawing (float)” and “fit the screen in the drawing (compression)” include resizing processing included when it is determined in step S12 that the image can be resized. This is an option for designating, and in the present embodiment, it indicates that there are “float” and “compression” options for resizing processing.

  The user of the mobile communication terminal 100 operates the operation unit 150 (in this embodiment, the touch panel function of the display unit 170 corresponds) to select a desired operation from the context menu. When a signal corresponding to the selection operation is input to the control unit 110, the operation determination unit 111 determines which of the options constituting the context menu has been selected.

  The operation determination unit 111 selects an item (that is, “fit a graph in a drawing (float)” or “fit a screen in a drawing (compression)”) that the user's operation designates the resizing process. In the case (step S14: Yes), the image extraction unit 113, the image size correction unit 114, and the coordinate correction unit 115 determine whether to perform the resizing process and which option is designated ("float" or "compression"). The image composition unit 116 and the display control unit 117 are notified to instruct execution of the resizing process.

  In response to an instruction from the operation determination unit 111, the image extraction unit 113 decomposes the currently displayed image (that is, a raster image showing a graph) into predetermined partial units (step S15). This image decomposition will be described with reference to FIG.

  The image including the target graph is, for example, a bar graph image as shown in FIG. Such an image before the resizing process is referred to as an “overall graph image”. This “whole graph image” is assumed to be a raster image.

  With respect to the graph image in this example, in the present embodiment, the graph main body, the background portion, and the like as shown in FIG. This “main part graph image” is assumed to be a raster image.

  Then, the rest of the entire graph image excluding the main part of the graph includes a portion indicating a line segment such as a vertical axis, a horizontal axis, and a scale line as shown in FIG. 7C, and FIG. 7D. As shown, there is a portion indicating characters such as label characters attached to each axis. In the present embodiment, an image showing a line segment (FIG. 7C) is a “line segment image”, and an image showing a character (FIG. 7D) is a “character image”. Both the line image and the character image are original raster images, but the line image can be converted into a vector image, and the character image can be converted into text data.

  The image extraction unit 113 extracts the main part graph image, the line segment image, and the character image as shown in FIGS. 7B to 7D from the entire graph image shown in FIG. By extracting each, the graph image (overall graph image) that is the object of resizing processing is decomposed. That is, the image (overall graph image) developed in the image memory 160 is decomposed and extracted into the above-described elements by using a known image recognition technique or region extraction technique, and the image of each element is extracted from the image memory. Expand to 160.

  When the element image is extracted from the entire graph image in this way, the image extraction unit 113 notifies the image size correction unit 114 to that effect. The image size correction unit 114 uses each element image extracted by the image extraction unit 113 so that the content of the original graph image (overall graph image) is appropriately visually recognized according to the screen size of the display unit 170. The resizing process is performed.

  As described above, in this embodiment, the “float” and “compression” options are prepared for the resizing process. If the option notified from the operation determination unit 111 is “Float” (Step S16: Yes), the image size correction unit 114 executes “Resize processing (1)” (Step S100). If this is the case (step S16: No), “resize processing (2)” is executed (step S200). Each of these processes will be described below with reference to FIGS.

  First, the “resizing process (1)” will be described with reference to the flowchart shown in FIG. When the processing is started, the image size correction unit 114 compares the aspect ratio of the entire graph image with the aspect ratio of the display screen, and determines whether the direction of the graph image to be adjusted to the screen size is the vertical axis direction. Determine if it is axial. For example, when a horizontally long graph image is displayed on a horizontally long screen as shown in FIG. 8A, if the horizontal direction of the graph image, that is, the horizontal axis direction is resized according to the horizontal size of the screen, the image will fit. Can be displayed.

  When a rectangular graph image is displayed on such a rectangular display screen, the graph axis parallel to the direction on the graph image to be aligned with the display screen (in the example of FIG. 8A, the horizontal axis) is “ The reference axis ”is a graph axis parallel to the direction on the graph image that does not match the display screen (in the example of FIG. 8A, the vertical axis) is the“ non-reference axis ”. The image size correction unit 114 determines such a reference axis and a non-reference axis from the aspect ratio of the display screen and the aspect ratio of the target image (step S101).

  When the reference axis and the non-reference axis are determined, the image size correction unit 114 specifies an image range obtained by removing characters arranged in the non-reference axis direction from the entire graph image (step S102). In the case of the entire graph image shown in FIG. 8A, the characters (“0”, “500” in FIG. 8A) of the label (vertical axis label) attached to the scale of the vertical axis that is the non-reference axis. ”,“ 1000 ”,“ 1500 ”,“ 2000 ”).

  When the image range excluding the characters in the non-reference direction is specified, the image size correction unit 114 displays the entire graph image so that the size of the image range in the reference axis direction fits (fits) the horizontal size of the display screen. The “magnification A” used for the calculation for resizing (deforming) is calculated (step S103).

  “Magnification A” is “a magnification that can be obtained at a lower compression rate in either the vertical or horizontal direction, obtained from a comparison between the aspect ratio of the main graph image and the aspect ratio of the display screen size”.

  When the magnification A is calculated, the image size correction unit 114 extracts each element image extracted in step S15 of the “display optimization process” (FIG. 3) and expanded in the image memory 160, that is, expanded in the image memory 160. Each of the main graph image (FIG. 7B), the line segment image (FIG. 7C), and the character image (FIG. 7D) is reduced at the same magnification by the magnification A (step S104).

  FIG. 8B shows an example of the positional relationship with the display screen when the element images reduced at the same magnification in this manner are temporarily combined. As described above, the magnification A is calculated so that the image range excluding the characters of the non-reference axis is within the horizontal size of the display screen. When displayed, as shown in FIG. 8B, the main part of the graph fits on the display screen in the horizontal direction. In this case, as shown in FIG. 8B, the main part of the graph and the characters of the non-reference axis do not fit on the display screen in the vertical direction.

  However, since the horizontal direction of the main part of the graph excluding the character part fits the horizontal size of the display screen, the reduction rate of the graph image is the lowest. Therefore, although the image is reduced in accordance with the display screen, good visibility can be maintained. In this case, scrolling is necessary for the vertical display, but unidirectional scrolling is sufficient, and operability is not significantly impaired.

  When the same size reduction with the magnification A is performed, the image size correction unit 114 notifies the image extraction unit 113 and the coordinate correction unit 115 to that effect. In response to the notification from the image size correction unit 114, the image extraction unit 113 converts the line segment image after the equal magnification reduction into a vector image (step S105). As a result, line segment images such as each axis and scale line of the graph are converted into vector images that are easier to deform and process. Here, since each element image that has been decomposed in advance is reduced (size correction), vector image conversion of the line segment image can be easily performed. Then, the image size correction unit 114 designates the size of the line segment such as the thickness and length numerically so that the line segment is the same as the line image reduced by the magnification A.

  Here, as shown in FIG. 8B, the characters of the non-reference axis do not fit in the display screen as they are. Therefore, the coordinate correction unit 115 corrects the coordinates of the characters on the non-reference axis with the magnification A in order to match the graph image that has been reduced to the same magnification with the magnification A (step S106).

  The image extracting unit 113 converts the character of the non-reference axis whose coordinates have been corrected in this way into text data by recognizing the character from the character image developed in the image memory 160 (step S107). In this case, the image size correction unit 114 designates the character size of the converted text data so as to be the same size as that of the character shown in the character image reduced by the equal magnification A.

  The image synthesizing unit 116 synthesizes the non-reference axis character converted into text data in this way into a graph image so as to be float-displayed at the coordinates corrected in step S106 (step S108). The display control unit 117 displays and outputs the image synthesized by the image synthesis unit 116 on the display unit 170 (step S109), and returns to the flow of “display optimization processing” (FIG. 3).

  Here, an example of the float display is shown in FIG. As shown in the figure, for characters that are not displayed when fitting the display screen with emphasis on the visibility of the graph image, information necessary for interpreting the graph can be obtained by using a float display that can be moved on the display screen. Can be visually recognized. In this case, since the character portion is converted to text data, the character size can be easily changed, and even when the character size is relatively small, character collapse that tends to occur when the same character size is used in the raster image, etc. Absent. Note that the float display portion can be moved in the left-right direction by an arbitrary operation by the user. Moreover, you may enable it to visually recognize the graph image of a back surface by drawing a float display part transparently.

  Next, the “resizing process (2)” executed when the option of the resizing process is “compression” will be described with reference to the flowchart shown in FIG.

  When the processing is started, the image size correction unit 114 compares the aspect ratio of the entire graph image with the aspect ratio of the display screen, and determines whether the direction of the graph image to be adjusted to the screen size is the vertical axis direction. Determine if it is axial. For example, when a horizontally long graph image is displayed on a horizontally long screen as shown in FIG. 9A, the horizontal direction of the graph image, that is, the horizontal axis direction is resized according to the horizontal size of the screen, so that the image fits well. Can be displayed.

  When displaying a rectangular graph image on such a rectangular display screen, the graph axis parallel to the direction on the graph image to be aligned with the display screen (in the example of FIG. 9A, the horizontal axis) is “ The reference axis is a graph axis parallel to the direction on the graph image that does not match the display screen (in the example of FIG. 9A, the vertical axis) is the non-reference axis. The image size correction unit 114 determines such a reference axis and a non-reference axis from the aspect ratio of the display screen and the aspect ratio of the target image (step S201).

  When the reference axis and the non-reference axis are determined, the image size correction unit 114 uses “magnification B” used for the operation for resizing (deforming) the entire graph image so that the entire graph image fits within the display screen. And “magnification C” are calculated (step S202).

  Here, the “magnification B” is “a magnification that can be obtained at a lower compression rate in either the vertical or horizontal direction, obtained from a comparison between the aspect ratio of the entire graph image and the aspect ratio of the display screen size”.

  “Magnification C” is “([total graph image height] − [non-reference axis character image height]) × magnification B ÷ [display screen height]” or “([total graph image Width] − [width of character image of non-reference axis]) × magnification B ÷ [width of display screen] ”.

  When the magnification B and the magnification C are calculated, the image size correction unit 114 reduces the entire graph image by the magnification B (step S203). Here, each element image developed in the image memory 160, that is, a main part graph image (FIG. 7B), a line segment image (FIG. 7C), and a character image (FIG. 7D) are displayed. Each is reduced by an equal magnification of B.

  FIG. 9B shows an example of the positional relationship with the display screen when the element images reduced at the same magnification in this manner are temporarily combined. As described above, the magnification B is calculated so that the entire graph image including the character portion fits in the horizontal size of the display screen. Therefore, when the entire graph image is reduced by the magnification B, the characters on the non-reference axis are displayed. The entire graph image including it fits on the display screen in the horizontal direction. In this case, as shown in FIG. 9B, the vertical direction of the entire graph image does not fit on the display screen.

  The image size correction unit 114 reduces the main part graph image and the line segment image at the magnification C only in the non-reference axis direction among the element images already reduced at the magnification B so that the entire graph fits on the display screen. (Step S204). In other words, the main part graph image and the line segment image are reduced to an unequal magnification.

  The positional relationship between the main part graph image and the line segment image that have been reduced to the same magnification using the magnification B and the magnification C, and the display screen when the character image that has been reduced to the same magnification at the magnification B is temporarily combined. An example is shown in FIG. Here, since the main part image and the line segment image are reduced only in the non-reference direction, the entire graph image excluding the character portion fits in the display screen as shown in the figure. Further, since the image is reduced using the magnification C, the image is reduced so as to leave a space for displaying the characters of the reference axis.

  When such non-magnification reduction is performed, the image size correction unit 114 notifies the image extraction unit 113 and the coordinate correction unit 115 to that effect. In response to the notification from the image size correction unit 114, the image extraction unit 113 converts the line segment image after non-equal reduction into a vector image that is easier to deform and process (step S205). Here, since reduction (size correction) is performed on each of the main part graph image and the line segment image that have been decomposed in advance, the vector image conversion of the line segment image can be easily performed. In this case, the image size correcting unit 114 designates the size of the line segment such as the thickness and the length by a numerical value, so that the same line as the line image that has been reduced at an unmagnified size using the magnification B and the magnification C is used. Try to be minutes.

  Further, the image extraction unit 113 performs character recognition from the character image developed in the image memory 160, thereby converting the non-reference axis character and the reference axis character into text data (step S206). In this case, the image size correction unit 114 designates the character size of the converted text data so as to be the same size as the size of the character shown in the character image reduced at the same magnification by the magnification B.

  When the character image is converted into text data by the image extraction unit 113, the coordinate correction unit 115 corrects the coordinates of each character. Here, as shown in FIG. 9C, since the coordinates in the non-reference axis direction (vertical direction) do not match the main part of the graph, the coordinates in the non-reference axis direction are corrected using the magnification C (step S207). ).

  When the coordinates of the character image are corrected, the coordinate correction unit 115 notifies the image composition unit 116 of the fact. The image synthesizing unit 116 synthesizes the non-equal scaled main part graph image developed in the image memory 160 and the line segment image converted into the vector image, and converts each character converted into the text data into a step. The image is synthesized with the coordinates corrected in S207 (step S208), and the display control unit 117 displays and outputs the synthesized image on the display unit 170 (step S209), and the “display optimization process” (FIG. 3) is performed. Return to flow.

  FIG. 9D shows a display example when the resize processing option is “compression”. As shown in the figure, the entire graph image is displayed so as to fit (fit) on the display screen. In this case, since the characters are reduced at the same magnification, even if the main part of the graph is reduced at the same magnification, the readability can be maintained without being deformed. Furthermore, since the data is converted to text data, the character size can be easily changed, and character collapse that is likely to occur when the image is reduced as a raster image does not occur.

  As described above, the display of the graph image is optimized according to the display screen size by the “resizing process (1)” or “resizing process (2)”. In this case, since the user can select the “float display” and “compression” options for the resizing process, it is possible to select an optimization method with better visibility according to the original size of the graph image, usage, or the like. it can.

  After completion of “resizing process (1)” or “resizing process (2)”, in “display optimization process” (FIG. 3), a predetermined end event (for example, powering off mobile communication terminal 100 or going to sleep mode). Each of the above-described processes is performed until an event that causes a stop of the display operation (such as a transition of) occurs (step S17: No). In this case, for example, the menu calling operation by the user is not performed (step S11: No), the displayed image is not resizable (step S12: No), and the user does not select the resizing process (step S14). : No), etc., “Resize processing (1)” and “Resize processing (2)” are not executed.

  Then, when the end event occurs, this process ends (step S17: Yes).

  As described above, according to this embodiment, a graph is displayed on a narrow display screen, such as when an image is displayed on a small device such as a mobile communication terminal, or when a small sub-display is used on a personal computer or the like. When displaying an image or the like, the visibility of the display image can be improved. Here, when the character portion is float-displayed, the reduction rate of the graph image can be minimized, which is effective when it is desired to check the graph in detail. On the other hand, when the entire graph image is displayed within the display screen, the entire graph can be checked in a list.

(Embodiment 2)
In the above embodiment, an example in which the present invention is applied to a case where a device having a display unit such as the mobile communication terminal 100 displays a graph image has been described. However, a device and a display device that perform processing on a display image The present invention can also be applied to cases where and are different devices.

  In this embodiment, an example in which the present invention is applied to a so-called screen transfer type thin client system will be described. In this case, a thin client system 200 as shown in FIG. 10 is assumed. As illustrated, the thin client system 200 according to the present embodiment includes a server 300 and a plurality of thin clients 400 connected to the server 300 via a communication network NW.

  In such a thin client system 200, an application or the like desired by the user of the thin client 400 is not executed by the thin client 400 but is executed by the server 300. The server 300 transfers only the screen generated by executing the application or the like to the thin client 400 via the communication network NW.

  As a result, the thin client 400 can obtain the same processing result as when an application or the like is executed on the thin client 400. In this case, the application to be executed and the data of the processing result are stored in the server 300 and are not stored in the thin client 400. Therefore, in the case where information that should not be leaked is handled by a mobile terminal, it is used as a system that emphasizes security.

  Therefore, the thin client 400 according to the present embodiment is assumed to be a mobile terminal, such as a mobile communication terminal or a portable personal computer. In addition, like a general thin client dedicated terminal, a configuration that does not include a storage device such as a hard disk device can be used, and a configuration in which a storage operation for processing as a thin client using the server 300 is limited. If so, a storage device may be included.

  That is, the thin client 400 can use various devices. For this reason, the display screen size of each thin client 400 is not uniform. The thin client 400 only functions as an input device for inputting an instruction to the server 300 and a display device for displaying a processing result screen on the server 300. Therefore, the server 300 generates a screen to be displayed on the thin client 400. For this reason, the processing according to the present invention is executed by the server 300.

  The configuration of such a server 300 will be described with reference to FIG. FIG. 11 is a block diagram illustrating a hardware configuration of the server 300 according to the present embodiment. As illustrated, the server 300 includes a control unit 310, a communication control unit 320, an input control unit 330, an input device 331, an output control unit 340, an output device 341, a storage unit 350, an image memory 360, and the like. .

  The control unit 310 includes, for example, a CPU and a RAM serving as a work area, and controls each unit of the server 300 by executing a predetermined operation program. That is, each component of the server 300 is controlled by the control unit 310, and information transmission between the components is performed via the control unit 310.

  The communication control unit 320 is configured by, for example, a NIC (Network Interface Card) or the like, and connects the server 300 and a device (for example, a router or a modem) that connects the server 300 and the communication network NW. Perform operations related to communication via the Internet.

  The input control unit 330 receives an input signal from the input device 331 connected to the server 300 and inputs the input signal to the control unit 310. The input device 331 is, for example, a keyboard or a pointing device such as a mouse.

  The output control unit 340 outputs an output signal from the control unit 310 to the output device 341 connected to the server 300. The output device 341 is, for example, a display device or a printer.

  The storage unit 350 includes a storage device such as a hard disk device, for example, and stores a program executed by the control unit 310, data used for processing, data indicating a processing result, and the like. In the present embodiment, since the server constitutes the thin client system 200, an application program executed in response to a request from the thin client 400 is also stored. In addition, data necessary for authentication of the thin client 400 is also stored in the storage unit 350.

  The image memory 360 is composed of, for example, a relatively high-speed semiconductor storage device such as a RAM, and develops image data indicating a processing result screen of an application executed by the control unit 310.

  Next, a functional configuration realized by the control unit 310 executing the program will be described with reference to FIG. FIG. 12 is a functional block diagram illustrating a functional configuration realized by the control unit 310. As illustrated, the control unit 310 functions as an operation determination unit 311, an image recognition unit 312, an image extraction unit 313, an image size correction unit 314, a coordinate correction unit 315, an image composition unit 316, a display control unit 317, and the like. . Each of these functions is the operation determination unit 111, the image recognition unit 112, the image extraction unit 113, the image size correction unit 114, the coordinate correction unit 115, the image composition unit 116, the display control unit 117, and the like exemplified in the first embodiment. Is the same function.

  However, in this embodiment, since the thin client 400 displays a screen of processing performed in the server 300 in response to an input from the thin client 400, an input signal to the operation determination unit 311 is obtained from the communication control unit 320. A control signal from the display control unit 317 is output to the communication control unit 320.

  The server 300 having such a configuration includes the “display optimization process” (FIG. 3), “resizing process (1)” (FIG. 4), “resizing process (2)” (FIG. 5) exemplified in the first embodiment. By executing the same processing, when the thin client 400 displays a graph image, the thin client 400 is optimized according to the display screen size. In this case, since the display screen size of each thin client 400 is notified to the server 300 when functioning as a thin client system, the server 300 responds to the display screen size of the thin client 400 that transfers the screen. Each process described above can be performed so that display optimization is performed.

  As described above, according to the present embodiment, when a device with a narrow display screen is used as a thin client, a graph image or the like can be displayed with good visibility according to the display screen size.

  The said embodiment is an example and the application range of this invention is not restricted to this. That is, various applications are possible, and all embodiments are included in the scope of the present invention.

  In each of the above embodiments, the line segment image is converted into the vector image. However, the display optimization may be performed with the raster image as it is. Similarly, display optimization may be performed as a raster image without converting a character image into text data. Thereby, the processing load concerning conversion can be reduced.

  Moreover, although the said embodiment demonstrated the optimization concerning the display of a graph image, if it is an image containing a character, this invention is applicable also to images other than a graph image. Moreover, the graph shown by the said embodiment is an example, and the kind of graph is arbitrary. In this case, the type of the graph may be determined by an image recognition technique or the like, and whether or not the resizing process may be performed and the reference axis / non-reference axis may be determined based on the type of the graph.

Further, when the present invention is realized by a terminal device such as the mobile communication terminal 100 illustrated in the first embodiment or a server device such as the server 300 illustrated in the second embodiment, the configuration and functions according to the present invention. Can be provided as a terminal device or a server device provided in advance, and an existing terminal device or server device according to the present invention can be applied by applying a program that realizes the same function as each function of the control unit 110 or the control unit 310. It can also function as an image generation device or a server device.

  The application method of such a program is arbitrary. For example, the program can be applied by being stored in a storage medium such as a CD-ROM or a memory card, or can be applied via a communication medium such as the Internet.

DESCRIPTION OF SYMBOLS 100 ... Mobile communication terminal, 110 ... Control part, 111 ... Operation discrimination | determination part, 112 ... Image recognition part, 113 ... Image extraction part, 114 ... Image size correction part, 115 ... Coordinate correction part, 116 ... Image composition part, 117 DESCRIPTION OF SYMBOLS ... Display control part, 120 ... Communication part, 121 ... Antenna, 130 ... Audio | voice processing part, 131 ... Speaker, 132 ... Microphone, 140 ... Memory | storage part, 150 ... Operation part, 160 ... Image memory, 170 ... Display part, 200 ... Thin client system, 300 ... server, 310 ... control unit, 311 ... operation discrimination unit, 312 ... image recognition unit, 313 ... image extraction unit, 314 ... image size correction unit, 315 ... coordinate correction unit, 316 ... image composition unit, 317 ... Display control unit, 320 ... Communication control unit, 330 ... Input control unit, 331 ... Input device, 340 ... Output control unit, 341 ... Output device, 350 ...憶部, 360 ... image memory, NW ... communication network, 400 ... thin client

Claims (13)

An image generation device for generating a display image including at least a raster image,
Character extraction means for recognizing characters included in the raster image and extracting a character image indicating the recognized characters;
Character coordinate calculation means for calculating coordinates on the raster image of the character image extracted by the character extraction means;
Image deformation means for deforming the raster image in a non-uniform aspect ratio in accordance with the screen size of the screen for displaying the display image;
Based on such aspect unequal multiples of deformation rate deformation of the raster image by the image deforming unit, the character size correcting means for correcting the display size of the character that said character extracting means has extracted the vertical and horizontal magnification,
On the basis of the vertical and horizontal non magnification of deformation rate of the raster image, so that the position relation of the character and the raster image is maintained, and the character coordinate correcting means for correcting the coordinates of the character coordinate calculation means has calculated,
Image combining means for combining at least the character image of the character size corrected by the character size correcting means with the coordinates corrected by the character coordinate correcting means on the raster image deformed by the image deforming means;
Output control means for outputting a display image synthesized by the image synthesis means;
An image generation apparatus comprising: The character size correcting means corrects the display size of the character to the same size in the vertical and horizontal directions based on the lower one of the vertical and horizontal deformation rates,
The character coordinate correction means corrects the coordinates based on the deformation rate with the higher deformation rate in either the vertical or horizontal direction,
The image generating apparatus according to claim 1. The character extraction means further comprises character conversion means for converting characters extracted from the raster image into text data,
The image synthesizing unit synthesizes the text data converted by the character converting unit with the deformed raster image;
The image generation apparatus according to claim 1, wherein the image generation apparatus is an image generation apparatus. The image deformation means deforms at least the raster image excluding the character extracted by the character extraction means;
The image generation apparatus according to claim 1, wherein the image generation apparatus is an image generation apparatus. A line segment extracting means for recognizing a line segment included in the raster image and extracting a line segment image indicating the recognized line segment;
Line segment coordinate calculating means for calculating coordinates on the raster image of the line segment image extracted by the line segment extracting means;
A line segment size correcting unit that corrects a display size of the line segment extracted by the line segment extracting unit based on a deformation rate applied to the deformation of the raster image by the image deforming unit;
Line segment coordinate correction means for correcting the coordinates calculated by the line segment coordinate calculation means based on the deformation rate of the raster image,
The image synthesizing unit synthesizes the line segment image corrected by the line segment size correcting unit with the coordinates corrected by the line segment coordinate correcting unit on the raster image deformed by the image deforming unit.
The image generation apparatus according to claim 1, wherein the image generation apparatus is an image generation apparatus. The image deformation means deforms at least the raster image excluding the line segment extracted by the line segment extraction means;
The image generating apparatus according to claim 5. The line segment extraction means further comprises line segment conversion means for converting a line segment image extracted from the raster image into a vector image,
The image synthesizing unit synthesizes the vector image converted by the line segment converting unit with the deformed raster image;
The image generation apparatus according to claim 5, wherein the image generation apparatus is an image generation apparatus. The raster image deformed by the image deformation means shows a graph,
The character extracted by the character extraction means is a character string attached to the graph.
The image generation apparatus according to claim 1, wherein the image generation apparatus is an image generation apparatus. The output control means outputs the display image synthesized by the image synthesis means to a display unit included in the image generation device.
The image generation apparatus according to claim 1, wherein the image generation apparatus is an image generation apparatus. The output control means outputs to an external device having a display unit for displaying the display image synthesized by the image synthesis means.
The image generation apparatus according to claim 1, wherein the image generation apparatus is an image generation apparatus. A server device of a thin client system that performs processing based on an input operation from a thin client device and transfers a display screen generated by the processing to the thin client device,
Character extraction means for recognizing a character included in a raster image generated by the processing and extracting a character image indicating the recognized character;
Character coordinate calculation means for calculating coordinates on the raster image of the character image extracted by the character extraction means;
Image deformation means for deforming the raster image in a non-equal size in accordance with the screen size of the thin client device;
Based on such aspect unequal multiples of deformation rate deformation of the raster image by the image deforming unit, the character size correcting means for correcting the display size of the character that said character extracting means has extracted the vertical and horizontal magnification,
On the basis of the vertical and horizontal non magnification of deformation rate of the raster image, so that the position relation of the character and the raster image is maintained, and the character coordinate correcting means for correcting the coordinates of the character coordinate calculation means has calculated,
Image composition for generating the display screen by synthesizing at least the character image of the character size corrected by the character size correction unit with the coordinates corrected by the character coordinate correction unit to the raster image deformed by the image deformation unit. Means,
A server device comprising: An image generation method for generating a display image including at least a raster image,
Recognizing characters included in the raster image and extracting a character image indicating the recognized characters;
Calculating coordinates on the raster image of the extracted character image;
Transforming the raster image into a non-equal aspect ratio in accordance with the screen size of the screen for displaying the display image;
A step of based on said take aspect unequal multiples of deformation rate deformation of the raster image, corrects the display size of the character that the extracted vertically and horizontally magnification,
On the basis of the vertical and horizontal non magnification of deformation rate of the raster image, so that the position relation of the character and the raster image is maintained, and correcting the coordinates of the character images the calculated,
Combining at least the corrected character size character image with the corrected coordinates on the deformed raster image;
Outputting the combined display image;
An image generation method comprising: A computer for controlling a device for generating a display image including at least a raster image;
A function of recognizing characters included in the raster image and extracting a character image indicating the recognized characters;
A function of calculating coordinates on the raster image of the extracted character image;
A function of deforming the raster image in a non-equal size in accordance with the screen size of the screen for displaying the display image;
A function based on the deformation rate of the vertical and horizontal non-magnification according to a modification of the raster image, corrects the display size of the character that the extracted vertically and horizontally magnification,
Based on the vertical and horizontal non magnification of deformation index of the raster image, so that the position relation of the character and the raster image is maintained, a function for correcting the coordinates of the character images the calculated,
A function of combining at least the corrected character size character image with the corrected coordinates on the deformed raster image;
A function of outputting the synthesized display image;
A program characterized by realizing.

Images