JP5186863B2 - Image aggregation device and image aggregation program - Google Patents

Description

  The present invention relates to an image aggregating apparatus that aggregates and displays a plurality of images in a stained glass shape, and more particularly to an image aggregating apparatus that improves the readability of characters in each image after aggregation.

  An image extracted as a search result by an image search engine or the like and reduced and displayed on a display device (hereinafter referred to as an extracted image) may have poor readability of characters in the extracted image. For example, in Patent Document 1, when displaying an image or a character on a small mobile device such as a PDA or a mobile phone, the image or the character is transparently superimposed in order to display a large amount of information in the display area. It is disclosed.

  Further, in Patent Document 2, a large amount of information can be displayed even in a small display area by aggregating a plurality of extracted images into a stained glass shape based on a region of interest process (ROI (Region Of Interest) process). Is disclosed.

JP 2005-322213 A JP 2006-92556 A

  By the way, in the display of an image (hereinafter referred to as an aggregated image) generated by aggregating extracted images in a stained glass shape as described above, characters included in each image constituting the aggregated image are reduced. The characters in each image may be divided by other extracted images, which may result in poor character readability.

  The present invention has been made in view of such circumstances, and an image aggregating apparatus and an image aggregating program that suppress the deterioration of the readability of characters in images constituting the aggregated image due to the generation of the aggregated image. The purpose is to provide.

The invention described in claim 1 is based on the search character information input by the input device , and extracts a plurality of image information related to the search character information as an extracted image, and a plurality of pieces extracted by the extraction unit When at least one of the extracted images is an extracted image including a character string and a pattern, and the area indicating the character string is larger than the area indicating the pattern, the extracted image is configured by a character string. A determination means for determining and a character string display area for displaying the character string in an enlarged manner are generated according to the number of extracted images, and an aggregated image is extracted from a plurality of extracted images based on the region of interest processing. The generation unit to be generated and the aggregated image generated by the generation unit are displayed, and each of the character string display areas generated by the generation unit is determined to be composed of character strings by the determination unit. An extraction image, a character string included in the extracted image, and display control means for displaying each said extracted image to an image aggregation device having a.

According to the second aspect of the present invention, the computer is extracted by extraction means and extraction means for extracting a plurality of image information related to the search character information as extracted images based on the search character information input by the input device. In addition, when at least one of the plurality of extracted images is an extracted image having a character string and a pattern, and the area indicating the character string is larger than the area indicating the pattern, the extracted image is configured by a character string. A determination unit that determines that the character string display area for displaying the character string in an enlarged manner is generated according to the number of extracted images, and the aggregated image is extracted from the plurality of extracted images extracted based on the region of interest processing Generating means for displaying the aggregated image generated by the generating means, and each character string display area generated by the generating means is constituted by a character string by the determining means. And the determined extracted image and the character string included in the extracted image, which is the image-intensive program for functioning as display control means, to be displayed for each said extracted image.

  According to invention of Claim 1, it can suppress that the readability of the character in the image which comprises an aggregated image is inferior.

According to invention of Claim 2 , it can suppress that the readability of the character in the image which comprises an aggregated image is inferior.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, exemplary embodiments of the invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a configuration diagram of an image aggregating apparatus according to an embodiment of the present invention.
As shown in FIG. 1, the image aggregating apparatus includes a search unit 11, a result analysis unit 12, a data management unit 13, a database 14, a text generation unit 15, an SG (Stained Glass) generation unit 16, a display control unit 17, and the like. Is done. The text generation unit 15, SG generation unit 16, display control unit 17 and the like constitute the display control means of the present invention.

  The image aggregation device is a so-called computer, that is, as shown in FIG. 2, a processing device such as a CPU 10a, SRAM (Static Random Access Memory), DRAM (Dynamic RAM), SDRAM (Synchronous DRAM), NVRAM (Non Volatile RAM). ) And the like, a ROM (Read Only Memory) 10c such as a flash memory, an I / F 10d that inputs and outputs to an external device such as an input device and a display device, and a magnetic disk such as a hard disk (not shown) are connected by a bus 10e. This is realized by the hardware configuration.

  Therefore, the CPU 10a reads a required program stored in the ROM 10c or the hard disk and performs an operation according to the program, thereby realizing each function in the image aggregation device. In addition, as such a program, it can be set as the program according to the flowchart mentioned later.

  The retrieval unit 11 retrieves an image for aggregation from the Internet, an in-house LAN, or the like based on an instruction from an input device such as a keyboard or a mouse. More specifically, as shown in FIG. 3, an image corresponding to character information in the image search input box 21 is searched from display images displayed on a display device such as a liquid crystal display or a CRT display. An example of the display image is a Web browser 20 or the like. In FIG. 3, for example, character information “GCC” is input to the input box 21, so an image having character information “GCC” or an image related thereto is searched. The extracted image extracted as the search result is transmitted to the result analysis unit 12.

The result analysis unit 12 receives the extracted image transmitted from the search unit 11, and analyzes whether the extracted image is composed of only characters, only the symbols, or whether they are mixed.
The data management unit 13 distinguishes and stores the extracted images in the database 14 according to the analysis result by the result analysis unit 12.

  The text generation unit 15 outputs a transmission request for an extracted image composed only of characters from the database 14 to the data management unit 13. Furthermore, when the text generation unit 15 acquires an extracted image based on the request, the text generation unit 15 acquires a character string in the extracted image and generates a character string display area (hereinafter referred to as a text layer) for displaying the character string. Then, a character string is added to each display layer in the text layer. The text layer to which the character string is added is transmitted to the display control unit 17.

  The SG generation unit 16 outputs a transmission request for an extracted image including only symbols from the database 14 to the data management unit 13. Further, when the SG generation unit 15 acquires the extracted image based on the request, the SG generation unit 15 acquires a pattern in the extracted image and generates a stained glass image (aggregated image) based on the region of interest processing. In the case of generating such an image, the technique disclosed in Patent Document 2 described above, the technique disclosed in Japanese Patent Laid-Open No. 2005-293576, Japanese Patent Laid-Open No. 2005-293577, or the like can be used. .

  The display control unit 17 causes the display device to display the text layer generated by the text generation unit 15 and the aggregated image generated by the SG generation unit 16.

Next, the operation of the image aggregation device will be described with reference to the drawings.
FIG. 4 is a flowchart showing an example of the operation of the image aggregation device, FIG. 5 is a diagram for explaining the extracted image and the display result according to the embodiment, and FIG. 6 is for explaining the extracted image and the display result according to the embodiment. FIG. 7 is another example of the display image according to the embodiment and the display image according to the comparative example.

  As shown in FIG. 4, the image aggregating apparatus first acquires all the text boxes in the page corresponding to the search character information (step S1). More specifically, when the extracted image is a page composed of only characters as shown in FIG. 5A and each character string is divided by a predetermined section, the extracted image is input to the input box 21. All text boxes in the page including the character information, for example, “GCC” are acquired. As a result, the text boxes Bx4, Bx6, Bx9, and Bx10 in FIG. Then, region numbers i = 1, 2, 3, and 4 are assigned in the order in which these acquired text boxes are acquired.

  Next, the image aggregating apparatus determines whether or not the acquired text box is the first text box (step S2). Here, whether or not it is the first text box is determined based on whether or not the acquired text box is acquired from the upper left to the lower right, and the first text box to be acquired is the first text box. . As a result, the text box Bx4 shown in FIG. 5 becomes the first text box.

Next, when the acquired text box is the first, the image aggregating apparatus sets the upper left corner of the text box as the minimum coordinate value (Xmin, Ymin) and the lower right corner as the maximum coordinate value (Xmax, Ymax) (step S3). ). Therefore, when the first text box is acquired, Xmin = X ₁₀ , Ymin = Y ₁₀ , Xmax = X ₁₁ , and Ymax = Y ₁₁ .

Next, the image aggregating apparatus determines whether or not the acquired text box is the last text box. According to FIG. 5, since the end of the text box is a text box Bx10 area number i = 4, as the next region number i = _{2, X 20} determines whether Xmin smaller (step S5). _{When X 20} is Xmin smaller than, the new _{X 20} and Xmin (step S6), and otherwise does not perform the processing of step S6.

Image centralizing device, _{likewise, Y 20} determines whether Ymin is smaller than (step S7). _{When Y 20} is Ymin smaller than, the new _{Y 20} and Ymin (step S8), and otherwise does not perform the processing of step S8.

Image centralizing device, _{then, X 21} determines whether Xmax greater (step S9). _{When X 21} is larger than Xmax is a new X21 to Xmax (step S10), and otherwise does not perform the processing of step S10.

Image centralizing device, _{likewise, Y 21} determines whether Ymax larger (step S11). _{When Y 21} is larger than Ymax is the new _{Y 21} to Ymax (step S12), the otherwise does not perform the processing in step S12.

If the image aggregation device determines that it is the last text box in the process of step S4, that is, if i = 4 is completed according to the present embodiment, it proceeds to the subsequent process. By repeating the process in step S12 in this way from the step S5, in Figure 5, X coordinate of the upper left corner of Bx9 (i = _3): the _{X 30} is Xmin, Bx4 (i = 1) X in the lower-right corner of the _coordinates: the _{X 11} is Xmax, Y coordinate of the upper left corner of _{Bx4 (i = 1): Y} 19 is a Ymin, Y coordinates of the lower right corner of _{Bx9 (i = 3): Y} 31 is Ymax. As a result, the text in the rectangular virtual area AR composed of the coordinates (Xmin, Ymin) and the coordinates (Xmax, Ymax) becomes the text layer generation target. This virtual area AR is the largest rectangle including a predetermined section as a layer generation target. The virtual area AR is referred to as text GERM.

  Next, the image aggregating apparatus acquires all the text boxes in the page, assigns region numbers j = 1, 2,... From the upper left to the lower right, and then acquires the jth text box (step S13), it is determined whether or not the coordinates of the text box are Xmin ≦ Xi0, Ymin ≦ Yi0, Xmax ≧ Xi1, and Ymax ≧ Yi1 (step S14). If these determination conditions are satisfied, the j-th text is acquired, and if there is text acquired so far, it is combined with the text (step S15). As a result of such processing, the text box Bx1 with j = 1 is excluded from the objects to be combined (aggregated), and the text box Bx3 with j = 3 becomes the object to be combined.

  Here, the image aggregating apparatus determines whether or not the combined text is the last text (step S16), and if it is not the last text, repeats the processing from step S13 to step S15. When the image aggregating apparatus finishes combining the last text, it stores the combined text in the database (step S17), and ends the process.

  As a result of such processing, in the text layer, as shown in FIG. 5B, the text in the text box combined in one display layer 31 is displayed divided into paragraphs for each text box. Then, the text in the text box combined with another extracted image is displayed on the other display layer 32 or the like. This makes it easy to grasp the entire sentence.

  Further, as shown in FIG. 6A, the text in the text box composed only of the acquired text box may be included as a combination target without being surrounded by the virtual area AR. According to the figure, the text included in the text boxes Bx4, 6, 9, and 10 described above may be displayed on the display layer 31 in the text layer. As a result, more information can be displayed in a display area limited by an information portable terminal such as a cellular phone or a PDA.

  In this embodiment, since there are a plurality of text boxes including search character information in the extracted image (page), the virtual area AR is set as the text GERM. However, if there is only one text box, the text The box may be the text GERM. Also, if the extracted image does not include search character information, the text box with the largest font, the text box placed at the top, the text box placed in the center, and the text box not included on other pages Etc. may be the text GERM.

  FIG. 7 is a diagram for explaining a display image according to the present embodiment and a display image according to a comparative example. In the display image according to the present embodiment, as shown in FIG. 7A, the text layer 33 described above is displayed on the Web browser 20 or the like. On the other hand, as shown in FIG. 7B, the display image according to the comparative example is displayed on the Web browser 20 or the like by reducing the extracted image extracted based on the search character information. Thus, the phenomenon that the extracted image is reduced in the comparative example, and as a result, the readability of the characters in the extracted image is deteriorated is suppressed by the image aggregation device according to the embodiment of the present invention.

(Second Embodiment)
Next, a second embodiment of the present invention will be described with reference to the drawings.
FIG. 8 is a flowchart showing an example of the operation of the image aggregation device, and FIG. 9 is an example of a display result for comparison.

  As shown in FIG. 8, the image aggregating apparatus first performs a search process from the Internet, an in-house LAN, or the like based on the search character information (step S21), and acquires an extracted image corresponding to the search character information (step S21). S22).

  Next, the image aggregating apparatus determines whether or not the extracted image is composed only of text (character string) (step S23). If it is determined that the extracted image is composed only of text, the image aggregating apparatus acquires page data in the order of pages as described in the first embodiment (step S24), and the text included in the text box is acquired. It is acquired (step S25), the text is added to the display layer up to the last text (steps S26 and S27), and when the addition of the text is completed, the text layer is displayed on the display device or the like (step S28).

  On the other hand, if it is determined that the extracted image is not composed only of text, the image aggregating apparatus then determines whether or not the extracted image is composed only of image information (design) (step S29). If it is determined that the extracted image is composed only of text, the image aggregating apparatus acquires all the extracted images based on the search results (step S30), creates an SG from these images (step S31), As shown in FIG. 9, a stained glass-like image (aggregated image) is displayed on a display device or the like (step S32).

  On the other hand, if the image aggregating apparatus determines that the extracted image is not composed only of image information, that is, determines that the extracted image is a mixture of text information and image information, the image aggregating apparatus acquires page data in page order ( Step S33). Then, it is determined whether or not the acquired page data is an image GERM (step S34).

  When it is determined that the page data is not the image GERM, that is, when it is determined that the page data is the text GERM, the image aggregation device acquires the text included in the text GERM. Here, whether or not the acquired page data is the image GERM is determined by assuming that the area indicating the image is P, the image area A of the entire page is T, and the area indicating the text is T, (P / A)> ( 1/3) or when P ≧ T is satisfied, the image is determined as GERM. Otherwise, it is determined as text GERM. The area P indicating the image is represented by the mathematical formula shown in FIG. 10A, and the area T representing the text is represented by the mathematical formula shown in FIG. Abs is a symbol indicating an absolute value.

  When determining that the page data is the text GERM, the image aggregating apparatus acquires the text included in the text GERM (step S35) and adds the text to the display layer (step S36). Then, the image aggregating apparatus determines whether or not the page data is the last page data (step S37).

  On the other hand, when it is determined that the page data is the image GERM, the image aggregating apparatus temporarily stores the page data as an image page (step S38), and adds an image thumbnail to the display layer (step S39).

  When the image aggregating apparatus completes the processing from step S35 to step S39 for all the acquired page data, it performs SG creation processing from the image page (step S40), displays the text layer (step S41), and performs processing. finish.

  As a result, as shown in FIG. 11, the aggregated image 36 based on the search character information “GCC” is displayed on the web browser 20 or the like, and the text layer 33 is also displayed. In the display layer 31, thumbnail images of images constituting the aggregated image 36 and character strings included in the constituting images are displayed. On the other hand, as shown in FIG. 12, if the extracted image 25 is reduced and displayed on the Web browser 20 or the like, the readability of the characters included in the images constituting the aggregated image 36 is not desirable.

  Further, as shown in FIG. 13, when one image constituting the aggregated image 36 is designated by the pointer Pt as the instruction image, characters included in the image that is the origin of the aggregated image are enlarged and displayed on the text layer 33. Is done. In this way, even if the characters included in the aggregation source image are segmented or reduced due to aggregation, the readability of the characters included in the aggregated image 36 is suppressed from decreasing.

  In the figure, regarding the region where the text layer 33 and the consolidated image 36 overlap, the display layers 31 and 32 of the text layer 33 are made opaque so that the text in the display layers 31 and 32 can be easily read. The display layers 31 and 32 in the text layer 33 may be transparent so that the image below the text layer 33 can be viewed from above the text layer 33, and the transparency thereof may be set. . Thereby, the visibility of the image hidden by the text layer 33 is improved.

(Other embodiments)
A display example in the case where the above-described text GERM and image GERM are mixed will be described below with reference to the drawings.
14 is an example of an extracted image in the image aggregation device, FIGS. 15 and 16 are examples of display layer display in the aggregate image, FIGS. 17, 18 and 19 are other examples of display layers in the aggregate image, and FIG. It is an example of another extracted image in an image aggregation device, and a display example of a display layer in an aggregate image.

  The image aggregating apparatus acquires an image as shown in FIG. 14 as an example of the extracted image, and generates an aggregated image as shown in FIGS. 15 and 16 together with the other extracted images and displays the aggregated image on a display device or the like. In each case, as shown in each figure, when the character string in the character string area in which the character strings are described is touched with the pointer Pt in the image after aggregation, the character strings are enlarged and displayed. The enlarged character string may be on the aggregated image or at a place other than on the aggregated image.

  Further, the image aggregating apparatus acquires an image as shown in FIG. 14 as an example of the extracted image, generates an aggregated image as shown in FIGS. 17, 18, and 19 together with the other extracted images, and displays it. As shown in each figure, the character string to be displayed may be changed by touching the designated position other than the character string with the pointer Pt.

  In this case, the character string to be displayed may correspond to the layout of the entire page shown in FIG. 14. For example, in FIG. 17, “the ratio of home ownership by country (2003)” closest to the indicated position. Can be displayed. Such display control is performed by displaying a character string arranged at the physical upper part of the clipped image when the designated position is at the upper part as viewed from the center of the clipped area (the center of the middle image in FIG. 17 and the like). Also good.

  Further, the image aggregating apparatus acquires an image as shown in FIG. 20A as an example of the extracted image, generates an aggregated image as shown in FIG. 20B together with the other extracted images, and displays it. For example, the character string closest to the designated position may be displayed as shown in FIG. Even in such a case, the closest character string based on the distance from the center of the extracted image 25 may be displayed on the text layer 33.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the specific embodiments according to the present invention, and various modifications are possible within the scope of the gist of the present invention described in the claims.・ Change is possible. For example, the program of the present invention can be provided not only by communication means but also stored in a recording medium such as a CD-ROM.

  ADVANTAGE OF THE INVENTION According to this invention, it can suppress that the readability of the character in the extraction image in an aggregate image is inferior, and industrial applicability is high.

It is a block diagram of an image aggregation apparatus. It is a block diagram which shows the hardware constitutions of an image aggregation apparatus. It is an example of a search screen. It is a flowchart which shows an example of operation | movement of an image aggregation apparatus. It is a figure for demonstrating the extraction image and display result which concern on embodiment. It is another figure for demonstrating the extraction image and display result which concern on embodiment. It is an example of the display image which concerns on embodiment, and the display image which concerns on a comparative example. It is a flowchart which shows an example of operation | movement of an image aggregation apparatus. It is an example of an aggregate image. It is a mathematical formula for calculating the area of the image GERM and the text GERM. It is an example of the display result which concerns on embodiment. It is an example of the display result concerning a comparative example. It is another example of the display result which concerns on embodiment. It is an example of an extracted image. It is an example of an aggregate image. It is an example of an aggregate image. It is an example of an aggregate image. It is an example of an aggregate image. It is an example of an aggregate image. It is an example of an extracted image and an aggregated image.

Explanation of symbols

10a CPU
10b RAM
10c ROM
10d I / F
10e Bus 11 Search unit 12 Result analysis unit 13 Data management unit 14 Database 15 Text generation unit 16 SG generation unit 17 Display control unit 20 Web browser 21 Input box 25 Extracted images 31, 32 Display layer 33 Text layer 36 Aggregated image AR Virtual region Pt Pointer Bt Search button

Claims (2)

Extraction means for extracting a plurality of pieces of image information related to the search character information as extracted images based on the search character information input by the input device;
When at least one of the plurality of extracted images extracted by the extracting means is an extracted image including a character string and a design, and the area indicating the character string is larger than the area indicating the design, the extracted image Determining means for determining that is composed of a character string;
Generating a character string display area for enlarging and displaying the character string according to the number of the extracted images, and generating an aggregated image from the extracted extracted images based on a region of interest process Means,
The aggregated image generated by the generating unit is displayed, and the extracted image determined by the determining unit to be composed of the character string in each of the character string display areas generated by the generating unit, and the extracted image Display control means for displaying the character string included in each extracted image;
An image aggregating apparatus. Computer
Extraction means for extracting a plurality of image information related to the search character information as extracted images based on the search character information input by the input device;
When at least one of the plurality of extracted images extracted by the extracting means is an extracted image including a character string and a design, and the area indicating the character string is larger than the area indicating the design, the extracted image Determining means for determining that is composed of a character string;
Generating a character string display area for enlarging and displaying the character string according to the number of the extracted images, and generating an aggregated image from the extracted extracted images based on a region of interest process means,
The aggregated image generated by the generating unit is displayed, and the extracted image determined by the determining unit to be composed of the character string in each of the character string display areas generated by the generating unit, and the extracted image Display control means for displaying a character string included in each extracted image,
Image aggregation program to function as

Images