JP6152633B2 - Display control apparatus and program - Google Patents

Description

  The present invention relates to a display control device and a program.

  In recent years, music, books, images, and other contents can be easily handled as data by the development of mobile computing as represented by WWW (World Wide Web) and mobile terminal devices. As well as being able to carry it easily and easily.

  In recent years, content data obtained by digitizing newspapers, magazines, books, and the like has been purchased and browsed on mobile terminal devices such as smartphones or tablet terminal devices.

  On the other hand, as a data format used for browsing such content data, a format produced by a character code base that allows the user to freely change the typesetting of the display according to the display size such as EPUB (Electronic PUBlication), A bitmap format having a printed page format that can accurately reproduce the editing intention of the editor (that is, the typesetting format is fixed) is known. Also, a PDF (Portable Document Format) format is known as a data format that can be produced by a character code base or a bitmap format. However, in any case, the PDF format prohibits the user from freely changing the typesetting style and has a style that can accurately reproduce the editing intention of the editor. .

In addition, when displaying an image such as a document based on bitmap format data in a printed page format, compared to displaying an image based on bitmap format data produced on a character code basis,
(1) Problems such as difficulty in obtaining source character code data due to copyright restrictions are difficult to occur, easy handling of data, and (2) garbled characters based on fonts and layout disturbances. Never
Has the advantage.

  In addition, the bitmap format of the printed page format can be converted into data simply by taking paper-based material as image data, so that the user can easily use it even when converting it into data. have.

  However, the bitmap format of the print page format is a display screen that displays only a part of one page when browsing with a small screen size such as a portable terminal device, etc., if the font size is increased to ensure ease of viewing. On the other hand, if the entire page is displayed, the characters are too small to be read.

  Therefore, recently, there has been proposed a method of disassembling and reconstructing content data such as various printed page format bitmap documents in the bitmap format of the printed page format (for example, Patent Documents 1 to 3).

JP 2004-5453 A JP-T 2009-531789 JP 2011-242987 A

  However, the above Patent Documents 1 to 3 disclose specific methods for disassembling and reconstructing content data including vertical writing of documents peculiar to Japanese, ruby assigned to the corresponding kanji, or prohibition processing. Not.

  The present invention has been made to solve the above-described problems, and its purpose is to provide a display control device capable of improving the viewability without depending on the display size in the bitmap format of the print page format. Is to provide.

  In order to solve the above-described problems, a display control device or the like according to the present invention is a display control device for displaying an image formed by a plurality of pixels arranged in a matrix on a display unit, and at least a document is An acquisition means for acquiring image data for displaying the imaged image as a document image on the display means from an external or storage means, and a size of a display area in the display means when the document image is displayed. Setting means for setting, recognition means for recognizing the line direction and line feed direction of the document in the document image based on the acquired image data, and binarization for binarizing each pixel value of the document image And the document image based on a pixel value in each of the binarized pixels for each first array line that is an array line of pixels in the document row direction of the document image. Line detection means for detecting the line of the document image, and for each of the binarized pixels for each of the detected lines, the second array line that is an array line of pixels in the document line feed direction of the document image. Based on the pixel value, character section detection means for detecting a section of characters included in each line as a character section, and based on the set area size of the display area, the detected character sections are displayed in the display area. An arrangement position determining means for determining an arrangement position for arranging the character image; and a section image corresponding to each character section, which is a part of the document image, is disposed at the determined position of each character section An image generation unit that generates a display image to be displayed in the display area and an output unit that outputs the generated image to the display unit are included.

  The display control device and the program according to the present invention can improve the user's viewability without depending on the area size of the display area even when the display area of the display means such as the portable terminal device is small. it can. The user's viewability can be improved without depending on the display area.

It is a block diagram which shows schematic structure of the portable communication terminal device of one Embodiment which concerns on this invention. It is a figure for demonstrating the flow at the time of imaging the document of a character code base relevant to this invention. It is a figure for demonstrating the flow at the time of imaging the document imaged by the bitmap format of the print page appearance relevant to this invention. It is a figure for demonstrating the display format of the document imaged by the bitmap format of the print page appearance relevant to this invention. It is FIG. (1) for demonstrating an example of the effect in the mobile communication terminal device which concerns on this invention. It is FIG. (2) for demonstrating an example of the effect in the mobile communication terminal device which concerns on this invention. It is a figure for demonstrating the binarization process in the character arrangement | positioning analysis process part of one Embodiment, and is a figure which shows an example of a binarization process. It is FIG. (1) for demonstrating the line detection process in the character arrangement | positioning analysis process part of one Embodiment. It is FIG. (1) for demonstrating the line detection process in the character arrangement | positioning analysis process part of one Embodiment. It is FIG. (1) for demonstrating the character division detection process in the character arrangement | positioning analysis process part of one Embodiment. It is FIG. (2) for demonstrating the character division detection process in the character arrangement | positioning analysis process part of one Embodiment. It is FIG. (3) for demonstrating the character division detection process in the character arrangement | positioning analysis process part of one Embodiment. FIG. 6 is a diagram (part 4) for explaining the character section detection processing in the character arrangement analysis processing unit of the embodiment; It is a figure for demonstrating the integrated correction | amendment process in the character arrangement | positioning analysis process part of one Embodiment. It is a figure for demonstrating the headline analysis process in the character arrangement | positioning analysis process part of one Embodiment. It is a figure for demonstrating the indentation analysis process in the character arrangement | positioning analysis process part of one Embodiment. It is a figure for demonstrating the ruby analysis process in the character arrangement | positioning analysis process part of one Embodiment. It is a figure for demonstrating the prohibited character analysis process in the character arrangement | positioning analysis process part of one Embodiment. It is FIG. (1) for demonstrating the division arrangement | positioning process in the division arrangement | positioning process part of one Embodiment. It is FIG. (2) for demonstrating the division arrangement | positioning process in the division arrangement | positioning process part of one Embodiment. It is a figure for demonstrating the image generation process in the image generation process part of one Embodiment. It is a flowchart which shows the operation | movement of the display process of the document image based on the browsing application in the portable terminal device of one Embodiment. In one Embodiment, it is a flowchart which shows the operation | movement of the character arrangement | positioning analysis process during browsing application execution. In one Embodiment, it is a flowchart (the 1) which shows operation | movement of the division | segmentation arrangement | positioning process during browsing application execution. In one Embodiment, it is a flowchart (the 2) which shows operation | movement of the division | segmentation arrangement | positioning process during browsing application execution. In one Embodiment, it is a flowchart (the 3) which shows operation | movement of the division | segmentation arrangement | positioning process during browsing application execution. It is a figure for demonstrating an example of the display process in the case of the horizontal composition in one Embodiment. It is a figure for demonstrating an example of the display process in the case of the vertical composition in one Embodiment. It is a figure for demonstrating the modification based on one Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, the following embodiment is embodiment at the time of applying the display control apparatus and program which concern on this invention with respect to a portable terminal device.

[1] Overview of Portable Terminal Device First, an overview of the portable terminal device 10 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of the portable terminal device 10 in the present embodiment.

  The portable terminal device 10 is, for example, a communication terminal device such as a PC (Personal Computer), a tablet information terminal device, a smartphone, or a portable game machine, and includes various hardware included in the portable terminal device 10. It has a configuration for executing a program (hereinafter referred to as “browsing application (document viewer)”) that cooperates and displays image data in which a document is imaged so as to be viewable to the user.

  In particular, the portable terminal device 10 can accurately reproduce the editing intention of an editor such as a PDF (Portable Document Format), and a printed page on which elements of a document such as characters and illustrations are fixedly arranged. It has a display function for displaying image data (hereinafter simply referred to as “document data”) configured in a format bitmap format.

  However, the PDF format data is not necessarily in the bitmap format but can be formed in the character code base format that refers to the font at the time of display, but even in the character code base format, the same as the bitmap format. In order to secure the editor's intention, the typesetting style cannot be freely changed.

  Then, when one document data is selected and a browsing application is executed based on a user operation, the portable terminal device 10 is an area designated in the display screen when displaying a document image set by the user ( Hereinafter, the selected document data is displayed based on the area size of “designated display area”.

Specifically, when the portable terminal device 10 executes a browsing application,
(1) Image analysis is performed on each arrangement of characters constituting the document in the document data, and an arrangement position (hereinafter referred to as “section image”) in each character section (hereinafter referred to as “character section”). , “Partition position”) and its size (hereinafter referred to as “partition size”),
(2) Based on the arrangement position and the division size of the character section, the arrangement position for arranging each character section in the designated display area designated in the display screen by the user is determined while maintaining the document structure of the document. Partition placement processing,
(3) A bitmap generation process for generating a bitmap-format display image for allocating a partition image of a corresponding character partition based on each determined layout position and displaying it in the designated display area; and
(4) Output processing for outputting the generated image to the display screen is executed, and the acquired document data can be displayed in the set designated display area.

  In order to implement each of these processes, the portable terminal device 10 manages a data storage unit 100 in which necessary data including a browsing application is stored and a plurality of document data (not shown) as shown in FIG. A communication control unit 110 that communicates with a server device and other communication devices, an application processing unit 120 that executes and manages various applications including the browsing application, and a display unit that has a display screen of a predetermined size 150, a display control unit 140 that controls various drawing operations on the display unit 150, an operation unit 160 that is used to input user operations, and a terminal management control unit 190 that controls the whole.

  The portable terminal device 10 may have functions other than document browsing. For example, the portable terminal device 10 has a map function such as navigation, an imaging function such as a camera, a telephone function, and a mail function such as e-mail. May have various necessary members such as a GPS receiver, a microphone, a speaker, and a communication unit. The above units are connected to each other by a bus B so that data is exchanged.

  For example, the data storage unit 100 of the present embodiment constitutes a storage unit of the present invention, and the application processing unit 120 includes an acquisition unit, a setting unit, a binarization unit, a line detection unit, a character segment according to the present invention. A detection unit, an arrangement position determination unit, and an image generation unit are configured. Further, for example, the output means of the present embodiment constitutes the display control means of the present invention, and the display unit 150 constitutes the display means and the display device according to the present invention. Furthermore, for example, the operation unit 160 of the present embodiment constitutes an operation unit according to the present invention.

[2] Principle of the present invention Next, the principle of the present invention will be described with reference to FIGS. FIG. 2 is a diagram for explaining the flow of displaying a character code-based document while imaging it. FIG. 3 displays a document that has already been imaged in the print page format bitmap format. It is a figure for demonstrating the flow in the case. FIG. 4 is a diagram for explaining a display format of a document imaged in the bitmap format of the print page format, and FIGS. 5 and 6 are diagrams for explaining an example of the effect of the present invention. It is.

  The portable terminal device 10 of the present embodiment displays a document image formed by a plurality of pixels arranged in a matrix of m rows and n columns on the display unit 150 so that the user can view the document image. Because

(1) The document data selected by the user is acquired from another communication terminal device via a network (not shown), or the document data stored inside is read and acquired.
(2) Based on the user operation, the area size of the designated display area designated in the display screen when displaying the document image is set,
(3) executing a character arrangement analysis process for detecting a character section corresponding to each character of the imaged document in the acquired document data and performing image analysis of the character arrangement;
(4) A section arrangement process for determining an arrangement position for arranging the detected character section in the designated display area seamlessly for the document data for each page,
(5) Assign a partition image corresponding to each character partition whose placement has been determined and execute bitmap generation processing,
(6) It has a configuration for outputting the generated bitmap image.

Usually, as shown in FIG. 2, when imaging a character code-based document,
(1) A corresponding font is selected based on a typesetting instruction by a user operation or the like and text data indicating a character code,
(2) The selected font is converted into a bitmap according to the typesetting instruction (that is, bitmap conversion processing is executed),
(3) While the image data converted into the bitmap is drawn in the video memory,
(4) A document imaged by the drawn drawing data is displayed. Therefore, in such a case, it is easy to change the document structure on the basis of a viewer's instruction such as a display area. For example, an imaged document is displayed in accordance with the changed display area. It is also possible.

  Therefore, in order to image such character code-based document data by reflecting the intention of the editor who edited (or produced) the document data, the character code and the character font used by the editor are determined. Necessary. Therefore, in the case of document data in which a character font used by the editor can be embedded in a document file in PDF format or the like, the document image can be displayed reflecting the editor's intention, but the character In the case of document data in which fonts cannot be embedded, it is necessary to refer to character fonts pre-installed in the portable terminal device 10, so that it is not possible to provide a document that reflects the editor's intention. In addition, there are many cases where characters included in a document cannot be accurately imaged.

  Usually, since fonts used in document data are various, there are many cases where it is not possible to prepare a character font used by an editor on the browsing side even in the same language. Therefore, when the corresponding character font is not embedded in the document data and is not installed in the portable terminal device 10 that browses the document data, another character font (proxy font) is used. . However, when a corresponding character pattern (pattern for imaging) is not defined for some character codes, or there are many character fonts in which character patterns different from other character fonts are defined, If the same character is not defined in the character font used for editing and the proxy font, the character included in the document data cannot be displayed, or a different character is displayed for the character. A garble will occur. Even if the corresponding character pattern is defined in the proxy font, the character width may be different from the editor's intention. In this case, the line feed position may be shifted depending on the font size. Sometimes.

  In addition, in character code-based document data, the text data used as the character code and the character font used by the editor are used to change the typesetting based on the viewer's instructions while reflecting the intention of the editor. In addition, when an image is included, it is essential to prepare a material such as the original image. However, the use of these raw materials may not be permitted due to copyrights, etc., and if the use of such raw materials is not permitted, the editor's intentions should be accurately determined when imaging the document data. Difficult to reflect. In particular, as described above, because the character font intended by the editor cannot be used, depending on the typesetting instruction such as setting of the display area, the layout of the document itself may be disturbed, and the image display itself may be defective. is there.

  As a result, when a character code-based document is imaged, the intention of the editor who generated or edited the document data is reflected in a data format other than the document data in which character fonts such as PDF are embedded. In addition, it is difficult to image and display the document data.

  On the other hand, as shown in FIG. 3, unlike the character code base, in order to display a document imaged in the print page format bitmap format, the print page format cannot be adjusted by the viewer. Instead of the character code, the image itself completed by the editor can be displayed while maintaining the display format. Therefore, in such a case, the management of copyright and the like is limited to the range of the paper layout style, and the handling of the data is remarkably facilitated as compared with the character code-based document data. Also, in such a data format, the image itself completed by the editor is displayed while maintaining the display format, similarly to document data in which character fonts such as PDF are embedded. Neither garbled characters nor layout disturbances occur.

  On the other hand, a document imaged by the above-described print page format bitmap format or by document data in which character fonts are embedded cannot be adjusted. For example, as shown in FIG. The document structure cannot be adjusted according to the user's ability, and the user's viewability is difficult. For example, as shown in FIG. 4A, when the size of the character is ensured while the size of the document image in the row direction is larger than the size of the display region in the row direction, the characters are displayed in the row direction. It is necessary to perform scroll processing for moving the area. In addition, when the display is not scrolled in the line direction, as shown in FIG. 4B, it is necessary to display the entire document in a reduced size, the characters become smaller, and the viewability also deteriorates.

  Therefore, the portable terminal device 10 according to the present embodiment determines the character segment corresponding to the character of each line of the imaged document based on the size of the set display area, By assigning an image corresponding to each character section to the determined arrangement position of each character section, each character is arranged while maintaining the document structure of the imaged document. In addition, the user's viewability can be improved.

  For example, as shown in FIG. 5, even if a moderate character size is ensured, scrolling in the line direction is restricted when the width of the display area in the line direction is narrow, while scrolling only in the line feed direction. A displayable character arrangement can be realized.

  In the case of producing an electronic book that is such an imaged document, as shown in FIG. 6, a general book that performs editing, plate making, printing, bookbinding, scanning, character recognition, editing, and format conversion is used. In the normal production process, the image data such as PDF or scanned data can be produced simply by converting the format after plate making. It can be produced.

[3] Portable Terminal Device Next, details of the configuration of the portable terminal device 10 of the present embodiment will be described using FIG. 1 described above.

  The data storage unit 100 includes, for example, a hard disk drive (hereinafter abbreviated as “HDD”), a solid state drive (hereinafter abbreviated as “SSD”), or a nonvolatile flash memory such as a NAND type or a NOR type. The

  In addition, the data storage unit 100 stores various applications executed by the application processing unit 120 and the terminal management control unit 190 including a browsing application, and content data including document data, and the application processing unit 120. As a work area of the terminal management control unit 190, a random access memory (RAM) and a read only memory (ROM) are also included.

  Specifically, the data storage unit 100 includes at least an application storage unit 101, a content data storage unit 102, and a ROM / RAM 103.

  The communication control unit 110 constructs a communication line with a server device or other communication device (not shown), and exchanges various data including document data.

The application processing unit 120 is mainly configured by a central processing unit (CPU), and based on various applications stored in the application storage unit 101 , each process is performed in conjunction with the display control unit 140 and the operation unit 160. To realize. In particular, the application processing unit 120 executes various processes in the browsing application for allowing the user to browse the document data according to the area size of the designated display area on the display screen of the display unit 150 designated by the user.

Specifically, the application processing unit 120
(1) Data acquisition processing for acquiring image data for displaying a document image on the display unit 150 from the outside or the data storage unit 100 via the communication control unit 110;
(2) Display size setting processing for setting the size of the designated display area in the display unit 150 when displaying a document image;
(3) A recognition process that is a process of character arrangement analysis processing that recognizes the line direction and line feed direction of an imaged document based on the acquired image data.
(4) In each page set as one page in the document image (hereinafter referred to as “document page”), which is one process of the character arrangement analysis process, each pixel value of the acquired document image is binarized. Binarization processing;
(5) This is a process of character arrangement analysis, and in each document page, binarization is performed for each pixel array line (hereinafter referred to as “first array line”) in the document image row direction. A line detection process for detecting a line of the document image based on the pixel value in each pixel;
(6) One process of character arrangement analysis processing, in each document page, for each detected line, an array line of pixels in the document line feed direction of the document image (hereinafter referred to as “second array line”). A character segment detection process for detecting a character segment included in each line as a character segment based on the binarized pixel value of each pixel;
(7) division arrangement processing for determining an arrangement position for arranging each detected character division in the display area based on the set area size of the designated display area;
(8) Bitmap generation processing for generating a display image for displaying a section image corresponding to each character section in a designated display area by placing a section image corresponding to each character section at the determined position of each character section When,
To output the generated bitmap image.

  In particular, in the character arrangement analysis process, the application processing unit 120 is based on the coordinate position in the image in the document image and the reference size of the character section for each document page (hereinafter also referred to as “document data being analyzed”). In addition to detecting each character section, setting of attribute information based on the speciality of each character section, that is, a headline character, indentation, ruby, prohibited character (hereinafter referred to as “prohibited character”), etc. It is determined whether or not it is a special character, and attribute information indicating that is set for a character section determined to be a special character.

  Then, the application processing unit 120 determines a position at which each detected character segment is arranged in the set designated display area in accordance with a predetermined arrangement condition based on the detected size and position of each character segment. That is, the application processing unit 120 uses the attribute information set by the character placement analysis process and the placement condition based on the size and the position of the character section together with the area size of the designated display area set by the user in the section placement process. Based on this, an arrangement position for arranging each detected character section in the display area is determined.

  The application processing unit 120 functionally executes a data management control unit 121 that executes acquisition, storage, and reading of document data in order to execute each of the above processes, and a specified display area that sets a specified display area by a user or the like. A setting unit 122, a character arrangement analysis processing unit 123 that executes character arrangement analysis processing, a division arrangement processing unit 124 that executes division arrangement processing, and an image data generation unit 125 that generates a bitmap image are realized. Details of each part of the application processing unit 120 of this embodiment will be described later.

  The display unit 150 has a display screen of a predetermined size, and is configured by, for example, a panel of electronic paper, a liquid crystal element, or an organic EL (Electro Luminescence) element, and is predetermined based on display data generated by the display control unit 140. The image of is displayed.

  The display control unit 140 generates display data necessary for display on the display unit 150, and outputs the generated display data to the display unit 150. In particular, the display control unit 140 generates and outputs display data for displaying the bitmap image generated by the application processing unit 120.

  The operation unit 160 includes various confirmation buttons, operation buttons for inputting operation commands, a number of keys such as a numeric keypad, and a touch panel formed to overlap the display unit 150, and is used when performing each operation. It has become. In particular, the operation unit 160 is used when designating a designated display area for displaying document data.

  The terminal management control unit 190 is mainly configured by a central processing unit (CPU) and includes various input / output ports such as a key input port and a display control port, and stores various applications stored in the data storage unit 100. By executing this, overall functions of the portable terminal device 10 are controlled comprehensively.

[4] Application Processing Unit [4.1] Data Management Control Unit Next, the data management control unit 121 in the application processing unit 120 of the present embodiment will be described.

  The data management control unit 121 is linked with the communication control unit 110 to exchange document data based on communication with a server device or another communication device (not shown) and manage data of document data stored in the data storage unit 100.

  Specifically, the data management control unit 121 includes resource data for the WWW system described in a markup language such as HTML (Hyper Text Markup Language) and a unique URL (Uniform) indicating the network address of the resource data. Based on the browsing function using Resource Locator), data communication is performed with a server device or other communication device (not shown). In particular, the data management control unit 121 acquires document data desired by the user based on the browsing function and the user operation during execution of the browsing application.

  Further, the data management control unit 121 acquires the document data stored in advance in the data storage unit 100 based on a user operation. However, the document data stored in the data storage unit 100 is data acquired (downloaded or transferred) from a server device or other communication device (not shown) by the browsing function.

  Note that the data management control unit 121 may be document data having a monochrome document image or document data having a color document image.

  Further, the data management control unit 121 may acquire image data (that is, document data) in an imaged document while reading the document with an image scanner. Then, the data management control unit 121 executes tilt correction on a document imaged by the image scanner based on a marker previously marked on the document or a ruled line laid out in a document page that can be used as a reference. . This inclination correction is important for ensuring that the line direction is horizontal and for ensuring that the line feed direction is vertical. However, the data management control unit 121 may not guarantee the level of each document data acquired by data communication. Therefore, the data management control unit 121 applies each document data acquired by the data communication based on a user instruction or automatically. It may be executed against.

[4.2] Designated Display Area Setting Unit Next, the designated display area setting unit 122 in the application processing unit 120 of the present embodiment will be described.

  The designated display area setting unit 122 sets a rectangular designated display area formed in the display screen of the display unit 150 according to a user instruction or the like.

  In particular, the designated display area setting unit 122 recognizes the coordinates on the designated display screen when instructed to enlarge and reduce the designated display area in the vertical and horizontal directions, and designates the designated display area according to the recognized coordinates. While setting the display area, the set area size of the designated display area is output to the partition arrangement processing unit 124.

  The designated display area setting unit 122 is instructed when the designated display area is designated at an arbitrary timing via the operation unit 160 even when the document data is displayed using the browsing application. The coordinates on the displayed screen are recognized, and the designated display area is set according to the recognized coordinates.

[4.3] Character Placement Analysis Processing Unit [4.3.1] Outline of Character Placement Analysis Processing Unit Next, the character placement in the application processing unit 120 of this embodiment will be described with reference to FIGS. The analysis processing unit 123 will be described.

  The character arrangement analysis processing unit 123 performs the above-described recognition processing, binarization processing, line detection processing, and character section detection processing for each page of the document structure of the acquired document data (that is, for each page of the original document format). Execute.

  In addition, when the character arrangement analysis processing unit 123 executes the recognition process, the binarization process, the line detection process, and the character section detection process, two or more character sections having a predetermined condition in each line are integrated as a correction process. If it is determined that each character section is a character section constituting a special character based on the integrated correction process and various predetermined conditions, and it is determined that the character section corresponds to the special character And a special character analysis process for setting the corresponding attribute as attribute information.

  In particular, the character arrangement analysis processing unit 123 performs a headline analysis process, an indentation analysis process, a ruby analysis process, and a prohibited character analysis process as the special character analysis process.

Specifically, the character arrangement analysis processing unit 123 performs the integrated correction process as follows:
(1) Based on the detection result of the first pixel when the character section is detected, the line direction of the document of two adjacent character sections belonging to the same line for each line (that is, the first array line) Character pitch indicating the arrangement interval in (direction),
(2) When the calculated character pitch has a predetermined character pitch condition, integrated correction is executed with the two character sections having the character pitch condition as the same character section.

On the other hand, the character arrangement analysis processing unit 123 performs the headline analysis processing as follows:
(1) Based on the detection result of the first pixel (for example, black pixel) at the time of detecting the document row, each row is adjacent to the document line feed direction (that is, the second array line direction). Calculate the interval between two rows as the row pitch,
(2) When the calculated line pitch has a predetermined line pitch condition, one of the two lines having the line pitch condition is found based on a base point that determines the line pitch. Identified as a line,
(3) Heading attribute information is set in the character section belonging to the specified heading line.

  The base point for determining the line pitch is that when the line pitch is determined by the upper end (in the case of horizontal assembly) or the left end (in the vertical combination) of the adjacent two preceding and subsequent stages, If the line pitch is determined by the lower end (in the case of horizontal assembly) or the right end (in the case of vertical assembly) of two adjacent preceding and succeeding rows, the following row will be “One row determined based on a base point that determines the row pitch”.

Then, the character placement analysis processing unit 123 performs the indentation analysis processing as follows:
(1) For each line, when the arrangement position of the character section located at the head of the line has a predetermined head placement condition, the head character section of the line having the head placement condition is set as the head of the paragraph of the document. Identified as a character,
(2) Indentation attribute information is set in the specified character section.

In addition, the character arrangement analysis processing unit 123 performs ruby analysis processing as follows:
(1) Based on the detection result of the first pixel (for example, black pixel) at the time of detecting the document row, for each row, two adjacent rows in the document line feed direction (second array line direction) Calculate the arrangement interval as the line pitch,
(2) When the calculated pitch width of the line pitch has a predetermined line pitch specifying condition, a line determined based on a base point for determining the line pitch among the two lines having the line pitch specifying condition Is identified as a ruby line that grants the ruby,
(3) Line direction of each character section in the ruby target line that is the next line to the first position in the line direction of each character section belonging to the specified ruby line and the line feed direction of the document in the ruby line Each of the second position is identified as a ruby target character in a ruby target line that minimizes the difference between the first position and the second position,
(4) Set ruby attribute information for the character sections belonging to the specified ruby line, and set the ID of the character section that is the target of ruby (specifically, the search position of the line number and the order of the corresponding lines). The ruby target attribute information is set in the character section belonging to the specified ruby target line.

Furthermore, the character arrangement analysis processing unit 123 performs the prohibited character analysis processing as follows:
(1) When the width in the line direction or the width in the line feed direction of each character section has a predetermined section width condition, the character section having the section width condition is specified as a prohibited character,
(2) Forbidden character attribute information is set in the specified character section.

  In the present embodiment, the character arrangement analysis processing unit 123 performs various calculations as preprocessing (preprocessing for correction processing) as described later for the character pitch or line pitch used in the integrated correction processing or the like. It is calculated by executing.

  The character arrangement analysis processing unit 123 executes various processes for each page in the original document structure of the document data. However, the character arrangement analysis processing unit 123, when there is a plurality of pages of document data, and the next page data exists, the character arrangement analysis processing unit 123 executes the character data for the data corresponding to the next page after executing the current page character arrangement analysis process. Execute placement analysis processing.

[4.3.2] Recognition processing As the recognition processing, the character arrangement analysis processing unit 123 identifies the page layout of the imaged document based on the flag information added to the acquired document data, and the document Recognize line direction and line feed direction of the document in the data.

  For example, the character arrangement analysis processing unit 123 determines, based on the flag information, whether the document of the acquired document data is horizontal writing (that is, horizontal writing) or vertical writing (that is, vertical writing). , Recognizing the line direction in which characters are arranged along the document (that is, the direction in which the characters are read) and the line feed direction of the document that is orthogonal to the line direction (that is, the line feed direction of the document). .

In addition, the character arrangement analysis processing unit 123 performs recognition processing instead of the above.
(1) Detecting the number of blank lines where no character exists in the second array line direction based on the pixel value of each pixel belonging to each first array line,
(2) detecting the number of blank lines in which no character in the first array line direction exists based on the pixel value of each pixel belonging to each second array line;
(3) The line direction and line feed direction of the document may be recognized based on the number of blank lines in the first array line direction and the number of blank lines in the second array line direction.

  In particular, in this case, the character arrangement analysis processing unit 123 recognizes an array line direction having a large number of blank lines as a line feed direction, and recognizes an array line direction having a small number of blank lines as a line direction.

  Usually, the distance between characters in the line direction is shorter than the distance between lines in the line feed direction. Therefore, the distance between the characters in the line direction and the distance between the lines in the line feed direction are calculated from the number of blank lines in the first array line direction and the number of blank lines in the second array line direction, and the two distances are compared. For example, the row direction can be recognized.

  Therefore, the character arrangement analysis processing unit 123 can recognize an array line direction with a large number of blank lines as a line feed direction, and can recognize an array line direction with a small number of blank lines as a line direction.

  In this case, as the number of blank lines in the first array line direction and the number of blank lines in the second array line direction, values between two representative lines and between two characters may be used. The average between each line and between each character may be used. Further, as will be described later, the calculated maximum partition size, average line pitch, and reference character pitch in the first array line direction and the second array line direction may be used.

[4.3.3] Binarization Processing Next, the binarization processing in the character arrangement analysis processing unit 123 of this embodiment will be described with reference to FIG. FIG. 7 is a diagram for explaining the binarization processing in the character arrangement analysis processing unit 123 of this embodiment, and is a diagram illustrating an example of the binarization processing.

  As the binarization processing, the character arrangement analysis processing unit 123 converts a document image formed in color into a grayscale image on a document page being analyzed, and forms the converted grayscale image or monochrome. Noise that directly binarizes a document image based on a predetermined threshold and removes quantization noise (spotted noise associated with binarization processing) in the binarized document image Perform correction.

  Specifically, when the acquired document image is a color image, the character arrangement analysis processing unit 123 executes a predetermined calculation process and converts it into a black and white grayscale image by the predetermined calculation process. For example, the character arrangement analysis processing unit 123 converts the RGB pixel values R (x, y), G (x, y), and B (x, y) to gray scale for each pixel based on (Equation 1). Pixel value P (x, y) is calculated. In addition, the character arrangement analysis processing unit 123 applies a character to the converted grayscale image in the case of a color image, or to the acquired document image in the case of a monochrome image based on a threshold value. The pixel value “1” constituting the image or the pixel value “0” constituting the background is converted.

  For example, when the gradation value of each pixel is 0 to 255, the character arrangement analysis processing unit 123 executes binarization processing using 200 as a threshold value. In addition, as noise correction, the character arrangement analysis processing unit 123 performs image processing based on a morphological operation for performing isolated point removal, discontinuous point filling, and the like on a binarized document image.

  Further, as shown in FIGS. 6A and 6B, the character arrangement analysis processing unit 123 executes white and black binarization processing for a horizontally or vertically written color or grayscale document image. .

[4.3.4] Line Detection Processing Next, line detection processing in the character arrangement analysis processing unit 123 of this embodiment will be described with reference to FIGS. 8 and 9 are diagrams for explaining the line detection processing in the character arrangement analysis processing unit 123 of the present embodiment.

  As the line detection processing, the character arrangement analysis processing unit 123 performs one pixel value when binarizing from the pixels belonging to each first array line for each first array line in the document page being analyzed. At least pixels having (that is, black or white pixel values) are detected, and each row of the imaged document is detected based on the presence or absence of the detected pixels.

  In the present embodiment, a pixel having a pixel value “1” indicating black (hereinafter referred to as “black pixel”) or a pixel having a pixel value “0” indicating white (hereinafter referred to as “white pixel”). The character arrangement analysis processing unit 123 detects at least any pixel. However, in the following description, the character arrangement analysis processing unit 123 will be described using a case where the first pixel (black pixel) is detected.

  Specifically, the character arrangement analysis processing unit 123 counts black pixels for each pixel line (first array line) in the document row direction in the document image. Then, the character arrangement analysis processing unit 123 determines that the line in which the black pixel count is “0” is a blank line and constitutes a line between the lines of the document in the document image, and the black pixel count is “1”. It is determined that the above-described line is a line forming each character in the document image as a line forming line on which the character is formed.

  In addition, the character arrangement analysis processing unit 123 detects one or more lines or line group areas in which blank lines are formed in the first array lines as inter-line intervals, and row formation lines in the first array lines are formed. A region of one or more lines or line groups is detected as a row.

  At this time, the character arrangement analysis processing unit 123 assigns a code (that is, identification information, which is “number” in the present embodiment) in order from the first line of the document image to the detected line. To do.

(For horizontal document structure)
For example, when the document image is formed with a rectangular size of (Sx × Sy), the coordinate values (x, y) of each pixel on the document image are (0, 0) to (Sx−1, Sy). -1) is assumed.

  In such a case, when the document structure of the document image is horizontal composition, the x direction is the row direction and the y direction is the line feed direction. In addition, the y line in which pixels are arranged in the x direction is the first array line, and the x line is the second array line.

  At this time, as shown in FIG. 8A, the character arrangement analysis processing unit 123 detects each pixel value for each y line and counts black pixels. Further, when the black pixel count is “1” or more, the character arrangement analysis processing unit 123 determines that the y line is a row formation line, and when the count is “0”, the y line is a blank line. judge. Further, as shown in FIG. 8B, the character arrangement analysis processing unit 123 detects, as the line spacing LS, the area of the y line in which blank lines are formed or the group of y lines in which blank lines are continuously formed. At the same time, the region of the y line group in which the row forming line is formed is detected as a row C (t), and codes (t = 1 to L) are sequentially applied from the top row as the head of the document image to the L row. Give.

(For vertical document structure)
For example, as in the case of horizontal composition, the document image is formed with a rectangular size of (Sx × Sy), and the coordinate values (x, y) of each pixel are (0, 0) to (Sx). −1, Sy−1) is assumed.

  In such a case, when the document structure of the document image is vertical composition, the y direction becomes the row direction and the x direction becomes the line feed direction. Further, the x line in which the pixels are arranged in the y direction becomes the first array line, and the y line becomes the second array line.

  At this time, as shown in FIG. 9A, the character arrangement analysis processing unit 123 detects a pixel value of each y line and counts black pixels. Further, the character arrangement analysis processing unit 123 determines that the x line is a row forming line when the count is “1” or more, and determines that the x line is a blank line when the count is “0”. Then, as shown in FIG. 9B, the character arrangement analysis processing unit 123 detects the x-line or x-line group region in which the blank line is formed as the inter-row LS, and the row formation line is formed. The x-line or x-line group area is detected as a row C (t), and a code (t = 1 to L) is assigned in order from the rightmost row as the head of the document image to the L row.

[4.3.5] Character Block Detection Processing Next, the character block detection processing in the character layout analysis processing unit 123 of this embodiment will be described with reference to FIGS. 10 to 13 are diagrams for explaining the character section detection processing in the character arrangement analysis processing unit 123 of the present embodiment.

  The character arrangement analysis processing unit 123 performs character detection processing based on the pixel value of each pixel belonging to the first array line and the second array line in the document page being analyzed, and the partition position on the document image. Each character section is detected while specifying the character. In particular, the character arrangement analysis processing unit 123, for each line of the document image detected as the above-described line detection process, from among pixels belonging to each second array line for each second array line belonging to the detected line, Pixels having one value when binarized are detected, and character sections are detected for each line based on the presence or absence of the detected pixels. Note that, similarly to the line detection process, in the character section detection process, in the following description, the character arrangement analysis processing unit 123 will be described using a case where the first pixel (black pixel) is detected.

  Specifically, the character arrangement analysis processing unit 123 counts black pixels (or white pixels) for each second array line that is a pixel line in the document feed direction in the document image for each line, A line in which a character count is “0” is determined to be a line constituting a space between characters in each line with a line having a pixel count of “0” as a blank line, and a character in which a character having a black pixel count of “1” or more is formed It is determined that the line constitutes each character in the document image as a formation line.

  And the character arrangement | positioning analysis process part 123 detects the area | region of the 1 or more line or line group in which the blank line in the 2nd arrangement line is formed for every line as a character space, The character formation line in the 2nd arrangement line is An area of the formed line group is detected as a character section. In particular, the character arrangement analysis processing unit 123 detects coordinates that define each character section.

  At this time, the character arrangement analysis processing unit 123 assigns codes to the detected character sections in order from the first character section of each line.

(For horizontal document structure)
For example, the document image is formed in a rectangular size of (Sx × Sy), and the coordinate values (x, y) of each pixel are (0, 0) to (Sx−1, Sy−1). Assume that they are arranged.

  In such a case, when the document structure of the document image is horizontal composition, the x direction is the row direction and the y direction is the line feed direction. In addition, the y line in which pixels are arranged in the x direction is the first array line, and the x line is the second array line.

  At this time, as shown in FIG. 10A, the character arrangement analysis processing unit 123 detects each pixel value of each x line for each detected row, and counts black pixels. The character arrangement analysis processing unit 123 determines that the x line is a character forming line when the count is “1” or more, and determines the x line as a blank line when the count is “0”. Furthermore, as shown in FIG. 10B, the character arrangement analysis processing unit 123 detects an area of the x line or the x line group in which the character forming line is formed as the character section C (t, m). Then, as shown in FIG. 11, the character arrangement analysis processing unit 123 has four corners that define each character section, that is, upper left coordinates (x1, y1), lower left coordinates (x1, y2), and upper right coordinates (x2, y1). In addition, the lower right coordinates (x2, y2) are detected, and codes are assigned in order from the leftmost character division line to the m division at the head of each line.

(For vertical document structure)
For example, as in the case of horizontal composition, the document image is formed with a rectangular size of (Sx × Sy), and the coordinate values (x, y) of each pixel are (0, 0) to (Sx). −1, Sy−1) is assumed.

  In such a case, when the document structure of the document image is vertical composition, the y direction becomes the row direction and the x direction becomes the line feed direction. Further, the x line in which the pixels are arranged in the y direction becomes the first array line, and the y line becomes the second array line.

  At this time, as shown in FIG. 12A, the character arrangement analysis processing unit 123 detects each pixel value of each y line and counts black pixels for each detected row. The character arrangement analysis processing unit 123 determines that the y line is a character forming line when the count is “1” or more, and determines that the y line is a blank line when the count is “0”. Furthermore, as shown in FIG. 12B, the character arrangement analysis processing unit 123 detects a region of the y line or the y line group in which the character forming line is formed as the character section C (t, m). Then, as shown in FIG. 13, the character arrangement analysis processing unit 123, like horizontal composition, has four corners that define each character section, that is, upper left coordinates (x1, y1), lower left coordinates (x1, y2), and upper right corners. The coordinates (x2, y1) and the lower right coordinates (x2, y2) are detected, and codes are assigned in order from the leftmost character partition line to the m section, which is the head of each line.

[4.3.6] Preprocessing for Correction Processing Next, preprocessing for correction processing in the character arrangement analysis processing unit 123 of this embodiment will be described.

  The character arrangement analysis processing unit 123 performs preprocessing for correction processing of the integrated correction processing and the special character analysis processing of the headline analysis processing, ruby analysis processing, and prohibited character analysis processing, as the maximum division size and average division of character divisions. The size and standard partition size, average line pitch and maximum character pitch, average line pitch, and distance between characters are calculated.

  Note that the character arrangement analysis processing unit 123 executes preprocessing for correction processing for each page determined by the original data format of the acquired document data (that is, the document structure of the imaged document).

(Maximum partition size for each row)
The character arrangement analysis processing unit 123 calculates the maximum section size in the line direction and the line feed direction for each line based on each detected character section.

  Specifically, the character arrangement analysis processing unit 123 calculates, for each line, the difference between the x and y coordinate values in the first array line of each detected character section, and the second of each detected character section. The difference between the x and y coordinate values in the array line is calculated. Then, the character arrangement analysis processing unit 123 calculates, for each line, the maximum value of the difference in the coordinate values in the first array line of each detected character section, and the maximum size Smax (L1 in the line direction) in the document page being analyzed. , T), and the maximum value of the difference between the coordinate values in the second array line of each detected character section is set to the maximum size Smax (L2, t) in the line feed direction in the document page being analyzed. Set.

  For example, in the case of horizontal composition, the character arrangement analysis processing unit 123 performs, for each line, a line direction that is the direction of the first array line of each character section, as shown in (Expression 2) and (Expression 3). The difference between the coordinate values, that is, the width of each character section is calculated, and the calculated maximum width Smax (x, t) of each character section is set to the maximum size Smax (L1, t) in the row direction. In addition, the character arrangement analysis processing unit 123 calculates, for each line, a difference in coordinate values in the line feed direction that is the direction of the second array line of each character section, that is, the height of each character section. The maximum height Smax (y, t) of each character section is set to the maximum size Smax (L2, t) in the line feed direction.

  In the expression, “t” indicates the line number, and “m” indicates the arrangement order from the top in each line. “.X1” or “.x2” indicates the x-coordinates at both ends of the character section, and “.y1” or “.y2” indicates the y-coordinates at both ends of the character section. For example, C (t, m). x2 represents the coordinate value of “x2” in the m-th character section from the beginning of the t-th line in the document page being analyzed.

  On the other hand, in the case of vertical composition, the character arrangement analysis processing unit 123, for each row, as shown in (Expression 4) and (Expression 5), in the row direction that is the direction of the first array line of each character section. The difference between the coordinate values, that is, the height of each character section is calculated, and the calculated maximum height Smax (y, t) of each character section is set to the maximum size Smax (L1, t) in the row direction. In addition, the character arrangement analysis processing unit 123 calculates, for each line, a difference in coordinate values in the line feed direction that is the direction of the second array line of each character section, that is, the width of each character section. The maximum width Smax (x, t) of the character section is set to the maximum size Smax (L2, t) in the line feed direction.

  As in (Expression 2) and (Expression 3), “t” in the expression indicates the line number, and “m” indicates the arrangement order from the top in each line. “.X1” or “.x2” indicates the x-coordinates at both ends of the character section, and “.y1” or “.y2” indicates the y-coordinates at both ends of the character section. For example, C (t, m). x2 represents the coordinate value of “x2” in the m-th character section from the beginning of the t-th line in the document page being analyzed.

(Average partition size based on all rows)
The character arrangement analysis processing unit 123 calculates an average section size in the line direction and the line feed direction based on each character section detected in all lines in the document page being analyzed.

  Specifically, as shown in (Expression 6) and (Expression 7), the character arrangement analysis processing unit 123 adds the maximum size in the line direction and the maximum size in the line feed direction calculated for each line, respectively. By dividing by all lines (that is, T lines), the average section size Sav (L1) in the line direction and the average section size Sav (L2) in the line feed direction in the document page being analyzed are calculated. In the formula, “T” indicates the number of rows.

  For example, in the case of horizontal composition, the character arrangement analysis processing unit 123 sets the calculated maximum width Smax (x, t) of each line to t = 1,. . . , T, and dividing by all rows (ie, T rows), the average partition size Sav (L1) in the row direction is calculated. In addition, the character arrangement analysis processing unit 123 sets the calculated maximum height Smax (y, t) of each line to t = 1,. . . , T, and dividing by all rows (ie, T rows), the average partition size Sav (L2) in the row direction is calculated.

  On the other hand, in the case of vertical composition, the character arrangement analysis processing unit 123 adds the calculated maximum height Smax (y, t) of each line and divides by all lines (that is, T lines). The average partition size Sav (L1) in the row direction is calculated. In addition, the character arrangement analysis processing unit 123 adds the calculated maximum width Smax (x, t) of each line, and divides by all lines (that is, T lines), thereby obtaining the average partition size Sav ( L2) is calculated.

(Standard partition size based on all rows)
The character arrangement analysis processing unit 123 calculates the standard partition size based on all lines using the maximum partition size in the line direction in each line and the average partition size in the line direction based on all lines in the document page being analyzed.

  Specifically, as shown in (Expression 8) and (Expression 9), the character arrangement analysis processing unit 123 calculates a difference value between the maximum partition size in the row direction in each row and the average partition size in the row direction based on all the rows. The difference value between the maximum partition size in the line feed direction and the average partition size in the line feed direction based on all lines is calculated, and the difference value Min | Smax (L1, t) that is the smallest among the calculated difference values. Set the maximum partition size rows ts1 and ts2 with -Sav (L1) | and Min | Smax (L2, t) -Sav (L2) | as standard rows. Then, the character arrangement analysis processing unit 123 calculates the line size Smax (L1, ts1) in the line ts1 set as the standard line and the line feed direction size Smax (L2, ts2) in the line ts2 set as the standard line. The standard section size Sst (L1) in the line direction and the standard section size Sst (L2) in the line feed direction in the document page being analyzed are set.

(Average line pitch based on all lines)
The character arrangement analysis processing unit 123 calculates an average line pitch for each line in the document page being analyzed based on the average coordinate value of the reference position in each character section detected in the line feed direction.

  Specifically, the character arrangement analysis processing unit 123 adds, for each line, the coordinates of the reference point in each character section such as the center coordinates or the coordinates of any of the four corners of the character section, and the number of character sections in the corresponding line. The average coordinate value Pt (av, t) for each row is calculated by dividing by. In addition, the character arrangement analysis processing unit 123 calculates the average of the difference between the average coordinate values in the adjacent lines based on the calculated average coordinate value Pt (av, t) for each line, and analyzes the calculated average. Set as the average line pitch in the middle document page.

  For example, in the case of horizontal composition, the character arrangement analysis processing unit 123, as shown in (Expression 10) and (Expression 11), the average coordinate value Pt (av, t) in the coordinate value y2 calculated for each line. Based on the above, the difference from the average coordinate value Pt (av, t + 1) in the coordinate value y2 with the row adjacent to the subsequent stage is calculated for all rows, and the average is calculated for each of the calculated coordinate value differences. The calculated average is set as the average line pitch SL (av).

  As in the above-described equations, C (t, m). y2 represents the coordinate value of “y2” in the m-th character section from the beginning of the character section t-th line, and “Nc” represents the number of character sections in the t-th line.

  On the other hand, for example, in the case of vertical composition, the character arrangement analysis processing unit 123, as shown in (Equation 12) and (Equation 13), the average coordinate value Pt (av, av) in the coordinate value x1 calculated for each line. t), the difference from the average coordinate value Pt (av, t + 1) in the coordinate value x1 with the row adjacent to the subsequent stage is calculated for all rows, and the average is calculated for each of the calculated coordinate value differences. The calculated average is set as the average line pitch SL (av).

Note that, similarly to (Expression 10) and (Expression 11), C (t, m). x1 indicates the coordinate value of “x1” in the m-th character section from the beginning of the t-th character section, and “Nc” indicates the number of character sections in the t-th line.

(Maximum character pitch of each line)
The character arrangement analysis processing unit 123 calculates, as the maximum character pitch, the maximum arrangement pitch of the character sections in the row direction for each line based on each detected character section in the document page being analyzed.

  Specifically, the character arrangement analysis processing unit 123 calculates, for each line, a difference between coordinate values at the same coordinate position of two adjacent character sections in the first array line of each detected character section. The maximum value of the coordinate value difference is set to the maximum character pitch Spmax (L1, t) in the line direction in the document page being analyzed.

  For example, in the case of horizontal composition, the character arrangement analysis processing unit 123 calculates the difference between the coordinate values of the upper left coordinate position x1 of two adjacent character sections in each row, as shown in (Equation 14). The maximum difference between the calculated coordinate values is set to the maximum character pitch Spmax (L1, t) in each line direction.

  As in the above formulas, “t” in the formula is the row number, “m” is the order of arrangement from the top in each row, and “.x1” is the x coordinate. For example, C (t , M). x1 represents the coordinate value of “x1” in the m-th character section from the beginning of the t-th line.

  On the other hand, for example, in the case of vertical composition, the character arrangement analysis processing unit 123 calculates the difference between the coordinate values of the coordinate position y2 at the right corner of two adjacent character sections in each row, as shown in (Equation 15). The maximum difference between the calculated coordinate values is set as the maximum character pitch Spmax (L1, t) in each line direction.

  As in the above-described expressions, “t” in the expression indicates the line number, “m” indicates the order of arrangement from the top in each line, and “.y2” indicates the y coordinate. For example, (L1, 1) indicates the first array line in the t row, and C (t, m). y2 represents the coordinate value of “y2” in the m-th character section from the beginning of the t-th row.

(Reference character pitch based on all lines)
The character layout analysis processing unit 123 uses the maximum partition size in the line direction in each line, the average partition size in the line direction based on all the lines, and the maximum character pitch of each line in the document page being analyzed, Is calculated.

  Specifically, as shown in (Equation 16), the character arrangement analysis processing unit 123 performs the maximum partition size Smax (L1, t) in the row direction in each row and the average partition size Sst (L1 in the row direction based on all rows). ) And the maximum partition size row having the smallest difference value among the calculated difference values is set as a standard row (Lst). Then, the character arrangement analysis processing unit 123 sets the maximum character pitch Spmax (L1, Lst) in the line set as the standard line (Lst) as the reference character pitch Sp (st) in the document page being analyzed.

As in the above formulas, “t” in the formula is the row number, and “Lst” is the maximum partition size Smax (L1, t) in the row direction in each row, as shown in (Formula 17). The row having the minimum difference value Min _t | ΔSp (L1, t) | of the average partition size Sst (L1) in the row direction based on all rows is shown.

[4.3.7] Character block integration correction processing
Next, the integrated correction processing in the character arrangement analysis processing unit 123 of this embodiment will be described with reference to FIG. FIG. 14 is a diagram for explaining the integrated correction processing in the character arrangement analysis processing unit 123 of the present embodiment.

The character arrangement analysis processing unit 123 calculates the distance between adjacent character sections for each line in the document page being analyzed as the character section integration correction processing, and also calculates the distance between the calculated character sections. Are each equal to or smaller than the reference character pitch calculated by the pre-processing of the correction processing. Then, the character arrangement analysis processing unit 123 connects the character sections determined to have a distance between the character sections equal to or less than the reference character pitch, and integrates them into one character section.

  Normally, for characters composed of two or more mutually independent parts, a space is formed between individual parts such as “hen” and “make” in the kanji, so that character lines are defined by blank lines as described above. In some cases, each part may be recognized as a separate character segment depending on the size of the pixel or the shape of the character. On the other hand, the space between these individual parts is narrower than the space formed between the characters.

  Therefore, in the present embodiment, two adjacent spaces having a space equal to or less than a reference value (that is, a reference character pitch) are specified so that a character composed of two or more independent parts is specified as one character section. One character section is integrated into a single character section.

  Specifically, as shown in (Expression 18) and (Expression 19), the character arrangement analysis processing unit 123 performs two adjacent character sections C (t, m) in the line direction for each line (t). It is determined whether or not the distance D (L1 (m, m + 1)) between C (t, m + 1) is equal to or smaller than the reference character pitch Sp (st) calculated by the preprocessing of the correction processing. Then, the character arrangement analysis processing unit 123 connects the character sections determined to have a distance between the character sections equal to or less than the reference character pitch Sp (st), and integrates them into one character section.

  Note that (Equation 18) shows a determination formula in the case of horizontal composition, and (Equation 19) shows a determination expression in the case of vertical composition. In the formula, “t” indicates a line number, “m” indicates the order of arrangement from the top in each line, and “.x” or “.y” indicates coordinates.

  For example, in the case of a horizontal document, if only line detection processing is performed, “ha” on the first line and “i” on the second line are detected as two character sections. become. Therefore, when the integrated correction process in the present embodiment is executed, “ha” and “i” are integrated as one character section as shown in FIG.

  Also, in the case of a vertically written document, as in the case of a horizontally written document, if the line detection process is simply performed, “ha” on the first line and “i” on the second line It will be detected as two character sections. Therefore, when the integrated correction process according to the present embodiment is executed, “ha” and “i” are integrated as one character section as shown in FIG.

[4.3.8] Headline Analysis Processing Next, the headline analysis processing in the character arrangement analysis processing unit 123 of this embodiment will be described with reference to FIG. FIG. 15 is a diagram for explaining the headline analysis processing in the character arrangement analysis processing unit 123 of the present embodiment.

  As the headline analysis processing, the character arrangement analysis processing unit 123 performs a predetermined condition (hereinafter referred to as “line pitch”) based on the average line pitch calculated by the preprocessing for correction processing for each line in the document page being analyzed. It is determined whether or not it has a “condition”), and for each character section belonging to the line having the line pitch condition, an attribute indicating the character section used for the heading is set as the heading attribute information.

  Specifically, as shown in FIG. 15 and (Equation 20) and (Equation 21), the character arrangement analysis processing unit 123 performs a predetermined coefficient multiple of the average line pitch SL (av) calculated by the pitch of each line (for example, It is determined whether it is larger than 1.5 times. When the character arrangement analysis processing unit 123 determines that each line is larger than a predetermined coefficient multiple (for example, 1.5 times) of the calculated average line pitch SL (av), the character arrangement analysis processing unit 123 finds the line used for the determination. A line is set, and heading attribute information (mode = 1) is set for each character section belonging to the line.

  In addition, (Formula 20) shows the judgment formula in the case of horizontal composition, and (Formula 21) shows the judgment formula in the case of vertical composition. Similarly to the above, “t” in the formula indicates the line number, “m” indicates the order of arrangement from the top in each line, and “.x2” or “.y2” indicates the x or y coordinate. Further, when the line t is determined as the heading line, all the character sections belonging to the line are used for the heading. Therefore, in the above calculation, the line pitch for any m and m + 1 character sections is calculated. It only has to be done. However, the character arrangement analysis processing unit 123 may calculate the line pitch of each line by averaging or other operations using a plurality of or all character sections on the same line.

[4.3.9] Indentation Analysis Processing Next, indentation analysis processing in the character arrangement analysis processing unit 123 of this embodiment will be described with reference to FIG. FIG. 16 is a diagram for explaining the indentation analysis processing in the character arrangement analysis processing unit 123 of the present embodiment.

  As the indentation analysis process, the character arrangement analysis processing unit 123 has a standard section size in the first array line in which the position in the first character section is calculated by preprocessing for correction processing for each line in the document page being analyzed. It is determined whether or not a predetermined condition (hereinafter referred to as “first arrangement condition”) is satisfied, and a character section having the first arrangement condition is a character section corresponding to an indented character. Is set as indentation attribute information.

  Specifically, as shown in FIG. 16 and (Equation 22) and (Equation 23), the character arrangement analysis processing unit 123 sets a predetermined character section in the row direction from the character description start position in the row direction. It is determined whether or not it is more than a distance away. When the character arrangement analysis processing unit 123 determines that the character section that is the head in the line direction is more than a predetermined distance from the start position where the character in the line direction is described, the character used for the determination Indentation attribute information (mode = 2) is set in the section.

  In addition, (Formula 22) shows the determination formula in the case of horizontal composition, and (Formula 23) shows the determination formula in the case of vertical composition. In the formula, “t” indicates a line number, “1” indicates a leading character section in each line, and “.x” indicates coordinates. “Xmin” indicates the x coordinate of the left character array start position in the document structure of the imaged document in the case of horizontal composition, and “Ymax” indicates the document of the imaged document in the case of vertical composition. The x coordinate of the character array start position at the uppermost end in the structure is shown.

[4.3.10] Ruby Analysis Processing Next, the ruby analysis processing in the character arrangement analysis processing unit 123 of this embodiment will be described with reference to FIG. FIG. 17 is a diagram for explaining the ruby analysis processing in the character arrangement analysis processing unit 123 of the present embodiment.

  As the ruby analysis processing, the character arrangement analysis processing unit 123 performs a predetermined condition (hereinafter referred to as “line pitch”) for each line in the document page being analyzed based on the average line pitch calculated by the preprocessing for correction processing. It is determined whether or not “specific condition” is satisfied, and for each character section belonging to the line having the line pitch specifying condition, an attribute indicating that the character section is used for ruby is set as ruby attribute information. .

  In addition, the character arrangement analysis processing unit 123 executes each character in the ruby target line that is the next line with respect to the first position in the line direction of each character section belonging to the specified ruby line and the line feed direction of the ruby line document. The second position in the row direction of each section is compared, and the character section of the ruby target line that minimizes the difference between the first position and the second position is specified as the ruby target character, and the character section of the ruby target character is determined. An attribute indicating a ruby target character is set as ruby target attribute information. Then, the character arrangement analysis processing unit 123 also sets information on the search position (line and arrangement order of the character sections to be ruby) for the target ruby character sections.

  Specifically, as shown in FIG. 17 and (Equation 24) and (Equation 25), the character arrangement analysis processing unit 123 performs the standard of the line feed direction (second array line direction) calculated by the pitch of each line. It is determined whether or not it is smaller than a predetermined coefficient multiple (for example, 0.8 times) of the partition size Sst (L2). If the character arrangement analysis processing unit 123 determines that each line is smaller than a predetermined coefficient multiple of the calculated standard section size Sst (L2), the line used for the determination is set as a ruby line, and the line Ruby attribute information (mode = 3) is set for each character section (C (t, m)) belonging to.

  Note that (Equation 24) shows a determination formula in the case of horizontal composition, and (Equation 25) shows a determination expression in the case of vertical composition. In the formula, “t” is the line number, “m” is the order of arrangement from the top in each line, and “.x1” and “.x2” or “.y1” and “.y2” are x or The y coordinate is shown. Further, when the line t is determined to be a ruby line, all the character sections belonging to the line are used for ruby, and therefore the line pitch for any m character sections is calculated in the above calculation. That's fine. However, the character arrangement analysis processing unit 123 may calculate the line pitch of each line by averaging or other operations using a plurality of or all character sections on the same line.

  Further, as shown in (Condition 1) and (Condition 2), the character arrangement analysis processing unit 123 sets the first position in the line direction of each character section belonging to the specified ruby line and the line feed direction of the document in the ruby line. On the other hand, the second position in the line direction of each character section in the ruby target line, which is the next line, is compared with each other, and the character section of the ruby target line having the smallest difference between the first position and the second position is determined as the ruby. Specify as the target character. Then, the character arrangement analysis processing unit 123 sets ruby target attribute information (mode = 4) in the character section (C (t + 1, i)) specified as the specified ruby target character, and at the same time, sets the ruby target character's ruby. The ID (C (t + 1, i)) of the character section of the ruby target attribute information is set to the character section to which the ruby attribute information is given.

  Note that (Condition 1) indicates the determination condition for horizontal composition, and (Condition 2) indicates the determination condition for vertical composition. “T” indicates the line number, “i” and “m” indicate the order of arrangement from the top in each line, and “.x1” or “.y2” indicates the coordinates.

[4.3.11] Forbidden Character Analysis Processing Next, forbidden character processing in the character arrangement analysis processing unit 123 of this embodiment will be described with reference to FIG. FIG. 18 is a diagram for explaining prohibited character analysis processing in the character arrangement analysis processing unit 123 of this embodiment.

  As the prohibited character analysis processing, the character arrangement analysis processing unit 123 sets each character partition for each character partition in which each attribute information of the headline analysis processing, the indent analysis processing, and the ruby analysis processing is not set in the document page being analyzed. Maximum width size in the row direction (first array line direction) (maximum width where black pixels exist in the character section) and minimum width size (minimum width where black pixels exist in the character section), and line feed The maximum width size and the minimum width size in the direction (second array line direction) are calculated. Then, the character arrangement analysis processing unit 123 determines whether or not each of the calculated differences satisfies a predetermined condition (hereinafter referred to as “prohibited character condition”), and the character section having the prohibited character condition. For, an attribute indicating a character section corresponding to a prohibited character is set as prohibited character attribute information.

Specifically, as shown in FIG. 18 and (Equation 26) and (Equation 27), the character arrangement analysis processing unit 123 performs the determination in the range of the character segment to be determined.
(1) Maximum width size Zmax (L1) and minimum width size Zmin (L1) in the row direction (first array line direction), maximum width size Zmax (L2) and minimum width size Zmin (L2) in the line feed direction, and , And
(2) The difference between the maximum width size Zmax (L1) in the row direction and the minimum width size Zmin (L1) (hereinafter referred to as “row direction size difference”) ΔZ (L1) and the maximum width size Zmax (in the line feed direction) L2) and the difference between the minimum width size Zmin (L2) (hereinafter referred to as “line feed direction size difference”) ΔZ (L2), respectively,
(3) Whether each difference is equal to or less than a predetermined coefficient multiple (for example, 0.35 times) of the standard partition sizes Sst (L1) and Sst (L2) in which the row direction and the line feed direction are prohibited character conditions Determine whether or not
(4) When the prohibition condition is satisfied, prohibition character attribute information (mode = 5) is set in the character section used for the determination.

  Note that (Expression 26) and (Expression 27) are “Zmax (L1)” and “Zmin (L1)” in the case of a horizontal determination formula, and become “x coordinates”, and “Zmax (L2)” and “ “Zmin (L2)” is the y coordinate. In addition, (Expression 26) and (Expression 27) are “Zmax (L1)” and “Zmin (L1)” in the case of a vertical determination formula, and become the y-coordinate, and “Zmax (L2)” and “ “Zmin (L2)” is the x coordinate.

[4.4] Partition Arrangement Processing Unit Next, the partition arrangement processing unit 124 in the application processing unit 120 of this embodiment will be described with reference to FIGS. 19 and 20. 19 and 20 are diagrams for explaining the partition arrangement processing in the partition arrangement processing unit 124 of this embodiment.

  For each page (that is, document page) in the document structure of the acquired document data, the section arrangement processing unit 124 continues the arrangement of the character sections of the previous document page, and analyzes each of the characters analyzed by the character arrangement analysis processing unit. For arranging the character sections in the designated display area according to the set area size of the designated display area, the predetermined arrangement condition, and the arrangement order of each character section (order of arrangement for each page) Execute the process.

  Specifically, the section arrangement processing unit 124 arranges each character section according to the line number of the character section, the arrangement order C (t, m) for each line, and the document structure (that is, horizontal composition or vertical composition). When a character section having each attribute information indicating a heading character, an indented character, ruby, a ruby target character, and a prohibited character is detected, the character section having each attribute information is set at a predetermined position according to a predetermined arrangement condition. Deploy.

That is, the partition arrangement processing unit 124
(1) Arrange character sections that do not have attribute information according to a predetermined arrangement order;
(2) When a character section having heading attribute information is detected, the character section is arranged in a designated display area set as a single line together with other character sections belonging to the heading line;
(3) When a character section having indentation attribute information is detected, the character section is placed at the position where the indentation is at the beginning of the line within the set designated display area,
(4) When a character section having ruby attribute information is detected, the character section is arranged at the ruby position of the ruby target character to which the ruby is given within the set designated display area,
(5) When a character section having prohibited character attribute information is detected, the character section corresponding to the prohibited character at the beginning of the line is the end of the preceding line, or the character section corresponding to the prohibited character at the end of the line. Is placed at the beginning of the next line.

  Then, the section arrangement processing unit 124 detects the coordinates of each character section when it is arranged (hereinafter referred to as “modified character section coordinates”), and sets the arrangement data having the coordinates of the four corners of the character section for each character section. Generate.

  For example, as shown in FIG. 19 (in the case of horizontal composition) and FIG. 20 (in the case of vertical composition), the section arrangement processing unit 124 determines the indented character section, the ruby character section, the ruby target character section, and the prohibited character section. In the designated display area according to the above arrangement conditions.

  In the present embodiment, the section arrangement processing unit 124 performs the arrangement of prohibited characters at the end of the line, and therefore, when executing the section arrangement process, the designated display area is used as shown in FIGS. 19 and 20. When placement processing is performed using a narrow placement area in the row direction (hereinafter referred to as “actual placement area”), and there is a character block of prohibited characters, it is in the designated display area and the previous character block of the prohibited character Are arranged on the same line. However, the actual arrangement area may be an area that is narrower than the designated display area by a space in which the maximum prohibited character section can be arranged.

[4.5] Image Data Generation Unit Next, the image data generation unit 125 in the application processing unit 120 of this embodiment will be described with reference to FIG. FIG. 21 is a diagram for explaining image generation processing in the image data generation unit 125 of the present embodiment. However, FIG. 21 is a diagram for explaining a case of generating a bitmap image in the case of horizontal composition.

  The image data generation unit 125 generates a bitmap image based on the arrangement data generated by the division arrangement processing unit 124, that is, each character division arranged in the designated display area.

Specifically, as shown in FIG. 21 , the image data generation unit 125, as shown in FIG. 21 , corresponds to the section image corresponding to each character section C (t, m), that is, each character surrounded by the coordinates on the document data. The pixel block (pixel value of each pixel) of the section C (t, m) is extracted, and the pixel block is extracted from the arrangement data generated by the section arrangement processing unit 124 at the four corners of each character section C (t, m). Assigned to coordinates (that is, modified character section coordinates, specifically, (x1, y1), (x2, y1), (x1, y2), and (x2, y2)) in the instruction designation area Each pixel is set to generate a bitmap image. In particular, the image data generation unit 125 sets each pixel value of RGB for a color image, and a gray scale value for each pixel of the modified character segment coordinates for a monochrome image.

[5] Display Processing Operation [5.1] Browsing Application Main Display Processing Next, the operation of the document image display processing based on the browsing application in the portable terminal device 10 of the present embodiment will be described with reference to FIG. . In addition, it is a flowchart which shows the operation | movement of the display process of the document image based on the browsing application in the portable terminal device 10 of this embodiment using FIG.

  In this operation, it is assumed that the document data of the imaged document image is already stored in the data storage unit 100, and the document image of the document data is displayed in the designated display area having a display size different from the page size of the document data. Is displayed.

  In addition, user operations that can be used while viewing a document image in predetermined document data include an instruction to change a designated display area, an instruction to change a display position by scrolling, and an end instruction.

  First, when the data management control unit 121 detects a user's instruction to start execution of a browsing application together with selection of document data that the user desires to browse through the operation unit 160 (step S101), the data management control unit 121 converts the selected document data into content data. The data is read from the storage unit 102 and expanded in the ROM / RAM 103 (step S102).

  Next, the character arrangement analysis processing unit 123 performs character segment analysis processing for analyzing the character segment of each character included in the document image of the read document data for each document page. Character segment data composed of attribute information such as headings and the like (step S103). Details of the character segment analysis processing in the character arrangement analysis processing unit 123 will be described later.

  Next, the designated display area setting unit 122 reads a predetermined standard designated display area (step S104). For example, the data management control unit 121 stores the designated display area at the time of the previous operation of the browsing application in the data storage unit 100 and reads out from the data storage unit 100 in this process.

  Next, the section arrangement processing unit 124 reads each character section analyzed by the character section analysis process executed in step S103 according to a predetermined arrangement state and attribute information, or the designated display area read in the process of step S104, or A partition placement process is performed for placement in the designated display area set in the process of step S109 (step S105). Details of the partition arrangement processing in the partition arrangement processing unit 124 will be described later.

  Next, the image data generating unit 125 arranges the pixel block in the original document image corresponding to the arranged character section in the character section in the arranged designated display area, and generates a bitmap image (step S106). .

  Next, the image data generation unit 125 outputs the corresponding portion of the bitmap image to the display unit 150 in conjunction with the display control, and waits for a user operation input (step S107). When a scroll instruction is issued, the corresponding portion of the bitmap image is continuously output in conjunction with the instruction.

  Next, when detecting an operation input by the user (step S108), the data management control unit 121 determines whether or not there is a display change instruction by scrolling (step S109) and whether or not there is an instruction to change the designated display area (step S110). To do.

  At this time, if the data management control unit 121 determines that the operation input by the user is an instruction to change the designated display area, the data management control unit 121 proceeds to the process of step S107, and determines that the operation input by the user is the designation of the modified page. When the process proceeds to step S105 and it is determined that the operation input by the user is not any operation input, a termination process of the browsing application is executed (step S111), and this operation is terminated.

[5.2] Character Arrangement Analysis Processing Next, the operation of the character arrangement analysis processing executed during the execution of the browsing application in the portable terminal device 10 of the present embodiment will be described using FIG. FIG. 23 is a flowchart showing the operation of character arrangement analysis processing during execution of the browsing application in the present embodiment.

  In this operation, it is assumed that the page layout of document data, that is, whether it is horizontal composition or vertical composition, is included in the document data as flag information. In this operation, when the execution of the character section analysis process is detected, the following process is executed for each document page.

  First, when the character layout analysis processing unit 123 detects the execution of the character segment analysis process in the process of step S103 described above (step S201), based on the flag information included in the document data stored in the ROM / RAM 103, The page layout of the document data is identified, and the line direction and line feed direction of the document in the document data are recognized (step S202).

  Next, the character arrangement analysis processing unit 123 converts the document image formed in color into a grayscale image, and the converted grayscale image or the document image formed in black and white is directly determined in advance. Based on the threshold value, binarization is performed, and noise correction is performed to remove noise in the binarized document image (step S203).

  Next, the character arrangement analysis processing unit 123 performs the line detection processing and the character segment detection processing, so that each pixel value (black pixel, white pixel) of each pixel binarized with respect to the recognized line direction and line feed direction is detected. (Pixel or both) is counted (step S204).

  Next, the character arrangement analysis processing unit 123 extracts a blank pixel line (blank line) in the row direction based on the count value of the pixel value in each pixel in the row direction, and the region width (that is, the row height or Each line including (width) is detected (step S205).

  Next, the character arrangement analysis processing unit 123, for each detected line, based on the count value of the pixel value in each pixel in the line feed direction, a blank pixel line (blank line) in the line feed direction and the characters that form the character A formation line is extracted, and each character section for each line is detected (step S206).

  Next, the character arrangement analysis processing unit 123 calculates a maximum partition size, an average partition size, a standard partition size, an average line pitch, a maximum character pitch, and a reference character pitch as preprocessing for correction processing (step S207). .

  Next, the character arrangement analysis processing unit 123 executes an integrated correction process for integrating two adjacent character sections having the character pitch condition in each row using the calculation result executed by the preprocessing (step S208). .

  Next, the character arrangement analysis processing unit 123 determines whether each line has a line pitch condition using the calculation result executed by the preprocessing, and belongs to the line having the line pitch condition. A headline analysis process for setting an attribute (headline attribute information) as a headline character in the character section is executed (step S209).

  Next, the character arrangement analysis processing unit 123 determines whether or not the first character section of each line has the first arrangement condition by using the calculation result executed by the preprocessing, and the first arrangement having the first arrangement condition. An indentation analysis process for setting an attribute (indentation attribute information) as an indented character in the character section is executed (step S210).

  Next, the character arrangement analysis processing unit 123 determines whether or not each line has a line pitch specifying condition using the calculation result executed by the preprocessing, and determines a character section having the line pitch specifying condition or the like as a ruby. Alternatively, it is set as a ruby target character (step S211).

  Specifically, the character arrangement analysis processing unit 123 sets an attribute (ruby attribute information) as ruby for each character section having the line pitch specifying condition, and determines that it is ruby when it is determined to be ruby and there is a character section. Characters that have a predetermined condition for the specified character section are set as the ruby target character attribute (ruby target character attribute information), and the character set as the ruby target character in the character section set to ruby A search position for searching for a partition is set.

  Next, the character arrangement analysis processing unit 123 uses the calculation result executed by the preprocessing, and forbids each character section for which heading attribute information, indentation attribute information, ruby attribute information, and ruby target attribute information are not set. It is determined whether or not the condition is satisfied, and forbidden character analysis processing for setting an attribute (forbidden character attribute information) as a forbidden character in the character section having the forbidden condition is executed (step S212).

  Finally, the character arrangement analysis processing unit 123, for each character segment, the reference coordinates (specifically, the coordinates of the four corners of the rectangle) of the character segment obtained by the above processes, the arrangement order information from the head, and the heading The character section data including the attribute information such as the above is generated and expanded in the ROM / RAM 103 (step S213), and this operation is terminated.

[5.3] Division Arrangement Processing Next, the operation of the division arrangement processing executed during the execution of the browsing application in the portable terminal device 10 of the present embodiment will be described with reference to FIGS. 24 to 26 are flowcharts showing the operation of the partition arrangement process during the execution of the browsing application in the present embodiment.

  This operation is executed for each page in the document structure of the document data (that is, a page in the original document format (hereinafter simply referred to as “document page”)), and the arrangement condition is ROM at the start of execution of the browsing application. It is assumed that / RAM 103 is expanded.

  In this operation, when a character segment is arranged, the position of coordinates (x, y) (hereinafter referred to as “next arrangement position”) for arranging the next character segment in the line direction is recognized, and It is assumed that the character section is arranged while calculating the remaining space in the line direction for performing a line break in the designated display area.

  First, when the partition placement processing unit 124 detects the execution of the partition placement processing in the processing of step S105 described above (step S300), the partition placement processing unit 124 determines the row direction from the region size of the designated display region set by the user or preset. The length information is acquired and each value used for the calculation of the subsequent process is initialized (step S301). Specifically, the partition arrangement processing unit 124 acquires the number of pixels in the row direction and the line feed direction to acquire the area size (Ax × Ay) of the designated display area, and the actual arrangement area (Bx × By). Set. At this time, the partition arrangement processing unit 124 initializes the length WD (L1) in the row direction, which is stored in the ROM / RAM 103 and used when calculating the remaining space of the designated display area.

  Next, the section arrangement processing unit 124 reads out and acquires the character section data of the character sections belonging to the corresponding document page from the ROM / RAM 103 (step S302). Specifically, the section arrangement processing unit 124 acquires the document page to which the character section displayed at the head of the designated display area belongs when the head document page or the designated display area is set.

  Next, the section arrangement processing unit 124 reads out the corresponding character section in the read document page together with its attribute information (step S304). Specifically, the section arrangement processing unit 124 executes this processing for the first character section or the document page already in accordance with the arrangement order of each character section obtained by the character arrangement analysis process in the read document page. If it is, the character section next to the character section read out last time is read out.

  Next, the section arrangement processing unit 124 determines whether or not the character section is a character section indicating ruby based on the read character section attribute information (step S305). Specifically, the section arrangement processing unit 124 determines whether or not the attribute information of the character section has ruby attribute information “mode = 3”. At this time, if the section arrangement processing unit 124 determines that the read character section attribute information is a character section indicating ruby, the process returns to step S304, and if it is determined that the character section is not ruby. The process proceeds to step S306.

  This process is a determination process for arranging character sections determined to be ruby after arranging character sections other than ruby.

  Next, the section arrangement processing unit 124 determines whether the character section is a character section belonging to the heading line based on the read character section attribute information (step S306). Specifically, the section arrangement processing unit 124 determines whether or not the headline attribute information “mode = 1” is included as the attribute information of the character section. At this time, when the section arrangement processing unit 124 determines that the read character section is a character section belonging to the heading line, the section arrangement processing unit 124 proceeds to the process of step S312 and determines that the character section does not belong to the heading line. In step S307, the process proceeds to step S307.

  Next, the section arrangement processing unit 124 determines whether or not the character section is a character section indicating an indented character based on the read character section attribute information (step S307). Specifically, the section arrangement processing unit 124 determines whether or not the indentation attribute information “mode = 2” is included as the attribute information of the character section. At this time, if the section arrangement processing unit 124 determines that the read character section is an indented character, the section arrangement processing unit 124 proceeds to the process of step S312. If it is determined that the character section is not an indented character, the section arrangement processing unit 124 proceeds to step S308. Transition to processing.

  Next, when the section arrangement processing unit 124 determines that the character section read out in step S307 is not an indented character, the section arrangement processing unit 124 executes a predetermined calculation to read out the character read out in the corresponding line of the designated display area. It is determined whether or not there is a remaining space for arranging the sections (step S308). Specifically, the section arrangement processing unit 124 adds the length WD (L1) in the row direction stored in the ROM / RAM 103 to the length in the row direction of the read character section (Δx and vertical in the case of horizontal writing). In the case of a combination, Δy) is added, and the value is the length in the row direction in the actual arrangement area acquired in the processing of step S302 (Bx in the case of horizontal combination and By in the case of vertical combination). It is determined whether it is smaller.

  At this time, if the section arrangement processing unit 124 determines that there is a remaining space for arranging the read character section in the corresponding line of the designated display area, the process proceeds to the process of step S321, and the read character section When it is determined that there is no remaining space for placing the process, the process proceeds to step S331.

  Next, when it is determined in step S308 that there is no arrangement space for arranging the read character section in the corresponding row of the designated display area, the section arrangement processing unit 124 includes attribute information of the read character section. Based on this, it is determined whether or not the character section indicates a prohibited character (step S311).

  Specifically, the section arrangement processing unit 124 determines whether or not the prohibited character attribute information “mode = 5” is included as the attribute information of the character section. At this time, if the section arrangement processing unit 124 determines that the read character section is not a forbidden character, the section arrangement processing unit 124 proceeds to the process of step S312. If it is determined that the character section is a forbidden character, the section arrangement processing unit 124 proceeds to the process of step S321. Transition.

  Next, if the section arrangement processing unit 124 determines that the character section read in the process of step S306 is a character section belonging to the heading line, the character section read in the process of step S307 is an indented character. If it is determined, or if it is determined that the character section read out in the process of step S311 is a prohibited character, the next arrangement position in the designated display area is broken, and the length WD (L1) in the row direction is set. Initialization (step S312) and the process proceeds to step S321. Specifically, the partition arrangement processing unit 124 determines the next arrangement position at the head position in the row direction while adding a predetermined line spacing value to the next arrangement position to determine the position in the line feed direction.

  Next, when it is determined in the process of step S308 that there is an arrangement space for arranging the read character section in the corresponding line of the designated display area, the section arrangement processing unit 124 reads the character section read in the process of step S312. Is determined to be a forbidden character, or when a line break is made in the processing of step S312 and the next arrangement position is determined at the head position in the row direction, the read character section is arranged at the next arrangement position. (Step S321).

  Next, the partition arrangement processing unit 124 adds the length in the row direction (Δx in the case of horizontal composition and Δy in the case of vertical composition) of the character section read to the next arrangement position in the row direction, The next arrangement position is updated (step S322).

  Next, the section arrangement processing unit 124 determines whether or not the section arrangement processing has already been executed for all the character sections with respect to the document data read by the process of step S302 except for ruby (step S331). At this time, if the partition placement processing unit 124 determines that the partition placement processing has already been executed for all character partitions except for ruby (that is, if it is determined that a character partition other than ruby has been placed), When the process proceeds to step S401 and it is determined that the partition placement processing has not been executed for all character partitions except for the ruby (that is, there is a character partition that has not yet been subjected to the partition placement processing other than ruby). If it is determined that there is, the process returns to step S304.

  On the other hand, if the section arrangement processing unit 124 determines in the process of step S331 that the section arrangement processing has already been executed for all character sections except for ruby, the section arrangement processing unit 124 applies the corresponding document page from the top of the corresponding document page. The character section to be read is read again (step S401).

  Next, the section arrangement processing unit 124 determines whether or not the character section indicates ruby based on the read character section attribute information (step S402). Specifically, the section arrangement processing unit 124 determines whether or not the attribute information of the character section has ruby attribute information “mode = 3”. At this time, if the section arrangement processing unit 124 determines that the read character section attribute information is a character section indicating ruby, the process returns to step 403, and if it is determined that the character section is not ruby, The process returns to step S401.

  Next, the section arrangement processing unit 124 searches for the ruby target character included in the attribute information, calculates the ruby position of the ruby target character (step S403), and arranges the character section as ruby at the corresponding position (step S404). ).

  Next, the section arrangement processing unit 124 determines whether or not the section arrangement processing has been performed for all character sections including ruby (step S405). At this time, if the partition placement processing unit 124 determines that the partition placement processing has not been executed for all the character partitions including the ruby, the processing returns to the process of step S401 and the partition placement for all the character partitions including the ruby is performed. If it is determined that the process has been executed, the process proceeds to step S402.

  Finally, the partition arrangement processing unit 124 determines whether or not there is a next document page (step S406). If it is determined that there is a next document page, the section arrangement processing unit 124 proceeds to the process of step S302, and the next document page. If it is determined that there is no, this operation is terminated.

[6] Example of Display Processing Next, an example of display processing in the present embodiment will be described with reference to FIGS. 27 and 28. 27 is a diagram for explaining an example of display processing in the case of horizontal composition in the present embodiment, and FIG. 28 is for explaining an example of display processing in the case of vertical composition in the present embodiment. FIG.

  In the case of horizontal composition, the portable terminal device 10 activates the browsing application based on user data, for example, based on document data (document page) in a bitmap format such as the JPEG format shown in FIG. Then, character segment data as shown in FIG. 27B is generated. Then, when the portable terminal device 10 detects and sets the designated display area (real arrangement area) as shown in FIG. 28C, the display according to the designated table area as shown in FIG. Output the screen.

  Further, in the case of vertical composition, the portable terminal device 10 selects a browsing application based on user data, for example, based on document data (document page) in a bitmap format such as the JPEG format shown in FIG. When activated, character segment data as shown in FIG. 28B is generated. Then, when the portable terminal device 10 detects and sets a designated display area (actual arrangement area) as shown in FIG. 27C, display according to the designated table area as shown in FIG. Execute.

[7] Modified Example Next, a modified example based on the above embodiment will be described with reference to FIG. In addition, FIG. 29 is a figure for demonstrating the modification based on the said embodiment.

  In the above-described embodiment, the portable terminal device 10 generates a bitmap image with a virtual page for one page up to the end of the document data in the line feed direction without using the concept of pages, and the user According to the scroll display, the portion desired by the user is displayed so as to be seamlessly browseable. However, the document image data may be displayed according to the designated display area using the concept of the page for the designated display area.

  In this case, the section arrangement processing unit 124 arranges each character section in order to provide a document image with a plurality of pages (hereinafter referred to as “modified pages”) according to the set designated display area. It has become. For example, the section arrangement processing unit 124 can secure an area for arranging the character sections on the modified page when all the character sections of one page in the original document image cannot be arranged on one modified page. Each time it becomes impossible, one modified page is added, and character sections that are not yet arranged (hereinafter referred to as “unplaced character sections”) are arranged on the added modified pages according to the arrangement conditions.

  In particular, in this case, as shown in FIG. 29, the section arrangement processing unit 124, after the arrangement of the character sections of the mp page in the document image (mp page in the document page) is finished, When one or more character sections can be arranged on the “page”)), the first character section in the (mp + 1) page of the document page is arranged in the unallocated area of the ongoing modification page in accordance with the arrangement condition, and np A modified page of the page is generated, and a modified page of the (np + 1) th page is generated by a character section that is not arranged on the ongoing modified page.

  Then, the image data generation unit 125 generates data for displaying the document image for each modified page based on the designated display area set by the user.

  As described above, the portable terminal device 10 according to the present embodiment is based on the binarized first array line and the pixel value of each pixel of the second array line, and the image area (section) of each character in the imaged document. Image) can be detected as character sections, so that each character section can be arranged in accordance with the set display area.

  Therefore, the portable terminal device 10 of the present embodiment can realize a displayable character arrangement while restricting scroll display in the row direction when the width of the display region in the row direction is narrow, for example. Even when the display area of a display means such as a portable terminal device is small, the user's viewability can be improved without depending on the size of the display area.

  Further, when detecting the character section, the portable terminal device 10 of the present embodiment can specify the size of each character constituting the imaged document image, the arrangement position on the document, or both. Therefore, each character section can be arranged in the display area based on the size of each character or the original arrangement position.

  Therefore, the portable terminal device 10 according to the present embodiment can realize display with an appropriate character size while maintaining the document structure of the document.

  In addition, the portable terminal device 10 according to the present embodiment is configured such that each first array line is a line indicating a line space of a document or there is a character depending on the presence or absence of the pixel value of the first pixel on each first array line. It is possible to detect whether the line is included in the line to be performed.

  That is, the portable terminal device 10 according to the present embodiment detects, for example, a first pixel having a pixel value indicating black, and when the first pixel does not exist on the first array line, When it is possible to detect that the first array line is a blank line in which no character exists and indicates a line space between documents, and when one or more first pixels are detected on the first array line Can detect that the first array line is a line included in a line in which a character exists.

  And the portable terminal device 10 of this embodiment detects each line (namely, line in which a character exists) and the space | interval between those lines which comprise a document by the adjacent 1st arrangement line group in which a 1st pixel exists. Therefore, based on the number of first array lines belonging to the first array line group, the size of the characters constituting the document in the line feed direction, that is, the height of the characters if the document is horizontal composition, the document is vertical composition If so, the width of the character can be detected.

  As a result, the portable terminal device 10 of the present embodiment can accurately detect the image area (division image) of each character in the imaged document as a character division.

  Further, the portable terminal device 10 of the present embodiment is a line in which each second array line indicates a space between characters depending on the presence or absence of the pixel value of the first pixel on each second array line for each row of the document. Or a line constituting a character can be detected.

  That is, the portable terminal device 10 according to the present embodiment detects, for example, the first pixel having a pixel value indicating black as described above, and the first pixel does not exist on the second array line. In this case, it can be detected that the second array line is a blank line in which no character exists and indicates a line between characters, and one or more first pixels are detected on the second array line. In such a case, the second array line can be detected as a line constituting a character.

  And since the portable terminal device 10 of this embodiment can detect each character and between those characters by the adjacent 2nd arrangement line group in which a 1st pixel exists, it is in the said 2nd arrangement line group. Based on the number of second array lines to which it belongs, it is possible to detect the size in the line direction of each character of each line, that is, the character width if the document is horizontal composition and the character height if the document is vertical composition.

  As a result, the portable terminal device 10 of the present embodiment can accurately detect the image area (division image) of each character in the imaged document as a character division.

  Moreover, since the portable terminal device 10 of this embodiment can specify the character comprised from two or more mutually independent parts as a character division of one character, the image area | region of each character in the imaged document ( (Division image) can be accurately detected as a character division.

  In addition, the portable terminal device 10 according to the present embodiment identifies the size of each character constituting the imaged document image, the arrangement position on the document, or both, and recognizes the arrangement rule of each character in the document. In addition, by reflecting the arrangement rule in the arrangement condition, the position in the display area of the character section indicating each character section can be determined while following the arrangement rule.

  Therefore, the portable terminal device 10 of the present embodiment can realize display with a suitable character size while maintaining the document structure such as heading, indentation, first character, ruby, or forbidden character. The user's viewability can be improved without depending on the display area.

  Moreover, since the portable terminal device 10 of this embodiment can arrange | position each character section which belongs to a heading line to the single line in a display area, it suits a display area with a document maintained. Can be displayed.

  Moreover, the portable terminal device 10 of this embodiment can arrange the character section detected as the ruby of the predetermined character at the position of the ruby of the ruby target character to which the ruby is given in the set display area. Therefore, the document can be displayed in accordance with the display area while maintaining the document structure.

  In addition, the portable terminal device 10 according to the present embodiment can arrange the character section identified as the first character of the paragraph in the document at the position where it is indented at the beginning of the line within the set display area. Therefore, the document can be displayed in accordance with the display area while maintaining the document structure.

  In addition, since the portable terminal device 10 according to the present embodiment can arrange the character section specified as the target character of the prohibition process in the document as the target character of the prohibition process even in the set display area, The document can be displayed in accordance with the display area while maintaining the structure.

  In addition, since the portable terminal device 10 according to the present embodiment can display a document in the display area without causing the user to be aware that the page has been switched even when the page of the document has been switched. The user's viewability can be improved.

DESCRIPTION OF SYMBOLS 10 ... Portable terminal device 100 ... Data storage part 101 ... Application storage part 102 ... Content data storage part 103 ... ROM / RAM
DESCRIPTION OF SYMBOLS 110 ... Communication control part 120 ... Application processing part 121 ... Data management control part 122 ... Designated display area setting part 123 ... Character arrangement | positioning analysis processing part 124 ... Section arrangement | positioning processing part 125 ... Image data generation part 140 ... Display control part 150 ... Display Unit 160 ... Operation unit 190 ... Terminal management control unit

Claims (10)

A display control device for displaying an image formed by a plurality of pixels arranged in a matrix on a display means,
Acquisition means for acquiring image data for displaying at least the image in which the document is imaged as the document image on the display means;
Setting means for setting a size of a display area in the display means when displaying the document image;
Recognition means for recognizing the line direction and line feed direction of the document in the document image based on the acquired image data;
Binarization means for binarizing each pixel value of the document image;
Row detection means for detecting a row of the document image based on a pixel value in each of the binarized pixels for each first array line that is an array line of pixels in the document row direction of the document image;
For each detected line, one of the pixels belonging to each second array line for each second array line, which is an array line of pixels in the document feed direction of the document image, is binarized. A character segment that detects at least a first pixel having a value of and detects a segment of a character included in each line as a character segment based on the presence or absence of the first pixel while specifying the size and segment position of the segment Detection means;
(1) Based on each character section of each detected line, a maximum section size indicating the size of the largest character section in the line direction in each line, an average section size indicating an average of the size of the character sections based on all lines, And calculating a maximum character pitch indicating a maximum value of a distance between two adjacent character sections in each line , and (2) a difference between the maximum section size in each line and the average section size based on all the lines is minimum. The maximum character pitch in a given line is calculated as a reference character pitch, and (3) each line belongs to the same line and is adjacent based on the detection result of the first pixel when the character section is detected A character pitch indicating an arrangement interval in the line direction of the document in two character sections is calculated. (4) The calculated character pitch is a character pitch condition determined in advance based on the calculated reference character pitch. And integration correction means performs integration correction two character compartments of calculation target of the character pitch as the same character compartments having a matter,
An arrangement position determining means for determining an arrangement position for arranging the corrected character sections in the display area based on the set area size of the display area;
Image generating means for generating a display image to be displayed in the display area by disposing a section image corresponding to each character section at a position where each character section is determined; ,
Output means for outputting the generated image to the display means;
A display control apparatus comprising: The display control device according to claim 1 ,
The row detection means is
Detecting at least a first pixel having one pixel value when binarized from pixels belonging to each first array line for each first array line;
A display control device that detects each line of the imaged document based on the presence or absence of the first pixel. In the display control device according to claim 1 or 2 ,
The arrangement position determining means is
A display control device that determines a position at which the detected character segment is arranged in the set display area in accordance with a predetermined arrangement condition based on the size and the segment position of each detected character segment. In the display control device according to any one of claims 1 to 3 ,
The arrangement condition is a condition in which a character section belonging to a heading line indicating a headline and a line to be used in the document is arranged as a single line in the set display area together with other character sections belonging to the heading line. Including
The character section detecting means is
Based on the section position in the document image of each character section of each line including the integrated character section, for each line, the arrangement interval between two adjacent lines in the line feed direction of the document is calculated as a line pitch,
Based on each calculated line pitch, an average line pitch averaged over all lines of the document image is calculated,
When the calculated row pitch has a row pitch condition determined based on the average row pitch, one of the two rows having the row pitch condition is determined based on a base point that determines the row pitch. Identify the line as the heading line,
The arrangement position determining means is
A display control apparatus that arranges character sections belonging to the heading line in the display area according to the arrangement condition. In the display control device according to any one of claims 1 to 3 ,
The arrangement condition includes a condition for arranging a character section detected as ruby of a predetermined character at a ruby position of a ruby target character to which the ruby is given in the set display area,
The character section detecting means is
Based on the section position in the document image of each character section of each line including the integrated character section, for each line, the arrangement interval between two adjacent lines in the line feed direction of the document is calculated as a line pitch,
Based on each calculated line pitch, an average line pitch averaged over all lines of the document image is calculated,
In the case where the calculated pitch width of the row pitch has a row pitch specifying condition determined based on the average row pitch, of the two rows having the row pitch specifying condition, a base point for determining the row pitch A line determined based on the ruby is specified as a ruby line that gives the ruby,
A first position in the line direction of each character section belonging to the specified ruby line and a second position in the line direction of each character section in the ruby target line, which is the next line to the line feed direction of the document in the ruby line. Each position is compared, and the character section of the ruby target line that minimizes the difference between the first position and the second position is specified as the ruby target character,
The arrangement position determining means is
A display control device that arranges each character section belonging to the ruby line at a position in the display area according to the arrangement condition based on a character section position in the display area of the ruby target character. In the display control device according to any one of claims 1 to 3 ,
The arrangement condition includes a condition for arranging the character section identified as the first character of the paragraph in the document at the position at which the first character of the line is indented in the set display area,
The character section detecting means is
For each line, the arrangement position of the character section located at the head of the line is separated from the start position where the characters in the line direction are described by a predetermined distance or more determined by the head arrangement condition based on the size of each character section. when you are to identify the beginning of a character section in the row as the first character in a paragraph of the document,
The arrangement position determining means is
A display control device that arranges a character section identified as the first character in the display area according to the arrangement condition. In the display control device according to any one of claims 1 to 3 ,
The arrangement condition includes a condition for arranging a character section corresponding to the prohibited characters in the document at a predetermined position in the set display area,
The character section detecting means is
In each of the detected line direction and line feed direction of each character section, calculate the maximum width and minimum width in which the first pixel in each character section exists,
Calculating the difference between the maximum width and the minimum width in the line direction of each detected character section, and the difference between the maximum width and the minimum width in the line feed direction of each detected character section;
When each calculated difference has a predetermined condition, the corresponding character section is set to the character section corresponding to the prohibited character,
The arrangement position determining means is
A display control device that arranges a character section corresponding to the prohibited character and a set character section in the display area according to the arrangement condition . In the display control device according to any one of claims 1 to 7 ,
The acquisition unit acquires the image data from the external or storage unit for each page of the imaged document;
The arrangement position determining means arranges the character section corresponding to the first character of each page in the set display area as the character section next to the character section corresponding to the last character of the previous page while following the arrangement condition. A display control device that determines a position to perform. In the display control device according to any one of claims 1 to 8 ,
The recognition means
Detecting the number of blank lines in which the character in the second array line direction does not exist based on the pixel value of each pixel belonging to each first array line;
Detecting the number of blank lines in which the character in the first array line direction does not exist based on the pixel value of each pixel belonging to each second array line;
A display control apparatus for recognizing a line direction and a line feed direction of the document based on the number of blank lines in the first array line direction and the number of blank lines in the second array line direction. A program for displaying on a display means an image formed by a plurality of pixels arranged in a matrix,
Computer
Acquisition means for acquiring image data for displaying at least the image in which the document is imaged as a document image on the display means;
Setting means for setting a size of a display area in the display means when displaying the document image;
Recognition means for recognizing the line direction and line feed direction of the document in the document image based on the acquired image data;
Binarizing means for binarizing each pixel value of the document image;
Row detection means for detecting a row of the document image on the basis of a pixel value in each binarized pixel for each first array line which is an array line of pixels in the document row direction of the document image;
For each detected line, one of the pixels belonging to each second array line for each second array line, which is an array line of pixels in the document feed direction of the document image, is binarized. A character segment that detects at least a first pixel having a value of and detects a segment of a character included in each line as a character segment based on the presence or absence of the first pixel while specifying the size and segment position of the segment Detection means,
(1) Based on each character section of each detected line, a maximum section size indicating the size of the largest character section in the line direction in each line, an average section size indicating an average of the size of the character sections based on all lines, And calculating a maximum character pitch indicating a maximum value of a distance between two adjacent character sections in each line , and (2) a difference between the maximum section size in each line and the average section size based on all the lines is minimum. The maximum character pitch in a given line is calculated as a reference character pitch, and (3) each line belongs to the same line and is adjacent based on the detection result of the first pixel when the character section is detected A character pitch indicating an arrangement interval in the line direction of the document in two character sections is calculated. (4) The calculated character pitch is a character pitch condition determined in advance based on the calculated reference character pitch. Integration correction means performs integration correction two character compartments of calculation target of the character pitch as the same character compartments having a matter,
An arrangement position determining means for determining an arrangement position for arranging the corrected character sections in the display area based on the set area size of the display area;
Image generating means for generating a display image for displaying in the display area by arranging a section image corresponding to each character section at a position where each character section is determined; as well as,
Output means for outputting the generated image to the display means;
A program characterized by functioning as

Images