JP2021118512A - Image data generation apparatus, image data generation system, image data generation method, and computer program - Google Patents

Abstract

To reduce a burden on a user.SOLUTION: An image data generation apparatus comprises: an identification unit which uses document image data having a plurality of character strings expressing writing target items needing writing of user information associated with a user, and thereby identifies writing regions of information for the character strings expressing writing target items; and an image data generation unit which generates written document image data by superimposing the user information in identified writing regions.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image data generator, an image data generation system, an image data generation method, and a computer program.

When applying for a service or changing the list information and registered information, it is common to write the necessary information in a document and submit it. In particular, the description of information in such a document is often done by handwriting. However, depending on the amount and the number of sheets to be described, it will be a heavy burden on the user. Therefore, conventionally, a technique has been proposed in which a document can be read by a scanner or the like and characters can be input to a file of the read document by a personal computer (see, for example, Non-Patent Document 1).

"Goodbye Handwritten 6/6 Pro", SOURCENEXT, [Searched on January 6, 2nd year of Reiwa], Internet (URL: https://www.sourcenext.com/product/pc/use/pc_use_002772/?i=rd&i= yss_sem_tegaki6 & utm_source = yss & utm_medium = sem & utm_campaign = tegaki6 & utm_term = name & argument = HGtSAqRz & dmai = a5dde02bbc4485 & yclid = YSS.1000399450.EAIaIQobChMI4MT4gOft5

In the technique of Non-Patent Document 1 described above, character information input by a personal computer can be printed on a document. However, even with the technique of Non-Patent Document 1, since it is necessary for the user to input all the characters, there is a problem that the burden on the user cannot be reduced depending on the amount and the number of characters to be input.

In view of the above circumstances, an object of the present invention is to provide a technique capable of reducing the burden on the user.

One aspect of the present invention uses image data of a document having a plurality of character strings representing a description target item for which user information regarding a user needs to be described, to provide a description area of the user information with respect to the character string representing the description target item. It is an image data generation apparatus including a specific unit to be specified and an image data generation unit that generates described document image data by synthesizing the user information with respect to the specified description area.

One aspect of the present invention is the image data generation device, wherein the specific unit has a positional relationship between a region of a character string representing the description target item obtained from the image data and a candidate region of the description region. The description area for each character string representing the description target item is specified from.

One aspect of the present invention is the image data generation device, wherein the specific unit estimates a character string representing the description target item based on the image data, and the character string representing the description target item is described. Using a model trained to output the position information of the description area of the user information, the description area is specified with respect to the character string representing the estimated description target item.

One aspect of the present invention is the image data generation device, wherein the specific unit calculates and calculates the degree of similarity between the image data and the registered image data of one or more documents registered in advance. The description area of the image data is specified by using the registered image data whose similarity is equal to or higher than the threshold value.

One aspect of the present invention is the above-mentioned image data generation device, further including a user information acquisition unit for acquiring the user information, and the user information acquisition unit is an external device, a certificate possessed by the user. The user information is acquired from any or all of the given memory, the certificate, the storage unit, and the voice input.

One aspect of the present invention is an image data generation system including the above-mentioned image data generation device and an image data registration device for registering registered image data used in the image data generation device. Using the image data of a document having a plurality of character strings representing the description target items for which the user information regarding the user needs to be described, a specific unit for specifying the description area of the user information with respect to the character string representing the description target item, and A registration unit that associates the position information of the description area specified by the specific unit, a character string representing the description target item, and the image data and registers the registered image data in the image data generation device. The identification unit of the image data generation device specifies the description area of the image data by using the image data and one or more registered image data registered by the image data registration device. The image data generation unit of the image data generation device is an image data generation system that generates described document image data by synthesizing the user information with respect to the specified description area.

One aspect of the present invention is the above-mentioned image data generation system, in which the registration unit displays the image data on a screen and a region of a character string representing a description target item designated by a user on the screen. The position information of the description area with respect to the character string representing the description target item and the image data are associated with each other and registered in the image data generation device as the registered image data.

One aspect of the present invention is the image data generation system, wherein a specific character string for specifying a character string representing the description target item is described in the description area of the image data. The registration unit acquires the position information of the specific character string and the area of the specific character string from the image data as the description area, and the position information of the acquired description area and the description specified by the specific character string. The character string representing the target item is associated with the image data and registered as the registered image data in the image data generation device.

One aspect of the present invention is the above-mentioned image data generation system, and a specific unit of the image data registration device includes a character string area representing a description target item obtained from the image data and a candidate area of the description area. The description area for each character string representing the description target item is specified from the positional relationship of, or the character string representing the description target item is estimated based on the image data, and the character string representing the description target item is described. Using a model trained to output the position information of the description area of the user information, the description area is specified with respect to the character string representing the estimated description target item.

One aspect of the present invention uses image data of a document having a plurality of character strings representing a description target item for which user information regarding a user needs to be described, to provide a description area of the user information with respect to the character string representing the description target item. This is an image data generation method for generating described document image data by synthesizing the user information with respect to the specified and specified description area.

One aspect of the present invention uses image data of a document having a plurality of character strings representing a description target item for which user information regarding a user needs to be described, to provide a description area for the user information with respect to the character string representing the description target item. It is a computer program for causing a computer to execute a specific step to be specified and an image data generation step to generate described document image data by synthesizing the user information with respect to the specified description area.

According to the present invention, it is possible to reduce the burden on the user.

It is a block diagram which shows the structure of the image data generation system in this invention. It is a schematic block diagram which shows the functional structure of the image data generation apparatus in 1st Embodiment. It is a figure which shows one specific example of the user information table in 1st Embodiment. It is a figure which shows one specific example of the collation data table in 1st Embodiment. It is a flowchart which shows the flow of the image data generation processing of the image data generation apparatus in 1st Embodiment. It is a schematic block diagram which shows the functional structure of the image data generation apparatus in 2nd Embodiment. It is a flowchart which shows the flow of the image data generation processing of the image data generation apparatus in 2nd Embodiment. It is a block diagram which shows the structure of the image data generation system in 3rd Embodiment. It is a schematic block diagram which shows the functional structure of the image data registration apparatus in 3rd Embodiment. It is a schematic block diagram which shows the functional structure of the image data generation apparatus in 3rd Embodiment. It is a flowchart which shows the flow of the image data generation processing of the image data generation apparatus in 3rd Embodiment. It is a schematic block diagram which shows the functional structure of the image data generation apparatus in 4th Embodiment. It is a flowchart which shows the flow of the image data generation processing of the image data generation apparatus in 4th Embodiment. It is a schematic block diagram which shows the functional structure of the image data generation apparatus in 5th Embodiment. It is a flowchart which shows the flow of the image data generation processing of the image data generation apparatus in 5th Embodiment. It is a schematic block diagram which shows the functional structure of the image data generation apparatus in 6th Embodiment. It is a flowchart which shows the flow of the image data generation processing of the image data generation apparatus in 6th Embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram showing the configuration of the image data generation system 100 in the present invention. The image data generation system 100 includes an image data generation device 10, a server 20, and a printing device 30. The image data generation device 10, the server 20, and the printing device 30 are communicably connected via the network 40. The network 40 may be a network configured in any way. For example, the network 40 is composed of the Internet.

The image data generation device 10 uses the image data of the document and the user information to generate the described document image data in which the user information is combined with the image data of the document. A document is a document having a plurality of character strings (hereinafter, referred to as "target item character strings") representing description target items for which user information needs to be described. The document is, for example, a form, a contract, or the like. These are examples, and the document may be any document as long as it has a plurality of target item character strings. The user information is information related to the attributes of the user, such as the user's name, address, telephone number, date of birth, and the like. The target item character string is a character string representing an item for which user information needs to be described, and is, for example, a character string such as a name, an address, a telephone number, and a date of birth.

The image data generation device 10 is a communication device or a printing device having a function of generating described document image data. When the image data generation device 10 is a communication device having a function of generating the described document image data, the image data generation device 10 is configured by using an information processing device such as a smartphone, a mobile phone, or a tablet terminal. When the image data generation device 10 is a printing device having a function of generating the described document image data, the image data generation device 10 is configured by using, for example, a printer, a multifunction device, or the like.

The server 20 holds the described document image data generated by the image data generation device 10. The server 20 holds, for example, the described document image data and the specific information for specifying the described document image data in association with each other. The specific information may be, for example, an ID (identification) or a password, or may be address information in which the described document image data is stored, or the ID or password or the described document image data is stored. The existing address information or the like may be coded information (for example, a one-dimensional barcode or a two-dimensional barcode). The server 20 is a cloud server on the network 40. The server 20 is configured by using an information processing device such as a personal computer.

The printing device 30 prints the described document image data. The printing device 30 may be, for example, a multifunction device or a printer. The printing device 30 is installed at a convenience store, a government office, or the like.

Next, with reference to FIG. 1, a series of flows of the image data generation system 100 in the present invention will be described with an example.
The image data generation device 10 acquires user information from the certificate 1 possessed by the user or by manual input by the user ((1) in FIG. 1). Certificate 1 is, for example, a driver's license, an Individual Number Card, a health insurance card, or the like. As a method for the image data generation device 10 to acquire user information from the certificate 1, for example, if the certificate 1 is a My Number card, it is read from the storage area (memory) of the My Number card, or converted into text by OCR (Optical Character Reader). A method of obtaining the data from the outside via the network 40 using the information of the certificate 1 and the like can be mentioned.

Next, the image data generation device 10 acquires the image data of the document by photographing or reading the document ((2) of FIG. 1). The image data generation device 10 generates the described document image data by synthesizing the acquired user information with the acquired document image data. Specifically, the image data generation device 10 specifies a description area of user information for the target item character string on the image data of the document, and the user information specified by the target item character string for the specified description area. The described document image data is generated by synthesizing. For example, when the target item character string is a "name", the image data generation device 10 synthesizes the information of the "name" included in the acquired user information in the description area corresponding to the "name". In a general document, there are a pattern in which either or both of the target item character string and the description area are surrounded by a frame, and a pattern in which the target item character string and / or the description area are not surrounded by a frame. The image data generation device 10 can be applied to any pattern. The specific configuration will be described later. Here, being surrounded by a frame means that each of the target item character string and the description area is surrounded by a frame on all sides.

The image data generation device 10 transmits the generated written document image data to the server 20 via the network 40 ((3) in FIG. 1).
The server 20 receives the described document image data transmitted from the image data generation device 10. The server 20 holds the received written document image data in association with specific information. After that, the server 20 transmits the specific information to the image data generation device 10 via the network 40 ((4) in FIG. 1). For example, it is assumed that the server 20 transmits an ID and a password as specific information.

The image data generation device 10 receives the specific information transmitted from the server 20. The image data generation device 10 displays the received specific information on the screen. The user inputs the specific information (for example, ID and password) displayed on the screen of the image data generation device 10 into the printing device 30 ((5) in FIG. 1).
The printing device 30 transmits the input specific information to the server 20 via the network 40 ((6) in FIG. 1).

The server 20 receives the specific information transmitted from the printing device 30. The server 20 acquires the described document image data specified by the received specific information. For example, the server 20 acquires the described document image data specified by the received ID and password. The server 20 transmits the acquired written document image data to the printing device 30 ((7) in FIG. 1).

The printing device 30 receives the described document image data transmitted from the server 20. The printing device 30 displays the received written document image data on the screen, and receives an instruction to execute printing from the user. When the print execution instruction is input, the printing device 30 prints the described document image data. As a result, the user can acquire the described document 3 in which the user information is described ((8) in FIG. 1). In the described document 3, the user information is described and printed in the description area corresponding to the target item character string. This eliminates the need for the user to fill in everything by hand. Therefore, it is possible to reduce the burden on the user.

Hereinafter, a specific configuration (first embodiment to sixth embodiment) of the image data generation device 10 in the present invention will be described. The case where the image data generation device 10 is a communication device having an image data generation function will be described in the first to third embodiments, and the image data in the fourth to sixth embodiments. A case where the generation device 10 is a printing device having an image data generation function will be described.

(First Embodiment)
In the first embodiment, a case where a document format of a document taken by an image data generation device as a communication device is used in which each of the target item character string and the description area is surrounded by a frame will be described.
FIG. 2 is a schematic block diagram showing a functional configuration of the image data generation device 10 according to the first embodiment.
The image data generation device 10 includes a communication unit 11, a camera 12, a display unit 13, an operation unit 14, a control unit 15, and a storage unit 16.

The communication unit 11 communicates with the server 20 via the network 40. The communication unit 11 transmits the described document image data to the server 20. The communication unit 11 receives the specific information transmitted from the server 20.
The camera 12 captures a document according to the operation of the user.

The display unit 13 is an image display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 displays various information. The display unit 13 displays, for example, the described document image data and specific information. The display unit 13 may be an interface for connecting the image display device to the image data generation device 10. In this case, the display unit 13 generates a video signal for displaying the described document image data and specific information, and outputs the video signal to an image display device connected to the display unit 13.

The operation unit 14 is an input device for inputting a user's instruction to the image data generation device 10. The operation unit 14 receives, for example, input of a correction instruction of the described document image data and a display instruction of specific information. The operation unit 14 is configured by using an existing input device such as a keyboard, a touch panel, and a button. Further, the operation unit 14 may be an interface for connecting the input device to the image data generation device 10. In this case, the operation unit 14 inputs the input signal generated in response to the user's input in the input device to the image data generation device 10.

The control unit 15 is configured by using a processor such as a CPU (Central Processing Unit) or a memory. The control unit 15 realizes the functions of the image data acquisition unit 151, the specific unit 152, the image data generation unit 153, the registration unit 154, and the communication control unit 155 by executing the program. Some of these functions (for example, image data acquisition unit 151, specific unit 152, image data generation unit 153, and registration unit 154) do not need to be mounted on the image data generation device 10 in advance, and additional application programs are provided. May be realized by being installed in the image data generation device 10.

The image data acquisition unit 151 acquires image data of a document taken by the camera 12.
The identification unit 152 specifies the description area of the user information for the target item character string by using the image data of the document acquired by the image data acquisition unit 151. Specifically, the specific unit 152 describes each target item character string based on the positional relationship between the area representing the target item character string obtained from the image data (hereinafter referred to as "item character string area") and the candidate area of the description area. Specify the description area for.

The image data generation unit 153 generates the described document image data by synthesizing the user information with respect to the description area specified by the specific unit 152. The image data generation unit 153 generates the described document image data by synthesizing the user information stored in the storage unit 16 with respect to the description area, for example. Here, the composition performed by the image data generation unit 153 means superimposing the user information on the description area on the image data.

The registration unit 154 registers the position information of the description area with respect to the target item character string specified by the specific unit 152, the target item character string, and the image data in the storage unit 16 as registered image data in association with each other. In the first embodiment, the registration unit 154 uses information (for example, coordinates, height and width) capable of designating a frame as the position information of the description area.
The communication control unit 155 controls the communication unit 11. For example, the communication control unit 155 controls the communication unit 11 to transmit the described document image data to the server 20.

The storage unit 16 stores the user information table 161, the collation data table 162, and the registered image data 163. The user information table 161 is a table in which information regarding user attributes is registered. The user information table 161 may be stored in the storage unit 16 in the format shown in FIG. The collation data table 162 is a table in which information regarding the target item character string is registered. The collation data table 162 may be stored in the storage unit 16 in the format shown in FIG. The registered image data 163 is data in which the position information of the description area with respect to the target item character string specified by the specific unit 152, the target item character string, and the image data are associated with each other. The registered image data 163 may be stored in the storage unit 16 in the data format of, for example, an xml (Extensible Markup Language) file. The storage unit 16 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device.

FIG. 3 is a diagram showing a specific example of the user information table 161 according to the first embodiment.
The user information table 161 is composed of a plurality of records associated with each value of the user ID, the attribute type, and the registered contents. The user ID represents identification information for identifying the user. The attribute type represents the type of the user's attribute. For example, the attribute type is a user's name, address, telephone number, date of birth, age, gender, and the like. The registered content represents the information of the user registered with the user or registered based on the certificate. In the first embodiment, it is assumed that the registered contents corresponding to the attribute types are registered in advance by the user in the user information table 161.

As shown in FIG. 3, in the user information table 161, user information corresponding to the attribute type is registered for each user ID. For example, in the user information table 161 shown in FIG. 3, the user identified by the user ID "AAA" has a name of "△△△△", an address of "○○○○", and a telephone number of "XXX-XXX". ", The date of birth is registered as" YY: MM: DD ". By registering the user information according to the attribute type in this way, the user information corresponding to the target item character string can be selected.

FIG. 4 is a diagram showing a specific example of the collation data table 162 according to the first embodiment.
The collation data table 162 is composed of a plurality of records in which each value of the target item character string and the character string related to the target item character string is associated with each other. The target item character string represents a character string representing a target item character string for which user information needs to be described. The character string related to the target item character string represents a character string for specifying the target item character string. By registering a plurality of character strings related to the target item character string, it is possible to deal with fluctuations when specifying the target item character string described in the document.

For example, in the example shown in FIG. 4, when the character string recognized from the image data by the specific unit 152 corresponds to any of "name, name, gender, name, Name", the target corresponding to the recognized character string. It is shown that the item string is identified as "name". For example, in the example shown in FIG. 4, when the character string recognized from the image data by the specific unit 152 corresponds to any one of "address, whereabouts, location, address, address", the target corresponding to the recognized character string. It is shown that the item string is identified as an "address".

FIG. 5 is a flowchart showing the flow of the image data generation process of the image data generation device 10 according to the first embodiment.
The camera 12 captures a document in response to a user operation (step S101). The camera 12 outputs the image data of the captured document to the control unit 15. The image data acquisition unit 151 acquires the image data output from the camera 12 (step S102). The identification unit 152 identifies the character string and the frame line (for example, the position of the area surrounded by the frame) in the image data by using the image data acquired by the image data acquisition unit 151 (step S103). Specifically, the identification unit 152 identifies the character string and the frame line (for example, the position of the area surrounded by the frame) in the image data by the existing OCR technology. As a result, when each of the target item character string and the description area is surrounded by a frame, the specific unit 152 specifies the frame line indicating the item character string area and the description area. When the target item character string is surrounded by a frame, the identification unit 152 specifies a frame line indicating the item character string area. When the description area is surrounded by a frame, the specific unit 152 specifies a frame line indicating the description area.

Next, the specifying unit 152 refers to the collation data table 162 and specifies the target item character string from the specified character string (step S104). Specifically, first, the specific unit 152 reads the collation data table 162 from the storage unit 16. Next, the specifying unit 152 collates each of the specified character strings with the character strings registered in the item of the character string related to the target item character string of the read collation data table 162. Then, the specific unit 152 acquires the value of the target item character string associated with the item of the character string related to the target item character string in which the character string matching the specified character string is registered. For example, when the specified character string is the "name", the specific unit 152 acquires the "name" as the value of the target item character string associated with the item of the character string related to the target item character string. In this way, the specifying unit 152 specifies the target item character string from the specified character string.

The identification unit 152 specifies a description area corresponding to the specified target item character string (step S105). Specifically, the specifying unit 152 specifies the description area for each target item character string from the positional relationship between the item character string area and the candidate area of the description area represented by the specified frame line. More specifically, the specific unit 152 describes each target item based on the rule base from the position of the frame of the item character string area including the character string (for example, "name") related to the target item character string and the positional relationship of the surrounding frames. Specify the description area for the character string. For example, the specific unit 152 describes based on a rule that the empty frame to the right of the frame of the item character string area including the character string related to the target item character string is the description area corresponding to the target item character string. Identify the area.

The image data generation unit 153 generates the described document image data using the user information (step S106). Specifically, first, the image data generation unit 153 refers to the user information table 161 and acquires the value of the registered content associated with the attribute type corresponding to the target item character string specified in the process of step S104. .. In the example of FIG. 3, when the target item character string specified in the process of step S104 is "name", "address", and "telephone number", the image data generation unit 153 uses the user information table 161 to indicate the attribute type "name". , "Address", "Telephone number", and the registered contents "△△△△", "○○○○", and "XXX-XXX" are acquired. Next, the image data generation unit 153 generates the described document image data by synthesizing the acquired value at the position of the specified description area on the image data. For example, the image data generation unit 153 synthesizes the registration content “△△△△” associated with the attribute type “name” in the description area corresponding to the target item character string “name”, and synthesizes the target item character string “△△△△”. The registration content "○○○○" associated with the attribute type "address" is synthesized in the description area corresponding to the "address", and the attribute type "name" is placed in the description area corresponding to the target item character string "phone number". The described document image data is generated by synthesizing the registered contents "XXX-XXX" associated with "".

The image data generation unit 153 outputs the generated written document image data to the communication control unit 155. The communication control unit 155 controls the communication unit 11 and saves the described document image data by transmitting the described document image data output from the image data generation unit 153 to the server 20 (step S107).
The registration unit 154 may display the described document image data generated by the image data generation unit 153 on the display unit 13 and accept the user's modification. If there is no correction from the user, that is, if there is no instruction to change the described document image data, the registration unit 154 sets the position information of the description area with respect to the target item character string specified by the specific unit 152 and the target item. The character string and the image data are associated with each other and registered as registered image data in the storage unit 16. On the other hand, when the user makes a correction, that is, when there is an instruction to change the described document image data, the registration unit 154 sets the position information of the description area with respect to the corrected target item character string and the target item character string. And the image data are associated with each other and registered in the storage unit 16 as registered image data.

According to the image data generation device 10 configured as described above, it is possible to reduce the burden on the user. Specifically, the image data generation device 10 has already described the information by specifying the information description area for the target item character string using the image data of the document and synthesizing the user information with the specified description area. Generate document image data. This eliminates the need for the user to fill in everything by hand. Therefore, it is possible to reduce the burden on the user.

<Modified example of the first embodiment>
In the present embodiment, the case where the user information is registered in advance has been described as an example, but the image data generation device 10 may acquire the user information by another method. For example, the image data generation device 10 may acquire user information from an external device via the network 40, may be acquired from the certificate 1, or may be acquired by voice recognition. When acquiring user information from an external device via the network 40, the image data generation device 10 accesses the external device using the information of the certificate 1 (for example, My Number card) having a storage area to obtain the user information. It is provided with a user information acquisition unit for acquisition. When the user information is acquired from the certificate 1, the image data generation device 10 includes a user information acquisition unit that acquires the user information by converting the image data of the certificate 1 into text using OCR technology. When the user information is acquired by voice recognition, the image data generation device 10 includes a user information acquisition unit that converts the voice into text by voice recognition and acquires the user information.

The identification unit 152 may specify the description area by using a model trained to output the position information of the description area of the user information with respect to the target item character string.
When specifying the description area, the identification unit 152 determines whether or not the registered image data similar to the image data acquired by the image data acquisition unit 151 is stored in the storage unit 16, and determines whether or not the registered image data similar to the image data acquisition unit 151 is stored in the storage unit 16. When the data is stored in the storage unit 16, the description area may be specified by using the registered image data 163 stored in the storage unit 16. When the description position is specified by the registered image data 163, the image data generation unit 153 generates the described document image data using the registered image data 163 instead of the acquired image data in the process of step S106 of FIG. You may. The identification unit 152 determines whether or not there is registered image data 163 similar to the image data based on the degree of similarity obtained by image matching. The specific unit 152 calculates the similarity between the image data and the registered image data 163, and when there is registered image data 163 whose calculated similarity is equal to or greater than the threshold value, similar registered image data is stored in the storage unit 16. Judge that there is. On the other hand, when there is no registered image data 163 whose similarity is equal to or higher than the threshold value, it is determined that similar registered image data is not stored in the storage unit 16. Then, when there are a plurality of registered image data 163 having a similarity equal to or higher than the threshold value, the specific unit 152 uses the registered image data having the highest similarity.
With this configuration, it is not necessary to estimate the writing area for documents of the same format. Therefore, it is possible to shorten the time required to generate the described document image data.

Some functions included in the image data generation device 10 may be mounted in another housing. For example, a functional unit (for example, an image data acquisition unit 151, a specific unit 152, an image data generation unit 153, a registration unit 154, a user information table 161) corresponding to a described document image data generation function included in the image data generation device 10. The collation data table 162 and the registered image data 163) may be mounted on another information processing device (for example, the server 20). In this case, another information processing device having a function of generating the described document image data functions as the image data generation device. When configured in this way, the image data generation device 10 transmits the image data acquired by the camera 12 to another information processing device via the network 40. Then, the described document image data generation process (process of FIG. 5) is executed in another information processing device.

(Second embodiment)
In the second embodiment, a case where at least the description area is not surrounded by a frame is used as the document format of the document taken by the image data generation device as the communication device. Hereinafter, only the differences from the first embodiment will be described.
FIG. 6 is a schematic block diagram showing a functional configuration of the image data generation device 10a according to the second embodiment.
The image data generation device 10a includes a communication unit 11, a camera 12, a display unit 13, an operation unit 14, a control unit 15a, and a storage unit 16a.

The control unit 15a is configured by using a processor such as a CPU or a memory. The control unit 15a realizes the functions of the image data acquisition unit 151, the specific unit 152a, the image data generation unit 153, the registration unit 154, and the communication control unit 155 by executing the program. Some of these functions (for example, the image data acquisition unit 151, the specific unit 152a, the image data generation unit 153, and the registration unit 154) do not need to be mounted on the image data generation device 10a in advance, and an additional application program is provided. May be realized by being installed in the image data generation device 10a.

The specific unit 152a estimates the target item character string based on the image data, and uses a model trained to output the position information of the user information description area with respect to the target item character string. Specify the description area for.

The storage unit 16a stores the user information table 161, the collation data table 162, the registered image data 163, and the model data 164. The storage unit 16a is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device.
The model data 164 is model data learned to output the position information of the description area of the user information with respect to the target item character string. The model data 164 is generated by learning using a large number of data indicating the relationship between the item name specified by the target item character string of various documents and the description position of the user information.

FIG. 7 is a flowchart showing the flow of the image data generation process of the image data generation device 10a according to the second embodiment. In FIG. 7, the same processing as in FIG. 5 is designated by the same reference numerals as in FIG. 5, and the description thereof will be omitted.
The identification unit 152a specifies a character string in the image data and the position information of the character string by using the image data acquired by the image data acquisition unit 151 (step S201). After that, the specifying unit 152a then refers to the collation data table 162 and specifies the target item character string from the specified character string (step S104). The identification unit 152a specifies a description area corresponding to the specified target item character string (step S202). Specifically, the specific unit 152a acquires the position information of the description area from the model data 164 by inputting the item name of the target item character string and the position information into the model data 164 stored in the storage unit 16a. By doing so, the description area corresponding to the target item character string is specified. After that, the image data generation device 10a executes the processes after step S106.

According to the image data generation device 10a configured as described above, the same effect as that of the first embodiment can be obtained.
Further, the image data generation device 10a specifies the description area for the estimated target item character string by using the model trained to output the position information of the information description area for the target item character string. Thereby, as in the first embodiment, the description area can be specified even when the description area is not surrounded by the frame. Therefore, it is possible to reduce the burden on the user.

<Modified example of the second embodiment>
In the present embodiment, the case where the user information is registered in advance has been described as an example, but the image data generation device 10a may acquire the user information by another method. For example, the image data generation device 10a may acquire the user information from an external device via the network 40, may be acquired from the certificate 1, or may be acquired by voice recognition. Since the specific method of the other method is the same as that of the first embodiment, the description thereof will be omitted.

Some functions included in the image data generation device 10a may be mounted in another housing. For example, a functional unit (for example, an image data acquisition unit 151, a specific unit 152a, an image data generation unit 153, a registration unit 154, a user information table 161) corresponding to a described document image data generation function included in the image data generation device 10a. The collation data table 162, the registered image data 163, and the model data 164) may be mounted on another information processing device (for example, the server 20). In this case, another information processing device having a function of generating the described document image data functions as the image data generation device. When configured in this way, the image data generation device 10a transmits the image data acquired by the camera 12 to another information processing device via the network 40. Then, the described document image data generation process (process of FIG. 7) is executed in another information processing device.

(Third Embodiment)
In the third embodiment, a configuration in which an image data generation device as a communication device specifies a description area using pre-registered registered image data will be described. The registered image data is generated by a device different from the image data generation device and registered in the image data generation device. Hereinafter, only the differences from each embodiment will be described.
FIG. 8 is a configuration diagram showing the configuration of the image data generation system 100b according to the third embodiment. The image data generation system 100b includes an image data generation device 10b, a server 20, a printing device 30, and an image data registration device 50. The image data generation device 10b, the server 20, the printing device 30, and the image data registration device 50 are communicably connected via the network 40. The image data generation device 10b and the image data registration device 50 may be directly connected by wire or wirelessly.

The image data registration device 50 registers the registered image data used in the image data generation device 10b in the image data generation device 10b. The registered image data used in the image data generation device 10b is data in which the position information of the description area with respect to the target item character string, the target item character string, and the image data are associated with each other. The image data registration device 50 is configured by using an information processing device such as a smartphone, a mobile phone, a tablet terminal, or a personal computer.

The image data generation device 10b generates the described document image data by using the document image data, the user information, and the registered image data registered in advance by the image data registration device 50.

FIG. 9 is a schematic block diagram showing a functional configuration of the image data registration device 50 according to the third embodiment.
The image data registration device 50 includes a communication unit 51, a camera 52, a display unit 53, an operation unit 54, a control unit 55, and a storage unit 56.

The communication unit 51 communicates with the image data generation device 10b directly or via the network 40. The communication unit 51 transmits the registered image data to the image data generation device 10b.
The camera 52 captures a document according to the operation of the user.
The display unit 53 is an image display device such as a liquid crystal display or an organic EL display. The display unit 53 displays various information. The display unit 53 displays, for example, image data. The display unit 53 may be an interface for connecting the image display device to the image data registration device 50. In this case, the display unit 53 generates a video signal for displaying the image data, and outputs the video signal to the image display device connected to the display unit 53.

The operation unit 54 is an input device for inputting a user's instruction to the image data registration device 50. The operation unit 54 receives, for example, input of an instruction for designating the item character string area and the description area and an instruction for modifying the description area. The operation unit 54 is configured by using existing input devices such as a keyboard, a touch panel, and buttons. Further, the operation unit 54 may be an interface for connecting the input device to the image data registration device 50. In this case, the operation unit 54 inputs the input signal generated in response to the user's input in the input device to the image data registration device 50.

The control unit 55 is configured by using a processor such as a CPU and a memory. The control unit 55 realizes the functions of the image data acquisition unit 551, the specific unit 552, the registration unit 553, the display control unit 554, and the communication control unit 155 by executing the program. Some of these functions (for example, image data acquisition unit 551, specific unit 552, registration unit 553, display control unit 554, and communication control unit 155) do not need to be mounted on the image data registration device 50 in advance. It may be realized by installing an additional application program on the image data registration device 50.

The image data acquisition unit 551 acquires image data of a document taken by the camera 52.
The identification unit 552 specifies a description area of user information for the target item character string by using the image data of the document acquired by the image data acquisition unit 551. The specific unit 552 outputs the position information of the specified description area, the target item character string, and the image data to the registration unit 553. As a method for specifying the description area by the specific unit 552, the following three methods can be mentioned.

The first method is a method in which the user specifies an item character string area and a description area for the target item character string. In the case of the first method, first, the identification unit 552 causes the display control unit 554 to display the image data of the document acquired by the image data acquisition unit 551 on the screen of the display unit 53. Next, the description area is specified by the position information of the item character string area designated by the user on the screen and the description area with respect to the target item character string.

As a second method, the user describes a specific character string (for example, an item name) for specifying the target item character string in the document description area in advance, and causes the user to read the specific character string. The method. In the case of the second method, the identification unit 552 specifies the description position of the specific character string from the image data of the document acquired by the image data acquisition unit 551. Specifically, the specific unit 552 specifies a specific character string in the image data and its position information by the existing OCR technology.

The third method is a method of specifying the description area as in the first embodiment and the second embodiment. In this case, the specific unit 552 performs the same processing as the specific unit 152 or the specific unit 152a. When the specific unit 552 specifies the description area by the same method as the specific unit 152a, the image data registration device 50 stores the model data 164.

The specific unit 552 displays the position information of the specified description area and the target item character string corresponding to the description area on the display unit 53, and displays the specified description area and the target item character string corresponding to the description area. You may accept the user's correction about the correspondence of. If there is no modification from the user, that is, if there is no instruction to change the correspondence between the specified description area and the target item character string corresponding to the description area, the specific unit 552 is the description area for the target item character string. The position information, the target item character string, and the image data are output to the registration unit 553. On the other hand, when the user makes a correction, that is, when there is an instruction to change the correspondence between the specified description area and the target item character string corresponding to the description area, the registration unit 553 receives the corrected target item character. The position information of the description area with respect to the column, the target item character string, and the image data are output to the registration unit 553.

The registration unit 553 registers the position information of the description area with respect to the target item character string output from the specific unit 552, the target item character string, and the image data in the image data generation device 10b as registered image data in association with each other.
The display control unit 554 controls the display of the display unit 53. For example, the display control unit 554 causes the display unit 53 to display the position information of the description area and the target item character string corresponding to the description area in response to the instruction of the specific unit 552. The position information of the description area and the target item character string corresponding to the description area are displayed on the display unit 53 when the user's modification is accepted.
The communication control unit 555 controls the communication unit 51. For example, the communication control unit 555 controls the communication unit 51 to transmit the registered image data to the image data generation device 10b.

The storage unit 56 stores the collation data table 561. The collation data table 561 is a table in which information regarding the target item character string is registered. Since the specific configuration of the collation data table 561 is the same as that of the first embodiment and the second embodiment, the description thereof will be omitted. The storage unit 56 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device.

FIG. 10 is a schematic block diagram showing a functional configuration of the image data generation device 10b according to the third embodiment.
The image data generation device 10b includes a communication unit 11b, a camera 12, a display unit 13, an operation unit 14, a control unit 15b, and a storage unit 16b.

The communication unit 11b communicates with the server 20 via the network 40. The communication unit 11b communicates with the image data registration device 50 directly or via the network 40. The communication unit 11b receives the registered image data from the image data registration device 50.

The control unit 15b is configured by using a processor such as a CPU or a memory. The control unit 15b realizes the functions of the image data acquisition unit 151b, the specific unit 152b, the image data generation unit 153, and the communication control unit 155 by executing the program. Some of these functions (for example, the image data acquisition unit 151b, the specific unit 152b, and the image data generation unit 153) do not need to be mounted on the image data generation device 10b in advance, and an additional application program can generate image data. It may be realized by being installed in the device 10b.

The image data acquisition unit 151b acquires the image data of the document taken by the camera 12 and the registered image data received by the communication unit 11b. The image data acquisition unit 151b stores the acquired registered image data in the storage unit 16b.
The specific unit 152b calculates the similarity between the image data and the registered image data of one or more documents stored in the storage unit 16b, and uses the registered image data whose calculated similarity is equal to or greater than the threshold value to obtain image data. Specify the description area of.

The storage unit 16b stores the user information table 161, the collation data table 162, and the registered image data 163b. The storage unit 16b is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device.
The registered image data 163b is the registered image data acquired by the image data acquisition unit 151b.

FIG. 11 is a flowchart showing the flow of the image data generation process of the image data generation device 10b according to the third embodiment. In FIG. 11, the same processing as in FIG. 5 is designated by the same reference numerals as in FIG. 5, and the description thereof will be omitted. At the time of executing the process of FIG. 11, it is assumed that the registered image data is registered in advance in the storage unit 16b of the image data generation device 10b.
The camera 12 captures a document in response to a user operation (step S301). The camera 12 outputs the image data of the captured document to the control unit 15b. The image data acquisition unit 151b acquires the image data output from the camera 12 (step S302). The specific unit 152b collates the acquired image data with the registered image data 163b stored in the storage unit 16b (step S303). Specifically, the specific unit 152b calculates the similarity between the acquired image data and the registered image data 163b stored in the storage unit 16b by image matching.

The specific unit 152b selects one registered image data from the registered image data 163b stored in the storage unit 16b based on the calculated similarity (step S304). For example, the specific unit 152b selects the registered image data having the highest degree of similarity. Then, the identification unit 152b specifies the description area of the acquired image data based on the position information of the description area included in the selected registered image data (step S305). After that, the image data generation device 10b executes the processes after step S106.

According to the image data generation device 10b configured as described above, the same effect as that of the first embodiment can be obtained.
Further, the image data generation device 10b specifies the description area of the acquired image data by using the position information of the description area of the registered image data similar to the acquired image data. As a result, the time required for the process of specifying the description area of the image data can be shortened. Therefore, it is possible to shorten the time required to generate the described document image data.

<Modified example of the third embodiment>
In the present embodiment, the case where the user information is registered in advance has been described as an example, but the image data generation device 10b may acquire the user information by another method. For example, the image data generation device 10b may acquire the user information from an external device via the network 40, may be acquired from the certificate 1, or may be acquired by voice recognition. Since the specific method of the other method is the same as that of the first embodiment, the description thereof will be omitted.

Some functions included in the image data generation device 10b may be mounted in another housing. For example, a functional unit (for example, an image data acquisition unit 151b, a specific unit 152b, an image data generation unit 153, a user information table 161 and a collation data table 162) corresponding to a described document image data generation function included in the image data generation device 10b. And the registered image data 163b) may be mounted on another information processing device (for example, the server 20). In this case, another information processing device having a function of generating the described document image data functions as the image data generation device. When configured in this way, the image data generation device 10b transmits the image data acquired by the camera 12 to another information processing device via the network 40. Then, the described document image data generation process (process of FIG. 11) is executed in another information processing device.

(Fourth Embodiment)
In the fourth embodiment, a case where the image data generation device as the printing device acquires the user information by another method (for example, reading by a card reader) will be described. Further, in the fourth embodiment, a case where a document format of a document read by an image data generation device as a printing device is used in which each of the target item character string and the description area is surrounded by a frame will be described. Hereinafter, only the differences from each embodiment will be described.
FIG. 12 is a schematic block diagram showing a functional configuration of the image data generation device 10c according to the fourth embodiment.
The image data generation device 10c includes a communication unit 11, a control unit 15c, a storage unit 16c, an image reading unit 17, a control panel 18, and a card reader 19. The card reader 19 is attached to the outside of the image data generation device 10c.

The image reading unit 17 reads the document to be read set on the document reading table as light and dark. For example, the image reading unit 17 reads an image printed on a document to be read set on a document reading table. The image reading unit 17 outputs the read image data to the control unit 15b.
The control panel 18 includes a display unit and an operation unit. The display unit is a display device such as a liquid crystal display and an organic EL display. The display unit displays various information according to the control of the control unit 15c. The display unit 13 displays, for example, the described document image data and specific information. The operation unit includes a plurality of buttons and the like. The operation unit accepts user operations. The operation unit receives, for example, input of a correction instruction of the described document image data and a display instruction of specific information. The display unit and the operation unit may be configured as an integrated touch panel.
The card reader 19 reads the user information stored in the storage area of the certificate 1.

The control unit 15c is configured by using a processor such as a CPU or a memory. The control unit 15c realizes the functions of the image data acquisition unit 151c, the specific unit 152, the image data generation unit 153, the registration unit 154, the communication control unit 155, and the user information acquisition unit 156 by executing the program. Some of these functions (for example, image data acquisition unit 151c, specific unit 152, image data generation unit 153, registration unit 154, and user information acquisition unit 156) need to be mounted on the image data generation device 10c in advance. Instead, it may be realized by installing an additional application program in the image data generator 10c.

The image data acquisition unit 151c acquires the image data read by the image reading unit 17.
The user information acquisition unit 156 acquires the user information read by the card reader 19.

The storage unit 16c stores the user information table 161c, the collation data table 162, and the registered image data 163. The storage unit 16c is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device.
The user information table 161c is a table in which information regarding user attributes is registered. The difference between the user information table 161c and the user information table 161 in the first embodiment is that the user information is not registered in the user information table 161c in advance. The user information acquired by the user information acquisition unit 156 is registered in the user information table 161c. Therefore, user information corresponding to all attribute types may not always be registered.

FIG. 13 is a flowchart showing the flow of the image data generation process of the image data generation device 10c according to the fourth embodiment. In FIG. 13, the same processing as in FIG. 5 is designated by the same reference numerals as in FIG. 5, and the description thereof will be omitted.
The image reading unit 17 reads the document to be read set on the document reading table (step S401). The image reading unit 17 outputs the image data read from the document to the control unit 15b. The image data acquisition unit 151c acquires the image data output from the image reading unit 17 (step S402). The card reader 19 reads the user information stored in the storage area of the certificate 1 (step S403). The card reader 19 outputs the read user information to the control unit 15b. The user information acquisition unit 156 acquires the user information output from the card reader 19 (step S404). The user information acquisition unit 156 registers the acquired user information in the user information table 161b. After that, the image data generation device 10b executes the processes after step S103.

According to the image data generation device 10c configured as described above, the same effect as that of the first embodiment can be obtained.
Further, the image data generation device 10c can acquire the user information by reading the user information from the storage area of the certificate 1 with the card reader 19 even when the user information is not registered in advance. As a result, the image data generation device 10c can generate the described document image data by synthesizing the user information with the image data of the text. Therefore, it is possible to reduce the burden on the user.

<Modified example of the fourth embodiment>
In the fourth embodiment, as in the first embodiment, the case where each of the target item character string and the description area is surrounded by a frame has been described, but the image data generation device 10c has been described. As in the second embodiment, it may be applied to an object in which either or both of the target item character string and the description area are not surrounded by a frame. When configured in this way, the image data generation device 10c includes a specific unit 152a instead of the specific unit 152, and the storage unit 16c newly stores the model data 164. As a specific process, the image data generation device 10c executes the processes of steps S201, S104, and S202 shown in FIG. 7 instead of steps S103 to S105.

The image data generation device 10c may be configured to acquire user information directly input by the user via the control panel 18 or user information input by voice recognition. With this configuration, it is possible to supplement the user information that is lacking in the certificate 1. Therefore, it becomes possible to print a document in which more user information is described.

Some functions included in the image data generation device 10c may be mounted in another housing. For example, a functional unit (for example, an image data acquisition unit 151c, a specific unit 152, an image data generation unit 153, a registration unit 154, and a user information acquisition unit 156) corresponding to a described document image data generation function included in the image data generation device 10c. , User information table 161c, collation data table 162, and registered image data 163) may be implemented in another information processing device (for example, server 20). In this case, another information processing device having a function of generating the described document image data functions as the image data generation device. When configured in this way, the image data generation device 10c transmits the image data acquired by the card reader 19 to another information processing device via the network 40. Then, the described document image data generation process (process of FIG. 13) is executed in another information processing device.

(Fifth Embodiment)
In the fifth embodiment, a case where the image data generation device as the printing device acquires the user information by another method (for example, reading by the printing device) will be described. Further, in the fourth embodiment, a case where a document format of a document read by an image data generation device as a printing device is used in which an area of a description target item and a description area are each surrounded by a frame will be described. Hereinafter, only the differences from each embodiment will be described.
FIG. 14 is a schematic block diagram showing a functional configuration of the image data generation device 10d according to the fifth embodiment.
The image data generation device 10d includes a communication unit 11, a control unit 15d, a storage unit 16c, an image reading unit 17d, and a control panel 18.

The control unit 15d is configured by using a processor such as a CPU and a memory. The control unit 15d realizes the functions of the image data acquisition unit 151c, the specific unit 152, the image data generation unit 153, the registration unit 154, the communication control unit 155, and the user information acquisition unit 156d by executing the program. Some of these functions (for example, image data acquisition unit 151c, specific unit 152, image data generation unit 153, registration unit 154, and user information acquisition unit 156d) need to be mounted on the image data generation device 10d in advance. Instead, it may be realized by installing an additional application program on the image data generator 10d.

The image reading unit 17d reads the document to be read and the certificate 1 set on the document reading table as light and dark. For example, the image reading unit 17d reads the image printed on each of the document to be read and the certificate 1 set on the document reading table. The image reading unit 17d outputs the image data of the read document and the image data of the certificate 1 to the control unit 15d. The image reading unit 17d may read the document and the certificate 1 at the same time, or may read them in order.

The user information acquisition unit 156d acquires user information from the image data of the certificate 1 acquired by the image data acquisition unit 151c.

FIG. 15 is a flowchart showing the flow of the image data generation process of the image data generation device 10d according to the fifth embodiment. In FIG. 15, the same processing as in FIG. 13 is designated by the same reference numerals as in FIG. 13, and the description thereof will be omitted.
The image reading unit 17d reads the document to be read and the certificate 1 set on the document reading table (step S501). The image reading unit 17d outputs the image data read from the document and the image data read from the certificate 1 to the control unit 15d. Here, it is assumed that the image reading unit 17d has read the document and the certificate 1 at the same time.

The image data acquisition unit 151c acquires the image data output from the image reading unit 17d (step S502). The user information acquisition unit 156d recognizes the image data of the certificate 1 and the image data of the document by using the image data acquired by the image data acquisition unit 151c (step S503). Specifically, the user information acquisition unit 156d determines the area of the certificate 1 in which the user information is described from the pre-registered format, recognizes the image data of the certificate 1, and uses other documents as documents. Recognize as image data. Then, the user information acquisition unit 156d acquires the user information from the image data recognized as the image data of the certificate 1 (step S504). Specifically, the user information acquisition unit 156d acquires user information (for example, name, address, date of birth, etc.) by converting image data into text by OCR and recognizing a character string. The user information acquisition unit 156d registers the acquired user information in the user information table 161c. After that, the image data generation device 10c executes the processes after step S103.

According to the image data generation device 10d configured as described above, the same effect as that of the fourth embodiment can be obtained.
Further, the image data generation device 10d can acquire the user information by analyzing the image data of the certificate 1 even when the user information is not registered in advance. As a result, the image data generation device 10d can generate the described document image data by synthesizing the user information with the image data of the text. Therefore, it is possible to reduce the burden on the user.

<Modified example of the fifth embodiment>
In the fifth embodiment, as in the first embodiment, the case where the target item character string and the description area are each surrounded by a frame has been described, but the image data generation device 10d has been described. It may be applied to the case where at least the description area is not surrounded by a frame as in the second embodiment. When configured in this way, the image data generation device 10d includes a specific unit 152a instead of the specific unit 152, and the storage unit 16c newly stores the model data 164. As a specific process, the image data generation device 10d executes the processes of steps S201, S104, and S202 shown in FIG. 7 instead of steps S103 to S105.

The image data generation device 10d may be configured to acquire user information directly input by the user via the control panel 18 or user information input by voice recognition. With this configuration, it is possible to supplement the user information that is lacking in the certificate 1. Therefore, it becomes possible to print a document in which more user information is described.

Some functions included in the image data generation device 10d may be mounted in another housing. For example, a functional unit (for example, an image data acquisition unit 151c, a specific unit 152, an image data generation unit 153, a registration unit 154, and a user information acquisition unit 156d) corresponding to a described document image data generation function included in the image data generation device 10d. , User information table 161c, collation data table 162, and registered image data 163) may be implemented in another information processing device (for example, server 20). In this case, another information processing device having a function of generating the described document image data functions as the image data generation device. When configured in this way, the image data generation device 10d transmits the image data acquired by the image reading unit 17d to another information processing device via the network 40. Then, the described document image data generation process (process of FIG. 15) is executed in another information processing device.

(Sixth Embodiment)
In the sixth embodiment, the configuration in which the image data generation device as the printing device specifies the description area using the registered image data registered in advance will be described. The system configuration of the image data generation system 100 in the sixth embodiment is the same as the configuration of the image data generation system 100b in the third embodiment. The image data generation system 100 in the sixth embodiment includes an image data generation device 10e, a server 20, a printing device 30, and an image data registration device 50. Hereinafter, only the differences from each embodiment will be described.

FIG. 16 is a schematic block diagram showing a functional configuration of the image data generation device 10e according to the sixth embodiment.
The image data generation device 10e includes a communication unit 11e, a control unit 15e, a storage unit 16e, an image reading unit 17, a control panel 18, and a card reader 19.

The communication unit 11e communicates with the server 20 via the network 40. The communication unit 11e communicates with the image data registration device 50 directly or via the network 40. The communication unit 11e receives the registered image data from the image data registration device 50.

The control unit 15e is configured by using a processor such as a CPU or a memory. The control unit 15e realizes the functions of the image data acquisition unit 151e, the specific unit 152e, the image data generation unit 153, the communication control unit 155, and the user information acquisition unit 156 by executing the program. Some of these functions (for example, the image data acquisition unit 151e, the specific unit 152e, the image data generation unit 153, and the user information acquisition unit 156) do not need to be mounted on the image data generation device 10e in advance, and are additional. It may be realized by installing the application program in the image data generation device 10e.

The image data acquisition unit 151e acquires the image data of the document read by the image reading unit 17 and the registered image data received by the communication unit 11e. The image data acquisition unit 151e stores the acquired registered image data in the storage unit 16e.
The specific unit 152e calculates the similarity between the image data and the registered image data of one or more documents stored in the storage unit 16e, and uses the registered image data whose calculated similarity is equal to or greater than the threshold value to obtain image data. Specify the description area of.

The storage unit 16e stores the user information table 161, the collation data table 162, and the registered image data 163e. The storage unit 16e is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device.
The registered image data 163e is the registered image data acquired by the image data acquisition unit 151e.

FIG. 17 is a flowchart showing the flow of the image data generation process of the image data generation device 10e according to the sixth embodiment. In FIG. 17, the same processing as in FIG. 13 is designated by the same reference numerals as in FIG. 13, and the description thereof will be omitted. At the time of executing the process of FIG. 17, it is assumed that the registered image data is registered in advance in the storage unit 16e of the image data generation device 10e.
The specific unit 152e collates the acquired image data with the registered image data 163e stored in the storage unit 16e (step S601). Specifically, the specific unit 152e calculates the similarity between the acquired image data and the registered image data 163e stored in the storage unit 16e by image matching.

The specific unit 152e selects one registered image data from the registered image data 163e stored in the storage unit 16e based on the calculated similarity (step S602). For example, the specific unit 152e selects the registered image data having the highest degree of similarity. Then, the identification unit 152e specifies the description area of the acquired image data based on the position information of the description area included in the selected registered image data (step S603). After that, the image data generation device 10e executes the processes after step S106.

According to the image data generation device 10e configured as described above, the same effect as that of the fourth embodiment can be obtained.
Further, the image data generation device 10e identifies the description area of the acquired image data by using the position information of the description area of the registered image data similar to the acquired image data. As a result, the time required for the process of specifying the description area of the image data can be shortened. Therefore, it is possible to shorten the time required to generate the described document image data.

<Modified example of the sixth embodiment>
The image data generation device 10e may acquire user information by analyzing the image data of the certificate 1 read by the image reading unit 17 as in the fifth embodiment, or the user may press the control panel 18. It may be configured to acquire the user information directly input via the user information or the user information input by voice recognition. With this configuration, it is possible to supplement the user information that is lacking in the certificate 1. Therefore, it becomes possible to print a document in which more user information is described.

Some functions included in the image data generation device 10e may be mounted in another housing. For example, a functional unit (for example, an image data acquisition unit 151e, a specific unit 152e, an image data generation unit 153, a user information acquisition unit 156, a user information table) corresponding to a description function for generating document image data included in the image data generation device 10e. 161c, collation data table 162, and registered image data 163e) may be mounted on another information processing device (for example, server 20). In this case, another information processing device having a function of generating the described document image data functions as the image data generation device. When configured in this way, the image data generation device 10e transmits the image data acquired by the image reading unit 17 to another information processing device via the network 40. Then, the described document image data generation process (process of FIG. 17) is executed in another information processing device.

<Modification example common to the first to sixth embodiments>
The image data generation devices 10, 10a, 10b, 10c, 10d, 10e may store the generated written document image data in the storage units 16, 16a, 16b, 16c, 16e of their own device. When configured in this way, the image data generation devices 10, 10a, and 10b as communication devices transmit the described document image data to the printing device 30. Then, the printing device 30 prints the received written document image data. Further, the image data generation devices 10c, 10d, and 10e as the printing device may print the described document image data by its own device, or may transmit the described document image data to the printing device 30. When the image data generation devices 10c, 10d, and 10e transmit the written document image data to the printing device 30, the printing device 30 prints the received written document image data.

The image data generation devices 10, 10a, 10b, 10c, and 10d according to the above-described embodiment may be realized by a computer. In that case, the program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system.

Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized by using a programmable logic device such as FPGA (Field Programmable Gate Array).

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs and the like within a range that does not deviate from the gist of the present invention.

10, 10a, 10b, 10c, 10d, 10e ... Image data generator, 20 ... Server, 30 ... Printing device, 50 ... Image data registration device, 11, 11b, 11e, 51 ... Communication unit, 12, 52 ... Camera, 13, 53 ... Display unit, 14, 54 ... Operation unit, 15, 15a, 15b, 15c, 15d, 15e, 55 ... Control unit, 151, 151b, 151c, 151e, 551 ... Image data acquisition unit, 152, 152a, 152b, 152e, 552 ... Specific unit, 153 ... Image data generation unit, 154, 555 ... Registration unit, 155, 555 ... Communication control unit, 156, 156d ... User information acquisition unit, 16, 16a, 16b, 16c, 16e, 56 ... Storage unit, 161, 161c ... User information table, 162, 561 ... Collation data table, 163, 163b, 163e ... Registered image data, 164 ... Model data, 17 ... Image reader, 18 ... Control panel, 19 ... Card Reader, 554 ... Display control unit

Claims (11)

Using the image data of a document having a plurality of character strings representing the description target items for which the user information regarding the user needs to be described, a specific unit for specifying the description area of the user information with respect to the character string representing the description target item, and a specific unit.
An image data generation unit that generates described document image data by synthesizing the user information with respect to the specified description area.
An image data generator comprising. The specific unit specifies the description area for each character string representing each description target item from the positional relationship between the area of the character string representing the description target item obtained from the image data and the candidate area of the description area. , The image data generation device according to claim 1. The specific unit estimates a character string representing the description target item based on the image data, and is a model trained to output the position information of the description area of the user information with respect to the character string representing the description target item. The image data generation device according to claim 1, wherein the description area is specified with respect to the character string representing the estimated description target item. The specific unit calculates the similarity between the image data and the registered image data of one or more documents registered in advance, and uses the registered image data whose calculated similarity is equal to or higher than the threshold value to obtain the image data. The image data generation device according to claim 1, wherein the description area of the above is specified. A user information acquisition unit for acquiring the user information is further provided.
The user information acquisition unit acquires the user information from any or all of an external device, a memory attached to a certificate possessed by the user, the certificate, a storage unit, and a voice input. The image data generation device according to any one of 4 to 4. An image data generation system including the image data generation device according to claim 4 and an image data registration device for registering registered image data used in the image data generation device.
The image data registration device is
Using the image data of a document having a plurality of character strings representing the description target items for which the user information regarding the user needs to be described, a specific unit for specifying the description area of the user information with respect to the character string representing the description target item, and a specific unit.
A registration unit that associates the position information of the description area specified by the specific unit, a character string representing the description target item, and the image data and registers the registered image data in the image data generation device.
With
The identification unit of the image data generation device identifies the description area of the image data by using the image data and one or more registered image data registered by the image data registration device.
The image data generation unit of the image data generation device is an image data generation system that generates described document image data by synthesizing the user information with respect to the specified description area. The registration unit displays the image data on the screen, and sets the area of the character string representing the description target item designated by the user on the screen and the position information of the description area with respect to the character string representing the description target item. The image data generation system according to claim 6, wherein the image data is associated with the image data and registered as the registered image data in the image data generation device. In the description area of the image data, a specific character string for specifying a character string representing the description target item is described.
The registration unit acquires the position information of the specific character string and the area of the specific character string from the image data as the description area, and the acquired position information of the description area and the above-mentioned specified by the specific character string. The image data generation system according to claim 6, wherein a character string representing a description target item is associated with the image data and registered as the registered image data in the image data generation device. The specific unit of the image data registration device is the description area for the character string representing each description target item based on the positional relationship between the character string area representing the description target item obtained from the image data and the candidate area of the description area. A model trained to specify or estimate a character string representing the description target item based on the image data and output the position information of the description area of the user information with respect to the character string representing the description target item. The image data generation system according to claim 6, wherein the description area is specified with respect to the character string representing the estimated description target item. Using the image data of a document having a plurality of character strings representing the description target items for which the user information regarding the user needs to be described, the description area of the user information with respect to the character string representing the description target item is specified.
An image data generation method for generating described document image data by synthesizing the user information with respect to the specified description area. Using image data of a document having a plurality of character strings representing a description target item for which user information regarding a user needs to be described, a specific step of specifying a description area of the user information with respect to the character string representing the description target item, and a specific step.
An image data generation step of generating described document image data by synthesizing the user information with respect to the specified description area, and
A computer program that lets a computer run.

Images