KR20180075086A - Apparatus and method for supporting input based in camera and projectors - Google Patents

Abstract

The present invention relates to an apparatus for supporting a document and a method thereof. More specifically, the present invention relates to the apparatus for supporting a document by applying a camera and a projector and the method thereof. When a target document to be authored by a user is present between the camera and the projector, a format of the target document or content written by a user is recognized by the camera and information on an item to be written at the next time is provided by using the projector, thereby helping a user to author the target document. In addition, according to the present invention, the apparatus for supporting a document by applying a camera and a projector comprises the camera, the projector, and a document supporting module. The camera creates image data by capturing the target document. The projector is to project and provide information on the target document to a region for displaying information on the item to be written at the next time. The document supporting module decides a format of the target document by using the image data of the target document recognized by the camera, checks whether to write the target item to decide an item to be written at the next time, and provides the information to the region for displaying the information on the item to be written next time through the projector.

Description

Technical Field [0001] The present invention relates to a document supporting apparatus and a method for supporting a camera and a projector,

The present invention relates to a document supporting apparatus and a method thereof, and particularly relates to a document supporting apparatus and a method thereof, in which when a target document to be created by a user exists between a camera and a projector, The present invention relates to a document supporting apparatus and a method of applying a camera and a projector that assist in creating a target document by providing guidance information to an item to be described in the document.

Generally, if you have to manually enter information into a document that has a form such as an application that you are not familiar with, it is difficult to know which item should be entered and which item should be omitted.

In addition, depending on such documents, the type and number of items to be input at the time of selecting a specific item may vary.

Because of these difficulties, it is often necessary to have an assistant to assist with the input or to spend a lot of time and effort on the author.

To address this difficulty, domestic registration number 10-1673509 (prior document 1) discloses devices, methods and graphical user interfaces for document manipulation.

Such prior art document 1 displays at least a portion of an electronic document having a predefined page layout at a first magnification level on the display; Detecting a first input indicative of a first insertion point of the document, the first insertion point being proximate to a first portion of text in the document; And in response to detecting the first input: selecting a second magnification level different from the first magnification level, the second magnification level being selected to display a first portion of text in a target text display size, While maintaining the defined page layout, at the second magnification level, a portion of the document containing the first portion of the text is displayed.

Next, Korean Laid-Open Publication No. 10-2015-0046939 (Prior Art 2) discloses an electronic document system that provides an electronic form creation order and a processing method thereof.

This prior art document 2 is an electronic document system that is legally valid by using an electronic document instead of a paper document in a face-to-face transaction at a financial institution window and provides an electronic form creation order that simplifies complicated tasks by providing an electronic form creation order And the banking terminal selects the item to be created by the customer on the form provided by the operator terminal as a display member in the order of creation, The second terminal reads out the order of creation of the items stored in the operator terminal and activates the format in the form of a pop-up window according to the order of creation of the items .

Japanese Laid-Open Patent Publication No. 2012-98773 (prior art document 3) discloses a terminal having an input support function, an input support system, an input support method, and a program.

In this prior art document 3, the terminal device acquires the time information of the information that the user intends to input, determines an optimal database based on the acquired time information, predicts the information that the user intends to input based on the database, So that the user can perform the information input operation efficiently and accurately.

The above-mentioned prior art document 1 displays a part of an electronic document, detects a first input indicating a first insertion point of a document, and then displays a second insertion point, To the general document There is a problem that it is difficult to target.

In the preceding document 2, the order of creation of the items is read out, and the forms are activated in the form of a pop-up window in the order of creation of the items, assisting the creation of documents, which is also difficult to apply to general documents.

Also, in the prior art document 3, the terminal device acquires the time information of the input information of the user and determines the optimal database based on the acquired time information, which is also difficult to apply to general document creation.

Korean Patent Registration No. 10-1673509 Korean Patent Publication No. 10-2015-0046939 Japan Special Feature 2012-98773

SUMMARY OF THE INVENTION The present invention has been conceived to solve the problems as described above, and it is an object of the present invention to provide a method and apparatus for recognizing a target document form or a description of a user using a camera when a target document to be created by a user exists between a camera and a projector, The present invention provides a document supporting apparatus and method in which a camera and a projector for assisting a user in creating a document by providing guide information of an item to be described next.

According to an aspect of the present invention, there is provided an image processing apparatus including: a camera for photographing a target document to generate image data; A projector for projecting and providing guide information to a guide information display area of an item to be described next of the target document; And a controller for confirming a target document form by using the image data of the target document recognized by the camera, confirming whether or not the target item is written, confirming the next target item, And a document support module for providing guidance information through the document support module.

In addition, the document support module of one aspect of the present invention provides highlight information as guide information in a guide information display area of the item to be described next.

Further, the document support module of one aspect of the present invention provides the description method guide information as guide information in the guide information display area of the next description target item.

Further, the document support module of one aspect of the present invention provides an example of presentation information as guide information in a guide information display area of the next target entry.

Further, the guide information display area determined by the document support module according to an aspect of the present invention is characterized in that it is a comment field of a next target entry item.

Further, the guide information display area determined by the document support module according to an aspect of the present invention is characterized by being the end of the description column of the item to be listed that has been completed immediately before.

According to an embodiment of the present invention, the document support module recognizes a character and a style line in image data of a photographed image taken by the camera, A document format determination unit for determining the type of the target document format based on the character and the format line recognized by the recognition unit by referring to the format, keyword, and title division value registered in the database; A sub-item confirmation unit for confirming a target item in which the description is written by the user based on the character recognized by the recognition unit by referring to the target document form determined by the document form determination unit; A target document format type determined by the document format determination unit and an item to be recorded confirmed by the item item identification unit are confirmed and a next item to be recorded in the corresponding document format is determined, A display area determination unit for determining a location of the area; The type of target document form determined by the document form determination unit and the item to be listed confirmed by the item identification unit are specified and guide information to be displayed in the guide information display area of the next item to be listed in the corresponding object document form is specified Display contents specifying unit; And a projector control unit for controlling the projector to display guide information in the guide information display area determined by the display area determination unit.

According to another aspect of the present invention, there is provided an image processing method comprising the steps of: (A) photographing a target document to generate image data; (B) confirming a target document format using the image data of the target document recognized by the camera, and confirming whether to write the target item; (C) identifying the guide information display area of the item to be listed after the document support module confirms the item to be described next; And (D) projecting and providing the guide information to the guide information display area of the next entry of the target document in the target document.

The step (B) of the other aspect of the present invention may further comprise: (B-1) recognizing characters and format lines in the image data of the photographed image photographed by the camera; (B-2) determining the type of the target document form based on the recognized character and the form line with reference to the form, keyword, and title distinguishing value registered in the database by the document supporting module; And (B-3) confirming whether or not a description target item for confirming a target item in which the description content is written by the user based on the recognized character with reference to the target document form determined by the document support module .

In the step (C) of the other aspect of the present invention, the step (C-1) further comprises the step of: (C-1) receiving, by the document support module, Establishing; And (C-2) the document support module establishing a position of the guidance information display area associated with the next entry.

Further, the guide information display area determined by the document support module in the step (C) of the other aspect of the present invention is characterized in that it is a comment field of a next target entry item.

Further, the guidance information display area determined by the document support module in the step (C) of the other aspect of the present invention is characterized by being the end of the description column of the item to be listed which has been completed immediately before.

Further, in the step (D) of the other aspect of the present invention, the projector provides highlight information as guide information in a guide information display area of the item to be described next.

Further, in the step (D) of the other aspect of the present invention, the projector provides the description method guide information as guide information in the guide information display area of the next description target item.

Further, in the step (D) of the other aspect of the present invention, the projector provides an example of presentation information as guide information in the guide information display area of the next description target item.

In the conventional method, a detailed description should be written on the application form so that the author can read it and understand it, or prepare the sample after preparing the sample in case the guide is in the residence. .

Compared to this conventional technology, the present invention provides comfort to authors and economies of manpower reduction because it does not require comprehension or unnecessary manpower.

In addition, since it is not necessary to prepare various samples in advance, the proposed invention can be utilized in a narrow space.

FIG. 1 is a configuration diagram of a document supporting apparatus to which a camera and a projector according to a preferred embodiment of the present invention are applied.
2 is an exemplary view of a target document to which the present invention is applied.
Fig. 3 is an example in which a projector is used to display highlight information as guide information in a guide information display area of an item to be described next.
4 is an exemplary diagram showing a state after the user completes the description of the description content in the item to be listed.
Fig. 5 shows an embodiment in which a description column of a next-to-be-described item to be described next is displayed in a highlight after the user finishes describing the description in the item to be listed.
Fig. 6 is an exemplary diagram after the user completes the description of the content in the description target item; Fig.
Fig. 7 shows an embodiment in which the next item to be described is displayed in a highlight after the user finishes describing the description in the item to be listed.
8 is a detailed configuration diagram of the document support module of FIG.
9 is a flowchart of a method of supporting a document using a camera and a projector according to a preferred embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments will be described in detail below with reference to the accompanying drawings.

The following examples are provided to aid in a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, this is merely an example and the present invention is not limited thereto.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. The following terms are defined in consideration of the functions of the present invention, and may be changed according to the intention or custom of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification. The terms used in the detailed description are intended only to describe embodiments of the invention and should in no way be limiting. Unless specifically stated otherwise, the singular form of a term includes plural forms of meaning. In this description, the expressions "comprising" or "comprising" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, Should not be construed to preclude the presence or possibility of other features, numbers, steps, operations, elements, portions or combinations thereof.

It is also to be understood that the terms first, second, etc. may be used to describe various components, but the components are not limited by the terms, and the terms may be used to distinguish one component from another .

FIG. 1 is a configuration diagram of a document supporting apparatus to which a camera and a projector according to a preferred embodiment of the present invention are applied.

Referring to FIG. 1, a document supporting apparatus using a camera and a projector according to a preferred embodiment of the present invention includes a camera 10, a projector 20, a document support module 30, and a database 40.

The camera 10 photographs a target document placed on a table or the like, and forms and outputs image data.

Here, the object document may include various objects that require electronic document sorting and classification through character recognition as well as various receiving and receiving form documents including an application form, an invoice, an invoice, a declaration form, a quote form, and an order form.

Then, the camera 10 transmits the image data of the photographed image to the document support module 30.

Then, the document support module 30 recognizes the character and the form line of the target document, grasps the type of the target document form, recognizes whether or not the description content of the target item is described, After identifying the item, the next item to be written next is identified, and the guide information is projected through the projector 20 in the guide information display area of the next item to be described.

Here, the document support module 30 may simply display the highlight in the comment field of the next-to-be-listed item, provide the description method guidance information, or provide a description example as guide information.

The guide information display area of the next-to-be-specified item designated by the document support module 30 may be the comment field of the next-to-be-listed item or may be the end of the to-be-listed to-be-completed item.

Here, when the guide information display area of the next-to-be-specified item designated by the document support module 30 is the end of the immediately preceding completed item to be listed, the guide information is provided rather than the highlight display, It is better to provide an example.

On the other hand, the projector 20 causes the guide information to be projected in the guide information display area of the next description target item under the control of the document support module 30. [

Here, as the guide information projected by the projector 20, it is possible to simply display the highlight in the comment field of the next-to-be-listed item, provide the information on the way-by-place method, or provide a description example.

The database 40 stores various target document formats, and stores a description order of items to be written in each target document format.

That is, the database 40 registers and manages format, keyword, and title discrimination value in advance in order to confirm the type of target document form.

Accordingly, the document support module 30 recognizes the character and the form line in the image data of the photographed image photographed through the camera 10, compares the character and the form line with various target document forms stored in the database 40, I confirm it.

In addition, the document support module 30 recognizes the description characters of the user in the image data of the photographed image photographed through the camera 10, identifies which entry is being written in the target document form in the target document format, The item to be described next is determined with reference to the description order of the description item of the target document format stored in the item information storage unit 40, so that the guide information is provided to the next item to be described using the projector 20.

The operation of the camera and the document supporting apparatus using the projector according to the present invention will be described below.

When the user creates a target document having a format as shown in Fig. 2, the document support module 30 uses the camera 10 to capture the target document and generates image data of the photographed image.

Then, the document support module 30 confirms the target document form type and confirms the description content of the description target item, and then, using the projector 20 as shown in Fig. 3, Highlight the guide information.

4, the document support module 30 confirms the next item to be described next in accordance with the content, and then, in the same manner as in FIG. 5, Display.

At this time, as shown in FIG. 6, when the user finishes describing the item to be listed, the document support module 30 confirms the next item to be described next, and displays it as shown in FIG. 7 .

8 is a detailed configuration diagram of the document support module of FIG.

1, the document support module includes a recognition unit 31, a document format determination unit 32, a description item confirmation unit 33, a display area determination unit 34, a display content specification unit 35, And a projector control unit 36.

The recognition unit 31 performs character recognition and format line recognition on the image data of the photographed image photographed by the camera 10.

In addition, the recognition unit 31 extracts coordinates of image pixels of the character and the form line.

The character recognition refers to a technique of optically or electrically detecting various characters and automatically converting them into codes that a device such as a computer can understand.

In general, the processing steps of character recognition are: 1. Pre-processing, 2. Document structure analysis / registration, 3. Character separation, 4. Pattern matching, 5. Post-processing Lt; / RTI >

Preprocessing is the process of correcting the tilts of optically or electrically converted characters, or removing shadows such as shadows.

Document structure analysis / registration refers to the task of separating or ordering images and letters (numbers), which are components of a document, or defining a document format.

Character separation is a process of extracting images of each character string, separating them by line or word, and separating each individual character again. Pattern matching refers to the operation of extracting candidate characters and calculating reliability by classifying the separated character images into models with the most similar characteristics using a pattern matching algorithm (Algorithm).

Finally, post-processing refers to the task of correcting recognition errors by word or research based on a dictionary or by cross-reference.

Of course, in addition to the character recognition method described above, various character recognition technologies have been developed and used, and various document recognition products have been developed at home and abroad and commercialized. The character recognition technology according to an embodiment of the present invention includes all character recognition technologies that are currently being developed and commercialized.

The recognition unit 31 includes a character recognizer (not shown) for recognizing a line unit character, a format line recognizer (not shown) for recognizing the format line, a coordinate extractor (Not shown).

The document format determination unit 32 determines the type of the target document format by classifying the format type according to the format, keyword, and title discrimination value registered in the database 40.

The document format determination unit 32 includes a format mapper (not shown) for mapping input image data with a pre-registered image format, a keyword filter (not shown) for filtering the image data, A title filter (not shown) for filtering the title of image data and image data registered in advance, metadata extraction of image data through format recognition of a form mapper, a keyword filter, and a title filter, And a document classifier (not shown) that processes the classification.

On the other hand, the description item confirmation unit 33 refers to the target document form determined by the document form determination unit 32 using the character recognized by the recognition unit 21, and confirms the description target item described by the user.

Next, the display area determination unit 34 receives the target document format type confirmed by the document format determination unit 32 and the description target item confirmed by the description item confirmation unit 33, The target item is confirmed, and the position of the guide information display area associated with the next description target item is determined.

Then, the display content specifying unit 35 receives the target document format type confirmed by the document format determination unit 32 and the target item to be identified confirmed by the description item confirmation unit 33, The highlight information to be displayed in the guide information display area of the item, the description method information and the description example are specified.

The projector control unit 36 controls the projector 20 to display the highlight information as guide information in the guide information display area determined by the display area determination unit 34 or displays a specific example in the display content specifying unit 35 Way guide information can be displayed.

Of course, the display area determining unit 34 determines the end of the to-be-recorded target item as the guide information display area, and when the display content specifying unit 35 specifies the guide information, the projector control unit 36 The guide information can be displayed in the guide information display area at the end of the item to be listed which has been completed.

9 is a flowchart of a method of supporting a document using a camera and a projector according to a preferred embodiment of the present invention.

Referring to FIG. 9, in a document supporting method using a camera and a projector according to an embodiment of the present invention, a camera captures an object document placed on a table, etc., and forms and outputs image data (S100).

Here, the object document may include various objects that require electronic document sorting and classification through character recognition as well as various receiving and receiving form documents including an application form, an invoice, an invoice, a declaration form, a quote form, and an order form.

Then, the camera transmits the image data of the photographed image to the document support module.

Then, the document support module performs character recognition and format line recognition on the image data of the photographed image photographed by the camera, compares the recognized character and the format line with the format, keyword, and title discrimination values stored in the database, The type of the form is determined (S110).

Next, the document support module refers to the determined target document form using the recognized character, and confirms the description target item described by the user (S120).

Next, the document support module confirms the next-to-be-listed item of the corresponding target document form based on the confirmed target document form and the confirmed to-be-written target item, and confirms the guide information display area of the next to-be-listed item (S130).

Then, the document support module receives the confirmed target document form and the identified target item to be displayed, and displays the highlight information in the guide information display area of the next target item in the corresponding target document form, the display contents such as the description example, (S140).

Thereafter, the document support module controls the projector to display the highlight in the determined display area, or controls the projector to display a specific description example or description method guide information.

Of course, the document support module may determine the end of the item to be listed which is currently completed as the guide information display area, and display the guide information in the guide information display area at the end of the item to be listed have.

On the other hand, in the conventional method, a detailed description is described in the application form, and after the author reads and understands it, it is written, or the guide is in the residence so that the author can help the author, The method to make it possible is mainly used.

Compared to this conventional technology, the present invention provides comfort to authors and economies of manpower reduction because it does not require comprehension or unnecessary manpower.

In addition, since it is not necessary to prepare various samples in advance, the proposed invention can be utilized in a narrow space.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments of the present invention are not intended to limit the scope of the present invention but to limit the scope of the present invention. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as being included in the scope of the present invention.

10: camera 20: projector
30: document support module 31: recognition unit
32: document form determination unit 33: description item confirmation unit
34: Display area determination unit 35: Display content specification unit
36: Projector control unit 40:

Claims (15)

A camera for photographing a target document to generate image data;
A projector for projecting and providing guide information to a guide information display area of an item to be described next of the target document; And
The target document form is confirmed by using the image data of the target document recognized by the camera, and whether or not the item to be listed is described is confirmed, and the next item to be listed is determined and determined. And a document support module for providing guide information through a camera and a projector. The method according to claim 1,
Wherein the document support module applies a camera and a projector to provide highlight information as guide information in a guide information display area of an item to be described next. The method according to claim 1,
Wherein the document support module applies a camera and a projector to provide the description method guide information as guide information in a guide information display area of an item to be described next. The method according to claim 1,
Wherein the document support module applies a camera and a projector to provide an example of description as guide information in a guide information display area of an item to be described next. The method according to claim 1,
And the guide information display area determined by the document support module is a comment field of a next target entry. The method according to claim 1,
Wherein the guide information display area determined by the document support module is at the end of a description column of an item to be listed which has been completed immediately before. The method according to claim 1,
The document support module
Recognizing a character and a form line in image data of a photographed image photographed by the camera,
A document format determination unit for determining the type of the target document format based on the character and the format line recognized by the recognition unit by referring to the format, keyword, and title division value registered in the database;
A sub-item confirmation unit for confirming a target item in which the description is written by the user based on the character recognized by the recognition unit by referring to the target document form determined by the document form determination unit;
A target document format type determined by the document format determination unit and an item to be recorded confirmed by the item item identification unit are confirmed and a next item to be recorded in the corresponding document format is determined, A display area determination unit for determining a location of the area;
The type of target document form determined by the document form determination unit and the item to be listed confirmed by the item identification unit are specified and guide information to be displayed in the guide information display area of the next item to be listed in the corresponding object document form is specified Display contents specifying unit; And
And a projector control unit for controlling the projector to display guide information in the guide information display area determined by the display area determination unit. (A) photographing a target document by a camera to generate image data;
(B) confirming a target document format using the image data of the target document recognized by the camera, and confirming whether to write the target item;
(C) identifying the guide information display area of the item to be listed after the document support module confirms the item to be described next; And
(D) projecting and providing guide information to a guide information display area of a next target entry of the target document by the projector. The method of claim 8,
The step (B)
(B-1) the document support module recognizing characters and format lines in the image data of the photographed image photographed by the camera;
(B-2) determining the type of the target document form based on the recognized character and the form line with reference to the form, keyword, and title distinguishing value registered in the database by the document supporting module; And
(B-3) confirming whether or not the item to be described for confirming the target item in which the description content is written by the user based on the recognized character with reference to the target document form determined by the document support module And a document supporting method using a projector. The method of claim 8,
The step (C)
(C-1) confirming the next target entry item of the target document format by inputting the target document format type and the confirmed target entry item, which are determined by the document support module; And
(C-2) The document supporting method includes a step of determining a position of a guide information display area related to an item to be described next. The method of claim 8,
Wherein the guide information display area determined by the document support module in the step (C) is a comment field of a next-to-be-listed target item. The method of claim 8,
Wherein the guidance information display area determined by the document support module in the step (C) is at the end of a description column of the subject matter to be described immediately before. The method of claim 8,
Wherein the projector provides the highlight information as guide information to a guide information display area of an item to be described next in the step (D). The method of claim 8,
In the step (D), the projector applies the camera and the projector to provide the description method guide information as the guide information in the guide information display area of the next description target item. The method of claim 8,
In the step (D), the projector applies a camera and a projector to provide an example of presentation as guide information in a guide information display area of a next target entry.

Images