JP2019120983A - Imaging control device and imaging control method - Google Patents

Abstract

To provide an imaging control device capable of acquiring a picked-up image obtained by reducing image distortion.SOLUTION: An imaging control device includes: an image enlargement part for enlarging an area including a center part of an image acquired from a camera for imaging a business form to display the area on a display screen; and an image acquisition part for acquiring an image which is enlarged and displayed and in which an area of a character recognition object of the business form is fit into the display screen as a picked-up image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging control apparatus and an imaging control method.

  Perform various application procedures electronically by using imaging data obtained by imaging various certificates such as a driver's license with a mobile phone or a smartphone and character information obtained by character recognition of the imaging data Has been done. An example of an image management apparatus described in Patent Document 1 is one that is imaged by a camera of a mobile phone and character recognition is performed.

JP, 2011-070601, A

However, in general, a lens capable of imaging at a wide angle of view is used as a lens used for a camera of a mobile phone or a smart phone. For this reason, although there is little distortion of the image in the vicinity of the center to be imaged, the image is distorted as it is influenced by the characteristics of the lens as it gets farther from the center of the imaging region. For example, a captured image is distorted in a so-called barrel shape on the peripheral side under the influence of lens distortion.
Then, since the characters imaged at a position away from the center of the imaging area become an image in a distorted state, the characters can not be recognized correctly if the image is subjected to an OCR (optical character recognition) process, It may be misrecognized.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an imaging control apparatus and an imaging control method capable of obtaining a captured image in which distortion of an image is reduced.

In order to solve the problems described above, according to the present invention, there is provided an image enlargement unit for enlarging a region including a central portion of an image obtained from a camera for imaging a form and displaying the region on a display screen; And an image acquisition unit that acquires, as a captured image, an image in a state where the character recognition target area of the form fits on the display screen.
Further, the present invention is the imaging control method in the terminal device, wherein the image enlargement unit enlarges an area including the central portion of the image obtained from the camera for imaging the form, and displays the region on the display screen. According to another aspect of the present invention, there is provided an imaging control method for acquiring, as a captured image, an image that is displayed in an enlarged manner and in which an area for character recognition of the form fits within the display screen.

  As described above, according to the present invention, an area including the central portion of an image obtained from a camera for imaging a form is enlarged and displayed on the display screen, and an image in a state where the character recognition target area fits on the display screen To get the As a result, it is possible to cut out an area including the area for character recognition which is the center of the imaging area and includes the area for character recognition, so that the image on the outer peripheral side of the imaging area is not used and image distortion is reduced. The captured image can be obtained. Since the captured image with reduced distortion of the image can be obtained, the character recognition rate can be improved when the character recognition process is performed.

It is a schematic block diagram which shows the structure of the slip data management system 1 by one Embodiment of this invention. It is a conceptual diagram explaining the concept of the picture picturized. It is a conceptual diagram explaining the concept of the picture picturized. It is a conceptual diagram explaining the concept of the picture picturized. FIG. 6 is a view showing an example of a screen displayed on a display screen of the terminal device 10; 5 is a flowchart illustrating an operation of the terminal device 10;

Hereinafter, a form data management system using an imaging control apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a form data management system 1 according to an embodiment of the present invention.
In the form data management system 1, a plurality of terminal devices 10, a form data management apparatus 20, and an OCR server 30 are connected via a network 40.

  The terminal device 10 is, for example, a smartphone or a mobile phone, and performs imaging of a form by connecting to a service providing server connected to a network or executing a predetermined application downloaded from the service providing server. The imaging data can be generated and transmitted to the form data management device 20, and procedures such as opening a bank account, applying for various insurances, and registering members of various services can be performed electronically. Hereinafter, although the case where one terminal device 10 is provided will be described, a plurality of terminal devices 10 may be provided.

  Also, here, the slips may be, for example, a document issued by a public organization such as a driver's license, a health insurance card (health insurance card) or a resident card, or a document designated as usable in performing the above-mentioned service. For example, a certificate of payment for electricity, gas, water, etc., credit payment statement, etc.), and the name and address of the user who applies for the procedure is described, and it is possible to prove that these names and addresses are correct. It is a document. By imaging such a form, performing character recognition processing by the OCR server 30, and acquiring character information from the form, various application procedures can be performed electronically by using the character information. For example, by using this character information, the user does not have to input all the name, address, etc. by oneself, and if there is no mistake in confirming the obtained character information, the form data management device 20 is transmitted. Can proceed with various application procedures. Further, the user's input information such as the user's name is correct by determining whether the character string of the name or the like input through the touch panel or the keyboard matches the character information of the character recognition processing result. It can also be determined.

The terminal device 10 is, for example, a mobile phone, a smartphone, or a tablet, and includes a camera 101, a display unit 102, an image enlargement unit 103, an image acquisition unit 104, and an imaging guide frame display control unit 105. Communication function.
The camera 101 receives light by an imaging device (for example, a CCD (solid-state imaging device)) through a lens that is an optical system, and generates imaging data. Generally, in mobile phones and smartphones, models that do not have an optical zoom function and models that have an optical zoom function, or a wide-angle lens and a telephoto lens are provided according to the imaging target. There is a model which can switch to a lens and can image. The camera 101 may or may not have an optical zoom function, and may or may not have a function of switching between a wide-angle lens and a telephoto lens for imaging. The optical system may be a camera having a single lens without an optical zoom function and a fixed angle of view.
The display unit 102 is, for example, a liquid crystal display device or a touch panel, and displays various types of information.

The image enlargement unit 103 enlarges an area including the central part of an image obtained from a camera that captures a form, and causes the display screen of the display unit 102 to display the enlarged area.
The image acquisition unit 104 acquires, as a captured image, an image which is enlarged and displayed and in which the character recognition target area of the form fits on the display screen.
The imaging guide frame display control unit 105 acquires information representing the screen size of the display screen or model information of the terminal device having the display screen, identifies the screen size of the display screen based on the acquired information, and is identified. Display an imaging guide frame of a size corresponding to the screen size on the display screen.

  Here, FIGS. 2 to 4 are conceptual diagrams illustrating the concept of an image to be captured. When imaging a form (for example, a driver's license) by a general method, the user places the form on a table such as a desk, and the user himself sits in a chair, and the height of the chest of the terminal device 10 By holding the terminal device from the driver's license, the terminal device is held at a position about ten centimeters away and imaged. If the driver's license is imaged in such a state, for example, as shown in FIG. 2, the entire driver's license (code 201) fits on the display screen (code 200), and the outer peripheral portion of the driver's license The image is taken slightly apart from the end of the. In such a case, depending on the characteristics of the lens of the camera that the terminal device has, it is affected by distortion, for example, as indicated by a curve 202 in the longitudinal direction and the lateral direction indicated by reference numeral 202, compared to the center of the imaging target The image gets distorted as it gets closer to the outer periphery of the image (so-called barrel distortion). When imaging data is obtained by releasing the shutter in such a state, the character recognition processing can be performed with high accuracy because distortion of the central portion of the imaging region (for example, the region indicated by reference numeral 203) is small. . On the other hand, outside of the central part of the image, the influence of distortion is greater than that of the central part, so the image of the driver's license is distorted, so even if character recognition processing is performed, the character recognition accuracy However, there is a possibility that misrecognition or character recognition itself can not be made.

  Next, for example, as shown in FIG. 3, the influence of distortion is reduced by imaging so that the driver's license (code 301) fits in the center part (code 303) of the imaging area of the display screen (code 300). It is possible to do. As a result, it is possible to image the driver's license in the region with less distortion among the vertical and horizontal curves indicated by reference numeral 302. However, when imaging is performed so that the driver's license falls within the center of such an imaging area, it is necessary to separate the terminal device from the driver's license (to be apart from the imaging distance in the example of FIG. 2) and perform imaging. . In this case, the terminal device is not lifted up to the chest level, but is lifted up to the height of the user's own face or overhead. Such a posture is a posture that is difficult for the user to shoot. Also, as the terminal device is lifted higher, for example, if the terminal device is lifted to a position higher than the eye level, the display screen of the terminal device can not be viewed, so the driver's license is properly in the imaging area range. There is a problem that it can not be confirmed whether the subject is in focus or in focus. In addition, the user looked at the display content of the display screen as to whether the height of the terminal device can be lifted to reduce the influence of distortion, and whether the character recognition rate can not be reduced. I can not judge by myself.

Therefore, as shown in FIG. 4, the terminal device 10 in the present embodiment enlarges and displays the vicinity of the central portion of the imaging region imaged by the camera 101 by performing so-called digital zoom, while widening the wide angle of the camera 101 most. When the shutter is released, the magnified image is acquired as imaging data. As a result, a part of the image obtained through the lens, including the central part of the imaging region, can be cut out and obtained as imaging data, and the image in the vicinity of the outer periphery of the imaging region has a large distortion. It can not be done.
Furthermore, since enlarged display is made on the display unit 102, by taking a posture to lift the terminal device 10 to such an extent that the driver's license falls within the display screen, the document can be contained within the range of less distortion. You can easily grasp how high you can lift it.
Further, by enlarging and displaying the image with such a magnification, the image of the form can be stored so as not to protrude from the display screen of the terminal device 10, and the image can be taken in a reasonable posture.

Here, the imaging guide frame display control unit 105 displays the imaging guide frame on the screen displayed in an enlarged manner. FIG. 5 is a view showing an example of a screen displayed on the display screen of the terminal device 10. In this figure, the enlarged display screen of the display unit 102 of the terminal device 10 has a size not in contact with the outer periphery of the display screen, and a predetermined distance from the end of the display area of the display screen toward the center A substantially rectangular imaging guide frame (reference numeral 110) set to is displayed. The user adjusts the position of the terminal device 10 in the horizontal direction and the position in the height direction so that the driver's license conforms to the imaging guide frame, and the outer circumferences of the imaging guide frame and the driver's license approximately match. By releasing the shutter, imaging data can be generated.
Here, as the size of the imaging guide frame is closer to the size of the display screen, the distance to the form can be shortened. That is, since the user can take such a posture as to bring the terminal device 10 close to the form so that the form is imaged largely, it is possible to perform imaging without raising too high.
The shape of the imaging guide frame is the case where the imaging target is known in advance, and when the outline shape of the form is rectangular, the shape of the imaging guide frame having an aspect ratio substantially similar to the aspect ratio of the outline of the form. It can be done. As a result, for the user, there is an advantage that the outline of the form can be easily aligned with the imaging guide frame.

Here, the magnification at the time of the enlargement display is a magnification that is enlarged when the captured image is displayed on the screen, and can be set arbitrarily. For example, a user with a height of about 140 cm to 190 cm places a certificate on a desk while sitting in a chair, holds the terminal device at a height that does not lift too much while sitting on the chair, and captures an image Conditions of magnification that can be imaged in a stress-free posture while visually recognizing a form within the guide frame, and can be imaged without erroneous recognition when character recognition processing is performed on imaged data Is about 1.5 times to about 3 times, preferably about 1.5 times to about 2.5 times. The distance from the form to the terminal at this time is about 18 to about 20 cm, and the height is a height between the chest height of the user and the height of the jaw. As a result, the terminal device 10 is lifted in the height direction as compared with the general imaging method, but the form can be imaged in a reasonable posture. Even if the terminal device 10 is lifted to the height of the nose or eyes, the form is displayed so as not to fit on the screen. Therefore, the user can set the height of the terminal device 10 so that the form fits in the display screen. Will be lowered. Therefore, the user is not required to lift the terminal device 10 to a position higher than necessary.
By enlarging the display with such a magnification, the display screen is not taken because it does not take the attitude of lifting the terminal device 10 away from the form more than necessary (lifting the terminal device 10 overhead, standing up from a chair, etc.) The shutter can be released in a state where it is easy to see. In addition, since the display screen is easy to view, it is possible to release the shutter after confirming whether or not the focus is on.

Here, it is also possible to capture a form by using an image scanner and recognize characters. In the case of using an image scanner, in order to obtain an image by scanning an imaging target, distortion of the image is less likely to occur than in the case of imaging using a camera provided in a terminal device. Therefore, the character recognition process can be performed with high accuracy, and the problem in the case of using the camera of the terminal device as described above hardly occurs.
In addition, the image scanner exists as a device separate from the terminal device, and the penetration rate is lower than that of a smartphone or a mobile phone, and therefore it can not be used by a user who does not own the image scanner. When an image scanner is used, it can not be used easily, for example, it can not be used outside the office, as compared to the case where an image is taken with a camera of a terminal device.

  In addition, depending on the lens characteristics and individual differences of the lens, the distortion of the image is not necessarily symmetrical between the left side and the right side (or upper side and lower side) of the imaging area. It is difficult to use the image at the upper and lower ends etc. as a target of character recognition processing.

  Returning to FIG. 1, the form data management device 20 is connected to the network 40 as a Web server, for example. The form data management device 20 receives the imaging data transmitted from the terminal device 10, and stores the data together with the identification information of the transmission source terminal device 10 or the user who transmitted the data. Further, the form data management device 20 further receives the character information which is the result of the character recognition processing transmitted from the OCR server 30, and stores the character information in association with the imaging data of the target subjected to the character recognition processing. You can also. This character information includes, for example, text information generated by character recognition of character information included in a partial image, and, for example, a character string representing a user's name, address, postal code, etc. is used. .

The OCR server 30 has a function of receiving imaging data transmitted from the terminal device 10, performing character recognition processing on imaging data, and acquiring character information. In addition, the OCR server 30 transmits the generated character information to the form data management device 20.
The network 40 is the Internet, a LAN (local area network), or the like, and may have a wireless section in part.
In the following embodiment, although character recognition processing based on imaging data will be described as being performed by the OCR server 30, the function of character recognition processing is provided in the terminal device 10, and the terminal device 10 performs character recognition processing. It can also be done.

Next, the operation of the terminal device 10 in the above-described form data management system 1 will be described. FIG. 6 is a flowchart for explaining the operation of the terminal device 10.
The terminal device 10 determines whether to activate the form mode based on whether or not an instruction to activate the form imaging mode has been input from the user via the touch panel or the like (step S101). When the start instruction of the form imaging mode is not input, step S101 is repeated and determined every constant time. On the other hand, when it is detected that an instruction to activate the form imaging mode is detected, the terminal device 10 activates the form imaging mode.
When the document photographing mode is activated, the terminal device 10 changes the camera 101 so that the wide angle of the camera 101 is wide (for example, the field angle is widest) and then performs imaging (step S102). For example, if the terminal device 10 is a model having an optical zoom function, the state of the optical zoom is changed to the widest angle, and if the model has both a wide-angle lens and a telephoto lens, Select to use a wide-angle lens for imaging. If the terminal device 10 does not have the function of changing the wide angle optically when the optical zoom function or the function of switching between the wide-angle lens and the telephoto lens is not provided, the process of step 102 is skipped and imaging is performed. .

  Next, the image enlargement unit 103 of the terminal device 10 sets the magnification of the enlargement display (step S103). If the magnification of this enlarged display is determined in advance, the setting value for determining the magnification is read out.

Next, the imaging guide frame display control unit 105 of the terminal device 10 detects the screen size (step S104). For example, the imaging guide frame display control unit 105 acquires information representing the screen size of the display screen or model information of the terminal device including the display screen, and identifies the screen size from the obtained information. The information indicating the screen size and the model information of the terminal device are stored, for example, inside the terminal device 10 in a storage area that stores specifications of the terminal device 10 and the like. The imaging guide frame display control unit 105 specifies the screen size by referring to this storage area. When the model information of the terminal device is obtained, the information on the screen size corresponding to the model is inquired to an external server or the like, or the table information in which the model and the screen size are associated is The screen size can be specified by storing it in an internal storage area and referencing this table. For example, the size of the shooting guide frame is determined in advance for each model, and the shooting guide frame corresponding to the model or screen size is determined in advance in response to the model or screen size of the terminal device 10 being specified. It can be selected and displayed.
Note that the screen size of the terminal device 10 is specified based on the model information and the information indicating the screen size, and the shooting guide frame is enlarged or reduced to correspond to the specified screen size (for example, inscribed) You may do it.

Next, when the screen size is specified, the imaging guide frame display control unit 105 displays the display size of the imaging guide frame so that the size corresponds to the set magnification of the enlarged display based on the specified screen size. Are determined (step S105).
When the display size of the imaging guide frame is determined, the image enlargement unit 103 enlarges the image captured by the camera 101 according to the set magnification and causes the display unit 102 to display the image, and the imaging guide frame display control unit In step 105, the imaging guide frame is superimposed and displayed on the enlarged screen according to the determined display size.
The image acquisition unit 104 of the terminal device 10 determines whether or not the shutter button is pressed (step S107). If the shutter button is not pressed, the determination of step S107 is repeated again after a predetermined time, and the shutter button is pressed. In the case where it is determined, the imaging data is generated by capturing an image of the area enlarged and displayed on the display unit 102 as imaging data (step S108). Here, by capturing an image of a magnified area of the screen based on the result of light reception by the solid-state imaging device of the camera 101 as imaging data, an image in which the imaging guide frame is not included in the imaging data is captured. Then, the terminal device 10 transmits the captured image data to the OCR server 30 (step S109).

According to the terminal device 10 in the embodiment described above, the accuracy of character recognition in the character recognition process can be improved.
Further, according to the above-described embodiment, when the document mode is activated, the enlarged display is performed without receiving an operation input for enlarging the display from the user, so the user intentionally enlarges the display There is no need to have the operation of

  Further, in the above-described embodiment, the OCR server 30 uses the frame line when capturing an object whose imaging target is a dimension such as a driver's license, a dimension of a column in which characters and the like are described, or a layout is determined in advance. On the other hand, it is possible to grasp the position where the character can exist, the position where the character string corresponding to the name or the character string corresponding to the address is written. Therefore, the OCR server 30 may have a function of recognizing characters from an image within the rectangular shape by detecting a rectangular shape of a frame line in imaging data. In the case of such an OCR server 30, when character recognition is performed on the imaging data imaged by the above-described terminal device 10, the imaging data has little distortion, so the rectangular shape of the frame line in the imaging data is also The imaging data in a less distorted state can be obtained. Therefore, the detection / recognition rate of the rectangular shape of the frame line can also be improved.

  The terminal device 10 in the embodiment described above may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer readable recording medium, and the program recorded in the recording medium may be read and executed by a computer system. Here, the “computer system” includes an OS and hardware such as peripheral devices. The term "computer-readable recording medium" refers to a storage medium such as a flexible disk, a magneto-optical disk, a ROM, a portable medium such as a ROM or a CD-ROM, or a hard disk built in a computer system. Furthermore, “computer-readable recording medium” dynamically holds a program for a short time, like a communication line in the case of transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include one that holds a program for a certain period of time, such as volatile memory in a computer system that becomes a server or a client in that case. Further, the program may be for realizing a part of the functions described above, or may be realized in combination with the program already recorded in the computer system. It may be realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within the scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... form data management system, 10 ... terminal device, 20 ... form data management apparatus, 30 ... OCR server 30, 40 ... network, 101 ... camera, 102 ... display device, 103 ... image enlargement unit, 104 ... image acquisition unit, 105: Imaging guide frame display unit

Claims (3)

An image enlargement unit for enlarging a region including the center of an image obtained from a camera for imaging a form and displaying the region on a display screen;
An image acquisition unit configured to acquire, as a captured image, the image displayed in the enlarged state and in a state in which the character recognition target area of the form fits on the display screen;
An imaging control apparatus having: Information representing the screen size of the display screen or model information of a terminal device provided with the display screen is acquired, and the screen size of the display screen is specified based on the acquired information, and a size corresponding to the specified screen size The imaging control apparatus according to claim 1, further comprising: an imaging guide frame display control unit configured to display an imaging guide frame of the present invention on the display screen. An imaging control method in a terminal device
The image enlargement unit enlarges an area including the central part of the image obtained from the camera that captures the form, and displays the enlarged area on the display screen,
The image acquisition control method acquires the image of the state which is an image displayed by the said expansion, and the area | region of character recognition object of the said form fits in the said display screen as a captured image.

Images