JP6206645B2 - Portable terminal, photographing method and program - Google Patents

Description

  The present invention relates to a portable terminal, a photographing method, and a program.

  A system has been devised in which an uneven portion on the surface of an object is photographed and the photographed image is used for inspection or authentication.

  For example, Patent Document 1 discloses a light diffusing member having a light transmission property and a light diffusing property, and having a substantially flat plate shape or a dome shape so as to cover an inspection region photographed by an inspection camera, and its light diffusion. A plurality of light irradiating members that are spaced apart from the member and radiate light through the light diffusing member through the inspection region, and extend from the periphery of the imaging observation hole provided in the light diffusing member toward the camera. A cylindrical body, and the camera is configured so that the inspection area can be photographed through the cylindrical body and the observation hole, and direct light from the light irradiation member does not enter the camera side opening of the cylindrical body. A light irradiation apparatus configured as described above is described.

JP 2008-102103 A

  However, there is a problem that it takes time and cost to arrange special devices such as Patent Document 1 in places where authentication and verification are performed.

  On the other hand, a technique has been realized in which a barcode is photographed using a camera provided in a widely used portable terminal, and identification and authentication are performed. The black and white portion of the bar code image can be easily identified. However, the captured image of the uneven surface of the object is easily affected by differences in the position and orientation of the object, the direction of light irradiation, etc., and it is difficult to obtain a captured image that can stably extract the same feature points. It was.

  Furthermore, although the viewing angle of the camera and the arrangement of illumination differ depending on the model of the mobile terminal, a captured image is obtained that can extract the same feature points for the same subject even when captured by these different types of mobile terminals. There is a need. For this purpose, it is necessary to hold the terminal so that the relative position and orientation of the subject to be photographed with respect to the different cameras and light sources for each portable terminal are in an appropriate state. Such a holding state is different for each terminal, and even if it is indicated by a numerical value of a distance or a posture, it is difficult for a person holding the mobile terminal to make such adjustment.

  The present invention has been made in view of the above problems, and a portable terminal, a photographing method, and a program capable of acquiring a photographed image that can stably extract the same feature point using a camera included in the portable terminal. The purpose is to provide.

  According to the present invention, the relative position and orientation of the light source unit, the camera unit, and the collation region are predetermined when photographing the collation region on which the light source unit, the camera unit, the display unit, and the matte pattern of the target object are applied. And a control unit that superimposes and displays a reference image for photographing the collation area in a state where the relationship is satisfied on a through image displayed on the display unit.

  The present invention is a photographing method in a portable terminal, and when photographing a collation area on which a satin pattern is applied to an object, the relative position and orientation of the light source section and the camera section of the portable terminal and the collation area have a predetermined relationship. In this photographing method, a reference image for photographing the collation region in a satisfied state is superimposed and displayed on a through image displayed on the photographing screen of the mobile terminal.

  In the present invention, when photographing the collation area where the matte pattern of the object is applied to the portable terminal, the light source unit of the portable terminal, the camera unit, and the relative position and orientation of the collation area satisfy a predetermined relationship. This is a program for executing a process of superimposing and displaying a reference image for photographing a verification area on a through image displayed on a photographing screen of a mobile terminal.

  ADVANTAGE OF THE INVENTION According to this invention, the picked-up image which can extract the same feature point stably can be acquired using the camera with which a portable terminal is equipped.

FIG. 1 is a configuration diagram of a mobile terminal according to an embodiment of the present invention. FIG. 2 is a diagram for explaining the relative position and orientation of the light source, the camera, and the collation region of the mobile terminal. FIG. 3 is a diagram illustrating a shooting screen on which a reference image is displayed. FIG. 4 is a diagram illustrating a shooting screen on which a reference image is displayed. FIG. 5 is a flowchart for explaining the operation of the embodiment of the present invention. FIG. 6 is a diagram for explaining a process in which the terminal device downloads the reference image from the database via the network.

  Embodiments of the present invention will be described below.

  FIG. 1 is a configuration diagram of a mobile terminal according to an embodiment of the present invention. The mobile terminal 10 according to the embodiment of the present invention includes a light source unit 11, a camera unit 12, a display unit 13, and a control unit 14. When the control unit 14 shoots the verification region of the target object, the verification is performed with the matte pattern of the target object in a state where the relative position and orientation of the light source unit 11, the camera unit 12, and the verification region satisfy a predetermined relationship. A reference image for photographing the region is superimposed on the through image displayed on the display unit 13.

  The mobile terminal 10 includes devices such as a smartphone, a mobile phone, and a tablet, and has a normal camera function. The light source unit 11 is an LED (Light Emitting Diode) or the like, and is turned on during camera shooting. The camera unit 12 includes a lens and an image sensor, a controller that controls the entire camera unit, and the like. The display unit 13 includes a liquid crystal display or the like. The control unit 14 controls the entire mobile terminal.

  The through image is an image generated from the output signal of the image sensor while the camera included in the mobile terminal 10 is operating in the shooting mode, and is also referred to as a preview image or a live view image. .

  The viewing angle of the camera is determined by the number and size of pixels of the image sensor of the camera, the focal length of the lens, and the like. Since the position of the light source of the mobile terminal, the number and size of pixels of the image sensor of the camera, the focal length of the lens, etc. differ depending on the model of the mobile terminal, the lighting arrangement and the camera viewing angle at the time of shooting also differ depending on the model of the mobile terminal It will be. In the present invention, an image obtained by photographing a collation area of an object under the same photographing condition (the same irradiation direction and photographed image size) between portable terminals having different illumination arrangements and camera viewing angles is used as a model of the portable terminal. Each time it is acquired in advance. And the control part 14 acquires the image corresponding to the portable terminal among the images acquired previously, and superimposes and displays it on a through image as a reference | standard image at the time of imaging | photography. The user operates the collation area and the reference image to be photographed on the photographing screen so that the orientation, the position, and the size match each other, so that the relative position and orientation of the light source of the mobile terminal, the camera, and the collation area ( b) is the same as when the reference image was taken. Thereby, the collation area | region of a target object can be image | photographed on the same imaging conditions also between portable terminals from which the arrangement | positioning of illumination of a camera, and a viewing angle differ.

  The images of the collation areas previously captured between the different mobile terminals under the same shooting conditions are stored in a storage unit such as a memory of the corresponding mobile terminal, and the control unit 14 reads out from the storage unit and uses it at the time of shooting. May be. In addition, a database in which images corresponding to various portable terminals are associated with information (model-dependent information) such as the position of the light source of the corresponding portable terminal, the pixel and size of the image sensor, the focal length of the lens, etc. 10, and the control unit 14 may read and use an image corresponding to the model-dependent information of the mobile terminal 10 from the database.

  The display mode of the reference image on the display unit 13 is arbitrary. For example, a pre-photographed image may be transparently displayed, and a contour line of a portion corresponding to a collation area in the pre-photographed image is extracted and displayed. May be. FIG. 3 and FIG. 4 show examples of display on different types of portable terminals in the case where the reference image is the contour line of the part corresponding to the collation area. As shown in FIG. 3, in a certain type of mobile terminal, the reference image is displayed in, for example, landscape orientation. The operator performs an operation so that the collation area of the target object matches the reference image displayed in landscape orientation. Further, as shown in FIG. 4, in other types of portable terminals, the reference image is displayed in, for example, a vertically long shape. The operator performs an operation so that the collation area of the target object matches the reference image displayed in the portrait orientation.

  The collation area of the object is not particularly limited as long as it is a satin pattern area, but it is desirable to determine the collation area in advance in consideration of the ease of collation. As a collation area, it is conceivable that an area at a position determined with reference to a reference portion that is common to articles is used as a collation area. For example, the reference part of an article includes a trademark, a logo, a manufacturer name, a manufacturer, a brand name, a product name, a product number, and the like of a product affixed, printed, stamped, or baked on the article. These reference parts have the same pattern regardless of the individual if they are the same article, and have a characteristic appearance on the appearance. If the position and orientation of these reference parts are determined as a reference and the characteristics are stored, the article collation area can be automatically detected.

  The satin pattern includes, for example, not only a case where a satin pattern is formed as a design, but also a case where a satin pattern is intentionally formed in a collation region for identification or collation of articles. Furthermore, the satin pattern is not only a satin pattern that has been applied to metal or synthetic resin (plastic, etc.), but also a wrinkle pattern that is obtained by surface treatment (for example, wrinkle processing) on leather products. ) Etc. For example, in the case of a leather bag that has been subjected to graining, minute irregularities are randomly formed on the surface of the leather of the bag, and these minute irregularities can be regarded as being similar to a satin pattern.

Next, the operation of the embodiment of the present invention will be described with reference to FIG.
When an operation to instruct execution of the collation area photographing process is performed on the mobile terminal 10, the control unit 14 turns on the camera function and displays a through image output from the camera unit 12 on the liquid crystal display (display unit 13). (S1).

  Moreover, the control part 14 reads the reference | standard image corresponding to the said portable terminal 10 from memory, and superimposes and displays it on the through image displayed on the liquid crystal display (S2).

  The operator adjusts the through image collation area and the reference image displayed on the shooting screen of the mobile terminal 10 so that the orientation, the position, and the size match each other, and presses the shutter button (S3). In response to this, the camera unit 12 performs a shooting process and stores the shot image data in the memory (S4).

  As described above, according to the present embodiment, an image obtained by capturing a collation region of an object under the same capturing condition between mobile terminals having different illumination arrangements and viewing angles of cameras is acquired in advance. When shooting the verification area, it is superimposed on the through image as a reference image. Then, by manipulating and photographing the matching area of the through image and the reference image on the photographing screen, it is possible to easily photograph the matching area of the target object under the same photographing condition even between different portable terminals.

  In S3 of the above flow, the control unit 14 recognizes the collation area in the through image output from the camera unit 12 based on a predetermined reference position, and based on the coordinates on the display screen and the like, Alternatively, zoom control may be automatically performed so that the size of the reference image matches.

  Further, since it is difficult to completely match the matching area of the through image with the reference image, the control unit 14 recognizes and displays the matching area in the through image output from the camera unit 12 based on a predetermined reference position. Based on the coordinates on the screen or the like, the camera unit 12 may be instructed to take a picture when the matching area of the through image is sufficiently close to the reference image.

  Further, as shown in FIG. 6, the control unit 14 of the terminal device 10 connects to a database in which various reference images are registered via a network such as the Internet, and registers the reference image or database corresponding to the portable terminal. All the reference data being downloaded may be downloaded and stored in the storage unit of the mobile terminal 10 and used as described above.

  In the above description, the reference image and the model-dependent information are registered in association with each other. However, the present invention is not limited to this. For example, model identification information such as the model name and model code of the mobile terminal is associated with the reference image. You may also register.

  Although the present invention has been described with reference to the preferred embodiments, the present invention is not necessarily limited to the above-described embodiments, and various modifications can be made within the scope of the technical idea.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
A light source unit;
A camera unit;
A display unit;
A reference image for photographing the collation area in a state in which the relative position and orientation of the light source unit, the camera unit, and the collation area satisfy a predetermined relationship when photographing the collation area on which the satin pattern of the object is applied. A control unit that superimposes and displays on the through image displayed on the display unit,
A mobile terminal comprising:

(Appendix 2)
The mobile terminal according to appendix 1, wherein the reference image is acquired from an image obtained by capturing the collation area in advance in a state where relative positions and orientations of the light source unit, the camera unit, and the collation area satisfy a predetermined relationship.

(Appendix 3)
The portable terminal according to Supplementary Note 1 or Supplementary Note 2, wherein the reference image is acquired from an image obtained by photographing a collation region of an object under the same photographing condition between portable terminals in which at least one of a light source position and a camera viewing angle is different. .

(Appendix 4)
The mobile terminal according to any one of appendix 1 to appendix 3, wherein the reference image corresponding to the mobile terminal is acquired from a database in which a plurality of types of reference images are stored.

(Appendix 5)
The mobile terminal according to appendix 4, wherein the database stores a reference image for each mobile terminal and information on the position of the light source and the viewing angle of the camera that captured each reference image in association with each other.

(Appendix 6)
The mobile terminal according to appendix 4, wherein the database stores a reference image for each model of the mobile terminal when the collation area of the target object is captured between the mobile terminals of different models under the same shooting condition.

(Appendix 7)
A shooting method on a mobile terminal,
When photographing the collation area on which the satin pattern of the object is applied, the relative position and orientation of the light source unit and the camera unit of the mobile terminal and the distance between the camera unit and the object satisfy a predetermined relationship. An imaging method for displaying a reference image for imaging the collation area superimposed on a through image displayed on an imaging screen of a mobile terminal.

(Appendix 8)
The photographing method according to appendix 7, wherein the reference image is acquired from an image obtained by photographing the collation region in advance in a state where relative positions and orientations of the light source unit, the camera unit, and the collation region satisfy a predetermined relationship.

(Appendix 9)
The imaging method according to appendix 7 or appendix 8, wherein the reference image is acquired from an image obtained by capturing a collation region of an object under the same capturing condition between mobile terminals in which at least one of a light source position and a camera viewing angle is different. .

(Appendix 10)
The imaging method according to any one of appendix 7 to appendix 9, wherein the reference image corresponding to the mobile terminal is acquired from a database in which a plurality of types of reference images are stored.

(Appendix 11)
The imaging method according to appendix 10, wherein the database stores the reference image for each mobile terminal and information on the position of the light source of the mobile terminal that captured each reference image and the viewing angle of the camera in association with each other.

(Appendix 12)
The imaging method according to appendix 10, wherein the database stores, for each model of mobile terminal, a reference image when a collation area of an object is captured under the same shooting condition between different types of mobile terminals.

(Appendix 13)
On your mobile device,
A reference for photographing the collation area in a state in which the relative position and orientation of the light source unit, the camera unit, and the collation area of the portable terminal satisfy a predetermined relationship when photographing the collation area on which the satin pattern of the object is applied. A program that executes a process of superimposing and displaying an image on a through image displayed on a shooting screen of a mobile terminal.

(Appendix 14)
14. The program according to appendix 13, wherein the reference image is acquired from an image obtained by photographing the collation area in advance in a state where relative positions and orientations of the light source unit, the camera unit, and the collation area satisfy a predetermined relationship.

(Appendix 15)
The program according to supplementary note 13 or supplementary note 14, wherein the reference image is acquired from an image obtained by photographing a collation region of an object under the same photographing condition between portable terminals in which at least one of a light source position and a camera viewing angle is different.

(Appendix 16)
The program according to any one of Supplementary Note 13 to Supplementary Note 15, wherein the reference image corresponding to the mobile terminal is acquired from a database in which a plurality of types of reference images are stored.

(Appendix 17)
The program according to claim 16, wherein the database stores a reference image for each mobile terminal and information on the position of the light source of the mobile terminal that has captured each reference image and the viewing angle of the camera in association with each other.

(Appendix 18)
The program according to appendix 16, wherein the database stores, for each model of mobile terminal, a reference image when a collation area of an object is captured under the same shooting condition between different types of mobile terminals.

DESCRIPTION OF SYMBOLS 10 Portable terminal 11 Light source part 12 Camera part 13 Display part 14 Control part

Claims (9)

A light source unit;
A camera unit;
A display unit;
A reference image for photographing the collation area in a state in which the relative position and orientation of the light source unit, the camera unit, and the collation area satisfy a predetermined relationship when photographing the collation area on which the satin pattern of the object is applied. A control unit that superimposes and displays on the through image displayed on the display unit,
Equipped with a,
The control unit corresponds to its own terminal among reference images for each portable terminal for controlling the position and orientation of the portable terminal so that the collation area of the target object can be photographed between the portable terminals of different models under the same photographing condition. A portable terminal using a reference image to be used . The control unit acquires the reference image corresponding to the own terminal from a database in which a plurality of types of the reference images are stored for each model of the mobile terminal.
The mobile terminal according to claim 1. 3. The mobile terminal according to claim 1, wherein the reference image is acquired from an image obtained by capturing the collation area in advance in a state where relative positions and orientations of the light source unit, the camera unit, and the collation area satisfy a predetermined relationship. . 4. The reference image according to claim 1, wherein the reference image is acquired from an image obtained by photographing a collation region of an object under the same photographing condition between mobile terminals having different light source positions and / or camera viewing angles . The portable terminal as described in. The mobile terminal according to claim 2 , wherein the database stores a reference image for each mobile terminal in association with information on a light source position and a camera viewing angle of the mobile terminal that captured each reference image. The portable terminal according to claim 2 , wherein the database stores a reference image for each model of the portable terminal when the collation area of the target object is photographed between the portable terminals of different models under the same photographing condition. A shooting method on a mobile terminal,
When photographing the collation area on which the satin pattern of the object is applied, the relative position and orientation of the light source unit and the camera unit of the mobile terminal and the distance between the camera unit and the object satisfy a predetermined relationship. A reference image for photographing the collation area is superimposed on a through image displayed on the photographing screen of the mobile terminal ,
The reference image superimposed and displayed on the through image is a reference for each portable terminal for controlling the position and orientation of the portable terminal so that the collation area of the object can be photographed between the portable terminals of different models under the same photographing condition. A shooting method which is a reference image corresponding to the terminal of the image . On your mobile device,
A reference for photographing the collation area in a state in which the relative position and orientation of the light source unit, the camera unit, and the collation area of the portable terminal satisfy a predetermined relationship when photographing the collation area on which the satin pattern of the object is applied. Display control processing to superimpose the image on the through image displayed on the shooting screen of the mobile terminal ,
The display control processing is performed on the own terminal among the reference images for each portable terminal for controlling the position and orientation of the portable terminal so that the collation region of the object can be photographed between the portable terminals of different models under the same photographing condition. A program that uses the corresponding reference image . In the mobile terminal,
Processing for obtaining the reference image corresponding to the own terminal from a database in which a plurality of types of the reference images are stored for each model of the portable terminal
9. The program according to claim 8, wherein:

Images