JP2021047448A - Imaging system, method for imaging, and program - Google Patents

Abstract

To provide an imaging system, a method for imaging, and a program that can acquire a taken image from which the same feature point can be safely extracted by using a camera of a portable terminal.SOLUTION: The imaging system of the present invention uses a portable terminal having a camera unit and a display unit. The imaging system includes: an acquisition unit for acquiring an image including a pattern formed on a surface of an object randomly; a control unit for displaying a transparent image of the image acquired by the acquisition unit so as to make the transparent image overlap with a live view image displayed in the display unit, when the camera unit takes an image of a comparison target object. The pattern is used to extract feature points.SELECTED DRAWING: Figure 1

Description

The present invention relates to a photographing system, an imaging method and a program.

A system has been devised in which an uneven portion on the surface of an object is photographed and the photographed image is used for inspection, authentication, or the like.

For example, Patent Document 1 describes a substantially flat plate-shaped or dome-shaped light diffusing member which has light transmissivity and light diffusivity and is arranged close to each other so as to cover an inspection area photographed by an inspection camera, and the light diffusing thereof. A plurality of light irradiation members that are arranged apart from each other on the outside of the members and irradiate light by transmitting the light diffusion member through the inspection area, and extend in the direction of the camera from the periphery of the observation hole for photographing provided in the light diffusion member. The tubular body is provided so that the camera can take an image of the inspection area through the tubular body and the observation hole, and the direct light from the light irradiation member does not enter the opening on the camera side of the tubular body. A light irradiation device configured as described above is described.

Japanese Unexamined Patent Publication No. 2008-102103

However, there is a problem that it takes time and cost to arrange a special device such as Patent Document 1 in a place where authentication and verification are performed.

On the other hand, a technique has been realized in which a bar code is photographed using a camera provided in a widely used mobile terminal to identify and authenticate. Black and white parts can be easily identified in the captured image of the barcode. However, the captured image of the unevenness on the surface of the object is easily affected by the difference in the position and orientation of the object, the irradiation direction of light, etc., and it is difficult to obtain a captured image that can stably extract the same feature points. It was.

Furthermore, although the viewing angle of the camera and the arrangement of lighting differ depending on the model of the mobile terminal, even when shooting with these different models of mobile terminals, a captured image capable of extracting the same feature points for the same shooting target is obtained. There is a need. For that purpose, it is necessary to hold the terminal so that the relative position and orientation of the shooting target with respect to the camera and light source, which are different for each mobile terminal, are in an appropriate state. Such a holding state is different for each terminal, and even if it is instructed by a numerical value of a distance or a posture, it is difficult for a person holding a mobile terminal to make such an adjustment.

The present invention has been made in view of the above problems, and provides a photographing system, a photographing method, and a program capable of acquiring a photographed image capable of stably extracting the same feature points by using a camera provided in a mobile terminal. The purpose is to provide.

The present invention is a photographing system using a portable terminal including a camera unit and a display unit, in which an acquisition unit that acquires an image including a pattern randomly formed on the surface of an object and a collation object are collated by the camera unit. A control unit that superimposes and displays a transparent image of an image acquired by the acquisition unit on a live view image displayed on the display unit at the time of shooting is provided, and the pattern is used for extracting feature points. It is a shooting system.

The present invention is a photographing method using a mobile terminal including a camera unit and a display unit, in which an image including a pattern randomly formed on the surface of an object is acquired and the collation object is photographed by the camera unit. At that time, the acquired transparent image of the image is superimposed and displayed on the live view image displayed on the display unit, and the pattern is a photographing method used for extracting feature points.

The present invention provides an acquisition process for acquiring an image including a pattern randomly formed on the surface of an object on a mobile terminal including a camera unit and a display unit, and when the camera unit captures a collation object. The control process of superimposing and displaying the transparent image of the image acquired by the acquisition process on the live view image displayed on the display unit is executed, and the pattern is a program used for extracting feature points.

According to the present invention, it is possible to acquire a captured image capable of stably extracting the same feature points by using a camera provided in a mobile terminal.

FIG. 1 is a configuration diagram of a mobile terminal according to an embodiment of the present invention. FIG. 2 is a diagram for explaining the relative position and orientation of the light source of the mobile terminal, the camera, and the collation area. FIG. 3 is a diagram illustrating a shooting screen on which a reference image is displayed. FIG. 4 is a diagram illustrating a shooting screen on which a reference image is displayed. FIG. 5 is a flowchart for explaining the operation of the embodiment of the present invention. FIG. 6 is a diagram for explaining a process in which a terminal device downloads a reference image from a database via a network.

Hereinafter, embodiments of the present invention will be described.

FIG. 1 is a configuration diagram of a mobile terminal according to an embodiment of the present invention. The mobile terminal 10 according to the embodiment of the present invention includes a light source unit 11, a camera unit 12, a display unit 13, and a control unit 14. When the control unit 14 takes a picture of the collation area of the object, the control unit 14 performs the collation with the satin pattern of the object in a state where the relative positions and orientations of the light source unit 11 and the camera unit 12 and the collation area satisfy a predetermined relationship. The reference image for photographing the area is superimposed and displayed on the through image displayed on the display unit 13.

The mobile terminal 10 includes devices such as a smartphone, a mobile phone, and a tablet, and has a normal camera function. The light source unit 11 is an LED (Light Emitting Diode) or the like, and is turned on when the camera takes a picture. The camera unit 12 includes a lens, an image sensor, a controller that controls the entire camera unit, and the like. The display unit 13 includes a liquid crystal display and the like. The control unit 14 controls the entire mobile terminal.

The through image is an image generated from the output signal of the image sensor displayed on the liquid crystal display while the camera included in the mobile terminal 10 is operating in the shooting mode, and is also called a preview image or a live view image. ..

The viewing angle of the camera is determined by the number and size of pixels of the image sensor of the camera, the focal length of the lens, and the like. Since the position of the light source of the mobile terminal, the number and size of the image sensor of the camera, the focal length of the lens, etc. differ depending on the model of the mobile terminal, the arrangement of lighting at the time of shooting and the viewing angle of the camera also differ depending on the model of the mobile terminal. It will be. In the present invention, a model of a mobile terminal can be used to capture an image of a matching area of an object under the same shooting conditions (same irradiation direction and shot image size) between mobile terminals having different lighting arrangements and camera viewing angles. Obtain it in advance for each. Then, the control unit 14 acquires an image corresponding to the mobile terminal among the images acquired in advance, and superimposes and displays it on the through image as a reference image at the time of shooting. The user operates the collation area and the reference image to be photographed on the shooting screen so that the orientation, position, and size match, so that the relative position and orientation of the light source of the mobile terminal, the camera, and the collation area (a in FIG. 2). The relationship (b) is the same as when the reference image was taken. As a result, it is possible to shoot the collation area of the object under the same shooting conditions even between mobile terminals having different camera illumination arrangements and viewing angles.

The images of the collation area previously shot between the different mobile terminals under the same shooting conditions are stored in a storage unit such as a memory of the corresponding mobile terminal, and the control unit 14 reads the images from the storage unit at the time of shooting and uses them. You may. In addition, the mobile terminal has a database in which images corresponding to various mobile terminals are linked with information (model-dependent information) such as the position of the light source of the corresponding mobile terminal, the pixel and size of the image sensor, and the focal length of the lens. It may be stored in the storage unit of the ten, and the control unit 14 may read the image corresponding to the model-dependent information of the mobile terminal 10 from the database and use it.

The display mode of the reference image on the display unit 13 is arbitrary. For example, a pre-captured image may be transparently displayed, or the contour line of a portion corresponding to the collation area in the pre-captured image is extracted and displayed. You may. FIG. 3 and FIG. 4 show examples in which the reference image is the contour line of the portion corresponding to the collation area and are displayed on mobile terminals of different models. As shown in FIG. 3, in a certain model of mobile terminal, the reference image is displayed in a horizontally long shape, for example. The operator operates so that the collation area of the object matches the reference image displayed in landscape orientation. Further, as shown in FIG. 4, in a mobile terminal of another model, the reference image is displayed in a vertically long shape, for example. The operator operates so that the collation area of the object matches the reference image displayed in portrait orientation.

The collation area of the object is not particularly limited as long as it is a satin pattern area, but it is desirable to determine the collation area in advance in consideration of the ease of collation. As the collation area, it is conceivable that the area at the position determined with reference to the reference part commonly existing in the article is used as the collation area. For example, the reference part of an article is a trademark, logo, manufacturer name, manufacturer, brand name, product name, product number, etc. of a product affixed, printed, stamped, or branded on the article. These reference parts have the same pattern regardless of the individual as long as they are the same article, and have a characteristic appearance in appearance. Then, if the position and orientation of these reference parts are set as a reference and the characteristics are stored, the collation area of the article can be automatically detected.

The satin pattern includes, for example, not only the case where the satin pattern is formed as a design, but also the case where the satin pattern is intentionally formed in the collation area for identification or collation of an article. Further, the satin pattern is not only a satin pattern applied by processing a metal or synthetic resin (plastic, etc.), but also a wrinkle pattern (wrinkle pattern) obtained by surface-treating a leather product (for example, embossing). ) Etc. are also included in the concept. For example, in the case of a leather bag that has been textured, minute irregularities are randomly formed on the surface of the leather of the bag, and these minute irregularities can be regarded as similar to a satin pattern.

Next, the operation of the embodiment of the present invention will be described with reference to FIG.
When the mobile terminal 10 is instructed to execute the collation area shooting process, the control unit 14 turns on the camera function and displays the through image output from the camera unit 12 on the liquid crystal display (display unit 13). (S1).

Further, the control unit 14 reads out the reference image corresponding to the mobile terminal 10 from the memory and superimposes and displays it on the through image displayed on the liquid crystal display (S2).

The operator adjusts the collation area of the through image displayed on the shooting screen of the mobile terminal 10 and the reference image so that the orientation, position, and size match, and presses the shutter button (S3). In response to this, the camera unit 12 performs a shooting process and stores the shot image data in the memory (S4).

As described above, according to the present embodiment, an image when the collation area of the object is photographed under the same shooting conditions between mobile terminals having different lighting arrangements and viewing angles of the camera is acquired in advance. When the collation area is photographed, it is superimposed and displayed on the through image as a reference image. Then, by performing an operation so as to match the matching area of the through image with the reference image on the shooting screen, it is possible to easily shoot the matching area of the object under the same shooting conditions even between different mobile terminals.

In S3 of the above flow, the control unit 14 recognizes the collation area in the through image output from the camera unit 12 based on a predetermined reference position, and based on the coordinates on the display screen, the recognized collation area is used. , The zoom may be automatically controlled so that the size of the reference image matches.

Further, since it is difficult to completely match the matching area of the through image with the reference image, the control unit 14 recognizes and displays the matching area of the through image output from the camera unit 12 based on a predetermined reference position. The camera unit 12 may be instructed to take a picture when the collation area of the through image is sufficiently close to the reference image based on the coordinates on the screen.

Further, as shown in FIG. 6, the control unit 14 of the terminal device 10 connects to a database in which various reference images are registered via a network such as the Internet, and registers the reference image or database corresponding to the mobile terminal. All the reference data described above may be downloaded, stored in the storage unit of the mobile terminal 10, and used as described above.

Further, in the above explanation, the reference image and the model-dependent information are registered in association with each other, but the present invention is not limited to this, and for example, the model identification information such as the model name and model code of the mobile terminal and the reference image are supported. You may attach and register.

Although the present invention has been described above with reference to preferred embodiments, the present invention is not necessarily limited to the above embodiments, and can be variously modified and implemented within the scope of the technical idea.

Some or all of the above embodiments may also be described, but not limited to:

(Appendix 1)
Light source and
With the camera part
Display and
A reference image for photographing the collation area in a state where the relative positions and orientations of the light source unit, the camera unit, and the collation area satisfy a predetermined relationship when photographing the collation area in which the satin pattern of the object is applied. A control unit that superimposes and displays on the through image displayed on the display unit,
A mobile terminal equipped with.

(Appendix 2)
The mobile terminal according to Appendix 1, wherein the reference image is obtained from an image obtained by preliminarily photographing the collation area in a state where the relative positions and orientations of the light source unit, the camera unit, and the collation area satisfy a predetermined relationship.

(Appendix 3)
The mobile terminal according to Appendix 1 or Appendix 2, wherein the reference image is obtained from an image obtained by photographing a matching area of an object under the same shooting conditions between mobile terminals having different positions of a light source and at least one of a viewing angle of a camera. ..

(Appendix 4)
The mobile terminal according to any one of Supplementary note 1 to Supplementary note 3, wherein the reference image corresponding to the mobile terminal is acquired from a database in which a plurality of types of reference images are stored.

(Appendix 5)
The portable terminal according to Appendix 4, wherein the database stores a reference image for each mobile terminal and information on the position of the light source of the mobile terminal that captured each reference image and the viewing angle of the camera in association with each other.

(Appendix 6)
The database is the mobile terminal according to Appendix 4, wherein a reference image when a collation area of an object is photographed between mobile terminals of different models under the same shooting conditions is stored for each model of the mobile terminal.

(Appendix 7)
This is a shooting method for mobile terminals.
When photographing the collation area with the satin pattern of the object, the relative position and orientation of the light source unit and the camera unit of the mobile terminal and the distance between the camera unit and the object satisfy a predetermined relationship. A shooting method in which a reference image for shooting the collation area is superimposed and displayed on a through image displayed on a shooting screen of a mobile terminal.

(Appendix 8)
The imaging method according to Appendix 7, wherein the reference image is obtained from an image in which the collation region is captured in advance with the relative positions and orientations of the light source unit, the camera unit, and the collation region satisfying a predetermined relationship.

(Appendix 9)
The imaging method according to Appendix 7 or Appendix 8, wherein the reference image is obtained from an image obtained by photographing a collation area of an object under the same imaging conditions between mobile terminals having different positions of a light source and at least one of a viewing angle of a camera. ..

(Appendix 10)
The photographing method according to any one of Supplementary note 7 to Supplementary note 9, wherein the reference image corresponding to a mobile terminal is acquired from a database in which a plurality of types of reference images are stored.

(Appendix 11)
The imaging method according to Appendix 10, wherein the database stores a reference image for each mobile terminal and information on the position of a light source of the mobile terminal that captured each reference image and the viewing angle of the camera in association with each other.

(Appendix 12)
The imaging method according to Appendix 10, wherein a reference image when a collation area of an object is photographed between mobile terminals of different models under the same shooting conditions is stored for each model of the mobile terminal.

(Appendix 13)
On mobile terminals
A reference for photographing the collation area in a state where the relative positions and orientations of the light source unit, the camera unit, and the collation area of the mobile terminal satisfy a predetermined relationship when photographing the collation area with the satin pattern of the object. A program that executes a process of superimposing an image on a through image displayed on the shooting screen of a mobile terminal.

(Appendix 14)
The program according to Appendix 13, wherein the reference image is obtained from an image obtained by preliminarily photographing the collation area in a state where the relative positions and orientations of the light source unit, the camera unit, and the collation area satisfy a predetermined relationship.

(Appendix 15)
The program according to Appendix 13 or Appendix 14, wherein the reference image is obtained from an image obtained by photographing a collation area of an object under the same photographing conditions between mobile terminals having different positions of a light source and at least one of a viewing angle of a camera.

(Appendix 16)
The program according to any one of Supplementary note 13 to Supplementary note 15, which acquires the reference image corresponding to a mobile terminal from a database in which a plurality of types of reference images are stored.

(Appendix 17)
The program according to Appendix 16 in which the database is stored in association with a reference image for each mobile terminal and information on the position of a light source of the mobile terminal that captured each reference image and the viewing angle of the camera.

(Appendix 18)
The program according to Appendix 16, wherein a reference image when a collation area of an object is photographed between mobile terminals of different models under the same shooting conditions is stored for each model of the mobile terminal.

10 Mobile terminal 11 Light source unit 12 Camera unit 13 Display unit 14 Control unit

Claims (9)

A shooting system that uses a mobile terminal equipped with a camera unit and a display unit.
An acquisition unit that acquires an image containing a pattern randomly formed on the surface of an object,
A control unit that superimposes and displays a transparent image of the image acquired by the acquisition unit on the live view image displayed on the display unit when the collation object is photographed by the camera unit.
With
The pattern is a photographing system used for extracting feature points. The image acquired by the acquisition unit further includes at least one of the product trademark, logo, manufacturer name, manufacturer, brand name, product name and product number, which is affixed, printed, stamped or branded on the object. The photographing system according to claim 1, which includes. The imaging system according to claim 2, wherein at least one of the trademark, logo, manufacturer name, manufacturing company, brand name, product name, and product number of the product is used for alignment at the time of photographing the collation object. It is a shooting method using a mobile terminal having a camera unit and a display unit.
Acquire an image containing randomly formed patterns on the surface of the object,
When the collation target is photographed by the camera unit, the acquired transparent image of the image is superimposed and displayed on the live view image displayed on the display unit.
The pattern is a photographing method used for extracting feature points. The acquired image further comprises at least one of a product trademark, logo, manufacturer name, manufacturer, brand name, product name and product number affixed, printed, stamped or branded on the object. The shooting method according to 4. The photographing method according to claim 5, wherein at least one of the trademark, logo, manufacturer name, manufacturing company, brand name, product name and product number of the product is used for alignment at the time of photographing the collation object. For mobile terminals equipped with a camera unit and a display unit
Acquisition process to acquire an image including a pattern randomly formed on the surface of an object,
A control process for superimposing and displaying a transparent image of an image acquired by the acquisition process on a live view image displayed on the display unit when a collation object is photographed by the camera unit.
To execute,
The pattern is a program used to extract feature points. The image acquired by the acquisition process further includes at least one of the product trademark, logo, manufacturer name, manufacturer, brand name, product name and product number, which is affixed, printed, stamped or branded on the object. The program according to claim 7, which includes. The program according to claim 8, wherein at least one of the trademark, logo, manufacturer name, manufacturing company, brand name, product name and product number of the product is used for alignment of the collated object at the time of photographing.

Images