JP2017098878A - Information terminal device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize accurate recognition even when preview display desired by a user is short of recognition accuracy, in recognizing an imaging target in an imaged picture, from the imaged picture.SOLUTION: A processed picture obtained by processing an imaged picture is used as preview display of the imaged picture. When the imaged picture exists in a region R511 that is unsuitable for recognition, preview display is shown in a region R512 that may not be desired by a user; when the imaged picture exists in a region R521 that is suitable for recognition, the preview display occupies a region R522 that may be desired by the user. Thereby, the user may spontaneously perform imaging suitable for recognition processing. The processing can include expansion, reduction, translation, lightness conversion, etc.SELECTED DRAWING: Figure 5

Description

  The present invention provides an information terminal device that realizes highly accurate recognition even when a preview display of an imaging desired by a user is inappropriate for recognition processing when recognizing the imaging target from a captured image, and Regarding the program.

  An apparatus for recognizing an object from an image can convert analog information described in a medium that can be easily distributed and presented from digital information to digital information, and can improve user convenience.

  For example, in Non-Patent Document 1, a feature point is detected from an image, a local image feature amount is calculated from around the feature point, and then compared with a local image feature amount accumulated in advance, the type of object Recognize

  In particular, the following devices are disclosed as devices for increasing the accuracy of recognition.

  Patent Document 1 discloses a method of recognizing an image by removing outliers that become noise by two-step Hough transform and specifying inlier clusters used for recognition. Patent Document 2 discloses a technique of reducing redundant dimensions while increasing the number of dimensions of local image feature amounts.

JP 2013-025799 JP 2012-181765 JP

D. G. Lowe, '' Object recognition from local scale-invariant Features, '' Proc. Of IEEE International Conference on Computer Vision (ICCV), pp.1150-1157, 1999.

  However, the conventional techniques such as Non-Patent Document 1, Patent Document 1, and Patent Document 2 described above have a problem in that recognition fails when there are few feature points and local feature values suitable for recognition.

  Therefore, the subject is examined as follows. There are three types of situations where a sufficient number of feature points cannot be obtained. The first situation is a case where there are few feature points in the imaging target (whole). The second situation is a case where even if there are many feature points in the imaging target (whole), only a part thereof is imaged in the vicinity of the imaging target. The third situation is a case where even if there are many feature points in the imaging target (whole), the imaging target is too far away from the imaging target.

  As a solution to the first situation, a method that secures a sufficient number of feature points for recognition as a whole even if there are few feature points for each image capture target by combining multiple image capture targets into one image capture target Is effective. For example, if you want to recognize a page of a catalog or the like where multiple products are posted on the same page, if an administrator sets an individual product image as an imaging target, a product that does not have sufficient feature points Although it becomes impossible to recognize, it is possible to secure sufficient feature points and improve recognition accuracy by collectively setting a plurality of product images as one imaging target.

  However, as described above, from the standpoint of an administrator or the like, even if a measure is taken to collectively set a plurality of product images as one imaging target, if the imaging unit is held over an individual product that the user is interested in, The second situation occurs because the number of feature points is insufficient. In addition, from the user's standpoint, the second situation may occur because the operation of holding the imaging unit so as to capture only the individual products that the user is interested in is a natural operation in line with intuition. It is thought that there are many.

  Here, the reason why the feature point is insufficient in the second situation is that the feature point is not reflected in the captured image. On the other hand, the reason that the feature point is insufficient in the third situation is that the resolution of the captured image is reduced and the image is taken. This is because the detailed pattern of the object is crushed. For example, when it is desired to recognize posters or exhibits, it is assumed that users are often placed in an environment where such recognition targets (posters, exhibits, etc.) cannot be obtained. Users have a strong tendency to image the recognition target from a distance, and the recognition often fails because the third situation occurs.

  In other words, as described above, in any of the first to third situations, there is a range suitable for recognition in the distance and / or positional relationship with the imaging target, and the range deviates from that range. However, there is a problem that the local feature amount becomes small and cannot be recognized, but the conventional technique cannot deal with the problem. Here, as illustrated in the above discussion, even if the feature point and local image feature amount to be recognized are set on the assumption that the administrator etc. captures images within a range suitable for recognition, From the standpoint of a user who performs image capturing, the problem is that, for example, the image capturing operation in the appropriate range is contrary to intuition, and the image is often deviated from the appropriate range. It was one of the causes.

  In addition, the problem is a problem that generally occurs when recognition is performed from a captured image, and is not limited to the case where recognition is performed using feature points and local image feature amounts. It was a problem that occurred.

  An object of the present invention is to provide an information terminal device and a program capable of recognizing an imaging target with high accuracy from a captured image in view of the above-described problems of the prior art.

  In order to achieve the above object, the present invention is characterized by the following (1) to (10).

  (1) An imaging unit that captures an imaged object to obtain a captured image, a recognition unit that analyzes the captured image and recognizes the imaged object, and a process that obtains a processed image by performing predetermined processing on the captured image And a display unit that displays the processed image as a preview, and the predetermined processing is a processing that is determined in advance according to a candidate for an imaging target and a candidate for how to perform the imaging, and the imaging When the captured image of the imaging target is in a state inappropriate for recognition in the recognition unit, the processed image obtained is in an inappropriate state as a preview in the display unit. When the captured image obtained by the imaging unit captures the imaging target is in a state suitable for recognition by the recognition unit, the processed image obtained is displayed on the display unit. In Characterized in that it is an appropriate state as a preview.

  (2) Further, in the feature (1), the predetermined processing is processing that extracts a partial region of the captured image as a processed image, and the imaging unit is in a state inappropriate for recognition. The inappropriate state due to being too close to the recognition target, and the inappropriate state as the preview in the display unit is an inappropriate state due to the recognition target being too large.

  (3) Further, in the feature (1), the predetermined processing is an enlargement process, and the state inappropriate for recognition is an inappropriate state due to the imaging unit being too close to the recognition target The inappropriate state as the preview in the display unit is an inappropriate state due to the recognition target being too large.

  (4) Further, in the feature (1), the predetermined processing is a reduction process, and the state inappropriate for recognition is an inappropriate state due to the imaging unit being too far from the recognition target. In addition, the inappropriate state as a preview in the display unit is an inappropriate state due to the recognition target being too small.

  (5) Further, in the feature (1), the predetermined processing is processing for adding a pincushion type distortion, and the state inappropriate for the recognition is that the imaging unit is too far from the recognition target. The inappropriate state, which is inappropriate as a preview in the display unit, is an inappropriate state due to the recognition target being too small.

  (6) In the feature (1), the predetermined processing is processing for moving the center position of the processed image so as to be different from the center position of the captured image. Is an inappropriate state due to the image of the recognition target being deviated from its center, and an inappropriate state as a preview in the display unit is a state in which the recognition target is deviated from the center It is characterized by that.

  (7) In the feature (1), the predetermined processing is processing for enhancing or reducing the gradation of the processed image more than the gradation of the captured image by brightness conversion, and the state inappropriate for the recognition is The imaging unit is in an inappropriate state because it captures the recognition target with a loss of gradation, and the inappropriate state as a preview in the display unit is that the recognition target impairs the gradation. It is characterized by being reflected.

  (8) In the features (1) to (7), the recognizing unit further calculates a local image feature amount from the captured image, and each of the local areas is calculated in advance for each of a plurality of predetermined recognition objects for reference. A recognition target that is determined to be similar by comparing with an image feature amount is recognized as corresponding to the imaging target in the captured image.

  (9) In the feature (2), the recognizing unit further calculates a local image feature amount from the captured image, and a local image feature amount calculated in advance for each of a plurality of predetermined recognition targets for reference. A recognition target determined to be similar by comparison is recognized as corresponding to the imaging target in the captured image, and the reference local image in the recognition target determined to be similar. Based on the distribution of the feature point coordinates of the corresponding feature amount obtained with the local image feature amount in the captured image, the region of the captured image and the region of the processed image as the partial region are obtained. It is characterized by estimating.

  (10) A program for causing a computer to function as the information terminal device according to the features (1) to (9).

  According to the features of (1) and (10) above, a preview that the user determines to be appropriate by using a processed image obtained by processing the captured image as an image that functions as a preview display for the captured image. At the same time when the display is obtained, the captured image can be secured in a state suitable for the recognition process, and by using the captured image as a target of the recognition process, a highly accurate recognition process can be realized.

  According to the features (2) to (7), the preview display that is determined to be appropriate by the user is displayed on the captured image according to the method of imaging assumed by the user. The image can be obtained by performing processing such as reduction processing, pincushion type distortion processing, center movement processing, and brightness conversion processing.

  According to the feature (8), the recognition result of the imaging target in the captured image can be obtained based on the local image feature amount.

  According to the feature (9), information on the area of the captured image and information on the area shown as a processed image to the user can be obtained in the recognition unit.

It is a functional block diagram of the information terminal device concerning one embodiment. It is a functional block diagram of the recognition part in one Embodiment. It is a functional block diagram of a processing part. It is a figure for demonstrating the content of the expansion process (process which extracts a one part area | region) by an expansion part as a process process. It is a figure which shows the example for demonstrating the effect at the time of applying the expansion process (process which extracts a one part area | region) by an expansion part as a process process. It is a figure for demonstrating the content of the reduction process by a reduction part as a process process. It is a figure which shows the example for demonstrating the effect at the time of applying the reduction process by a reduction part as a process process. It is a figure for demonstrating the content of the translation process (center movement process) by a translation part as a process. It is a figure which shows the example for demonstrating the effect at the time of applying the translation process (center movement process) by a translation part as a process process.

  FIG. 1 is a functional block diagram of an information terminal device according to an embodiment. The information terminal device 1 includes an imaging unit 2, a recognition unit 3, a processing unit 4, and a display unit 5. The functional outline of each part 1-5 of FIG. 1 is as follows.

  The imaging unit 2 images the imaging target under the user's imaging operation U (t) at each time t, and outputs the captured image P1 (t) to the calculation unit. Here, it is assumed that the user performs imaging by the imaging operation U (t) so that the captured image P1 (t) includes a known imaging target in advance. The imaging target may be, for example, a marker having a pattern with known characteristics, a printed material, a three-dimensional object, or the like. As a hardware configuration for the imaging unit 2, a digital camera provided as a standard in a portable terminal can be used.

  The user's imaging operation U (t) with respect to the imaging unit 2 at each time t is taken by placing the imaging unit 2 in an arrangement (camera position and orientation) desired by the user with respect to the imaging target. Refers to operations. That is, it refers to a “holding” operation (normal imaging operation) by holding the imaging unit 2 (camera) in the hand or attaching the imaging unit 2 (camera) to a stand. Since such an operation generally changes with time (including a case where a constant state is maintained), it is described as an imaging operation U (t). In addition, when the light source is arranged with respect to the imaging target and the user can adjust the light source, adjusting the position of the light source may be included in the imaging operation U (t).

  The recognition unit 3 recognizes the imaging target from the captured image P1 (t) captured by the imaging unit 2 at each time t, and outputs the recognition target R (t) to the user or the like. As recognition processing in the recognition unit 3, existing methods such as QR code (registered trademark) reading, character recognition, and specific object recognition using local image feature amounts can be used.

  In the embodiment, the recognition unit 3 further processes parameters and the like in the processing when the processing unit 4 (described later) processes the captured image P1 (t2) and outputs the processed image P2 (t2) at the current time t2. An instruction I (t2) can also be instructed to the processing unit 4. Here, the recognition unit 3 can determine the processing instruction I (t2) at the current time t2 based on the recognition result R (t1) at the past time t1 (t1 <t2). The past time t1 for the determination is not limited to one time, and a plurality of times may be used, or a past predetermined period with respect to the current time t2 may be used.

  Note that an embodiment in which the processing instruction I (t2) is omitted is possible, and in this case, the processing unit 4 always uses the processing unit 4 as described later in each embodiment regardless of the actual recognition state in the recognition unit 3. The corresponding predetermined type of processing is applied to the predetermined parameters. In contrast, in the embodiment using the processing instruction I (t2), the processing unit 4 performs a predetermined type of processing according to each embodiment of the processing unit 4 without depending on the actual recognition state in the recognition unit 3. Is applied in the same way, but the parameter for the application is set based on the past recognition result in the recognition unit 3 according to the trend in the past history of the user's imaging operation U (t). Is possible. The processing instruction I (t2) will be described again in supplementary explanation (1) described later.

  FIG. 2 is a functional block diagram of the recognition unit 3 in one embodiment. The recognition unit 3 includes a calculation unit 31, a storage unit 32, and a collation unit 33. In one embodiment in which recognition is performed using local image feature amounts, the processing content of each unit is as follows.

  First, the calculation unit 31 detects the feature point of the imaging target from the captured image P1 (t) captured by the imaging unit 2. A characteristic point such as a corner in the recognition target can be used as the characteristic point to be detected. As detection methods, existing methods for detecting characteristic points such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) can be used.

  Next, the calculation unit 31 calculates a local image feature amount from the captured image P1 (t) captured by the imaging unit 2, with the detected feature point coordinates as the center. As a method for calculating the local image feature amount, an existing method for calculating a characteristic amount such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) can be used.

  The plurality of feature points and local image feature amounts calculated as described above by the calculation unit 31 are output to the matching unit 33 as the feature information F (t) of the captured image P1 (t) at each time t.

  The storage unit 32 stores, for each of a plurality of predetermined recognition targets as reference targets, feature information calculated by the same process as the process performed on the captured image by the calculation unit 31 from the image of the recognition target. Here, in order to speed up the processing in the collation unit 33 described below, the feature information may be stored after being summarized by vector quantization, a hash function, or the like.

  The collation unit 33 evaluates the similarity between the feature information F (t) of the imaging target input from the calculation unit 31 and the feature information of each recognition target stored in the storage unit 32 at each time t, If there is a recognition target having a similarity higher than a preset threshold, the recognition result R of the imaging target captured in the captured image P1 (t) is the one or more recognition targets or the recognition target having the highest similarity. (t) is output to the processing unit 4 or the user. Note that the recognition result R (t) at each time t may be held in the information terminal device 1 without being output to the user or the like. In this case, after receiving an output request from the user or the like, the recognition result R (t) may be output to the user or the like.

  Here, for the similarity evaluation, an existing method using a Hamming distance, Euclidean distance, Mahalanobis distance, or the like between feature information can be used. In addition, after determining the correspondence between the most similar feature information based on the distance between individual feature information between the recognition target and the captured image, the sum of the similarities in the determined correspondence is obtained. Alternatively, the overall correspondence and similarity between the feature information between the recognition target and the captured image may be obtained by a known RANSAC (Random Sample Consensus).

  The processing unit 4 inputs the captured image P1 (t) from the imaging unit 2 at each time t, and the result of performing image conversion processing such as enlargement / reduction on the captured image P1 (t) is processed image P2 (t ) To the display unit 5.

  The display unit 5 inputs the processed image P2 (t) from the processing unit 4 at each time t, and displays the processed image P2 (t) to the user. As the display unit 5, a display provided as a standard in a mobile terminal can be used.

  As described above, the processed image P2 (t) obtained by processing the captured image P1 (t) captured by the imaging unit 2 at each time t is displayed on the display unit 5. Accordingly, from the user's standpoint, the display unit 5 provides a so-called camera preview interface when the imaging unit 2 as a camera is imaging an imaging target.

  Therefore, the user performs an imaging operation on the imaging unit 2 at each time t while confirming whether the imaging desired by the user himself / herself is performed by viewing the display on the display unit 5 as a camera preview. U (t) will be performed. In the present invention, the display unit 5 providing the camera preview does not display the captured image P1 (t) obtained by the imaging unit 2 as it is, but displays the processed image P2 (t) processed by the processing unit 4. By doing so, the following effects can be achieved. That is, there is an effect that the following first matter and second matter are achieved simultaneously.

  The first matter is that the processed image P2 (t) displayed on the display unit 5 is a raw image captured by the imaging unit 2 in the standpoint of the user who is not particularly aware that the processing is being performed. And the image is captured in a form desired by the user himself (for example, a form focused only on an article that the user is paying attention to). The second matter is that the recognition unit 3 performs recognition using the captured image P1 (t) (not the processed image P2 (t)), thereby realizing highly accurate recognition of the imaging target.

  Therefore, in the present invention, by simultaneously achieving the first matter that the image is taken in the form desired by the user (and the user can feel it) and the second matter that realizes highly accurate recognition. Thus, the above-mentioned problems of the prior art can be solved. That is, since the technique of preview display of the display unit 5 through the processing unit 4 is not adopted in the conventional technology, the captured image P1 (t) is displayed as it is as a preview and is also subject to recognition processing at the same time. From the standpoint, a satisfactory preview display was obtained, and even if the first item was achieved, the recognition process could not always be realized with high accuracy, and the second item was not necessarily achieved. Similarly, in the prior art, when the second item is achieved, the first item is not necessarily achieved. On the other hand, according to the present invention, the first matter and the second matter can be achieved simultaneously.

  As described above, the processing unit 4 performs processing on the captured image P1 (t) in such a manner that the first item and the second item are simultaneously achieved, thereby obtaining a processed image P2 (t). Therefore, when the processed image P2 (t) provides a preview display desired by the user, the captured image P1 (t) can realize high-precision recognition. As with the processing.

  The processing is typically performed in a manner that allows the user to voluntarily perform an imaging operation U (t) as described below.

  That is, when the user first tries to capture a desired imaging target in the preview image without particularly being aware of the imaging operation U (t1) at the initial time t1 when the user started imaging, the conventional technology As described in the above problem, the captured image P1 (t1) may not be in a preferable state for realizing high-accuracy recognition. Therefore, at such an initial time t1, the processed image P2 (t1) as the preview display target is processed so that it is not captured in the state desired by the user.

  Further, the user who has viewed the processed image P2 (t1) as the preview display of the undesirable state at the initial time t1 can consciously adjust the imaging operation U (t) after the time t1, thereby performing the subsequent time t2. In FIG. 5, the behavior of obtaining a processed image P2 (t2) as a preview display in a desired state is voluntarily taken. Therefore, the captured image P1 (t2) at the time t2 only needs to be such that high-accuracy recognition can be realized.

  As described above, as apparent from the consideration of the user behavior (imaging operation U (t)) from the initial time t1 to the subsequent time t2, the processing in the processing unit 4 is performed by the captured image P1 (t) and the processed image. What is necessary is just to set it as the process by which the relationship with P2 (t) satisfies the following three (relation 1)-(relation 3).

  As (Relationship 1), as described above as the initial time t1, when the captured image P1 (t1) is not suitable for high-accuracy recognition due to the imaging operation U (t1) by the user, the processed image P2 ( It is assumed that t1) is not a desired preview for the user.

  As (Relationship 2), when the captured image P1 (t2) is in a state suitable for high-accuracy recognition by the imaging operation U (t2) by the user, as described as the subsequent time t2, the processed image P2 ( At the same time, t2) is set to a state where the preview desired by the user is obtained.

  As (Relationship 3), an intermediate state between (Relationship 1) and (Relationship 2) is as follows. That is, as described as the user behavior from the initial time t1 to the subsequent time t2, if the above (Relationship 1) is established as a result of imaging, the user is voluntary. The relationship between (Relation 1) and (Relation 2) so that the user can be motivated to adjust the imaging operation to converge to the imaging operation state that satisfies the above (Relation 2). The relationship between the captured image P1 (t) and the processed image P2 (t) in the state is set.

  Note that, according to the above (Relationship 3), when (Relationship 2) is satisfied from the beginning of imaging in the user's imaging operation U (t) or in the middle of imaging, the user tries to capture the imaging target. (For example, in the case where a poster as an imaging target is continuously imaged and the transition to the imaging of another poster is not completed), the user satisfies (Relationship 2) The motivation to continue imaging while maintaining (imaging operation state) is given. Here, it has been described that the user is motivated to “converge” with respect to (Relationship 3), but the imaging operation state of “Convergence” and reached (Relationship 2) is not necessarily an optimal imaging operation state with only one point. It is not necessary, and it only has to exist as an imaging operation state having a certain range.

  As above, the processing process by the processing unit 4 is conceptually explained together with the “concept”. Details of the processing performed by the processing unit 4 will be described below. In addition, although the above (Relation 1) to (Relation 3) has been described, the following processing examples are examples that naturally satisfy (Relation 3) by satisfying (Relation 1) and (Relation 2). Yes.

  The processing unit 4 can process the captured image P1 (t) to obtain a processed image P2 (t) according to various embodiments. FIG. 3 is a functional block diagram of the processing unit 4 as an element configuration for realizing various embodiments. The processing unit 4 includes an enlargement unit 41, a reduction unit 42, a translation unit 43, and a brightness conversion unit 44. By enlarging in the enlarging unit 41, by reducing the reducing unit 42, by performing translation processing in the translation unit 43, and by performing lightness conversion processing in the lightness conversion unit 44, respectively, the captured images P1 (t) To obtain a processed image P2 (t).

  Here, each of the units 41 to 44 is assumed to be responsible for processing in each embodiment, and only one of the functional units may be applied, or processing may be performed by a combination of two or more arbitrary functional units. Processing may be realized. In addition, it is possible to set in advance by a manual judgment or the like by an administrator or the like which parameter is applied to which processing is applied to an imaging target that is actually assumed.

  In particular, when the feature information of a plurality of predetermined recognition objects is pre-registered in the storage unit 32 constituting the recognition unit 3 described with reference to FIG. It is sufficient to register in advance whether to apply the processing (or a combination thereof). Note that what kind of processing is appropriate depends on what is to be recognized and how the user's imaging operation can be performed. Will be described later in the detailed description of .about.44. Therefore, with respect to a series of recognition targets whose feature information is stored in the storage unit 32, pre-registration and processing unit 4 in the storage unit 32 so as to be a series of recognition targets for which common processing is appropriate. It is preferable to perform presetting of the processing content in.

  In other words, the predetermined processing performed in the processing unit 4 in conjunction with the information registered in advance in the storage unit 32 is a candidate for an imaging target and a candidate for how the imaging target is to be captured (for example, an imaging target Candidates are catalog pages and tend to be captured too close as imaging inappropriate for recognition processing, or conversely, candidates for imaging are posters and imaging inappropriate for recognition processing The processing may be determined in advance according to the imaging target, such as whether the image is likely to be imaged too far away and the candidate for the imaging method).

  In addition, as to what parameters the preset processing is applied to, the recognition unit 3 applies the processing unit 4 at the current time t2 based on the recognition result R (t1) at the past time t1 as described above. The parameter to be determined may be determined.

  Hereinafter, each of the units 41 to 44 will be described in detail.

  4 and 5 are diagrams for explaining the contents of the enlargement process as the processing by the enlargement unit 41 and the effects thereof.

Here, as shown in FIG. 4, the captured image P1 (t) obtained by the imaging unit 2 is assumed to have a size of 2N _x × 2N _y . That is, it is assumed that the captured image P1 (t) includes 2N _x pixels in the horizontal direction and 2N _y pixels in the vertical direction, so that the total is 2N _x × 2N _y pixels. Further, as shown in FIG. 4, the pixel position will be described on the assumption that the center of the captured image P1 (t) is the origin O of the xy coordinate system. According to the coordinate system, for example, the four vertices of the captured image P1 (t) are (± N _x , ± N _y ) (an arbitrary number of codes). (Normally, the upper left corner of the image is often described as the origin, but here, for convenience, the center of the image is taken as the origin.)

As shown in FIG. 4, the enlargement unit 41 generates a processed image P2 (t) that has been enlarged as a processing process, as an image that captures only a portion near the center O of the captured image P1 (t). In other words, the size (number of constituent pixels) is 2rN _x × 2rN _y and the four vertices are (± rN _x , ± rN _y ) (with a predetermined ratio r (0 <r <1) to determine how much is defined as a part. As with the captured image P1 (t), the processed image P2 (t) enlarged as a rectangular image having a boundary line parallel to the x-axis and y-axis and centering on the origin O. Is generated.

Since the processed image P2 (t) is configured as a part of the captured image P1 (t), the captured image P1 (t) (although the number of constituent pixels is smaller than that of the captured image P1 (t)). ), The image has a content as if it was photographed close to only a part of it, and has undergone a processing process as an “enlargement” process. Further, the enlargement unit 41 may adjust the resolution and output the processed image P2 (t) according to the screen area size displayed on the display unit 5. For example, if the display area of the display unit 5 is the size 2N _x × 2N _y as with the captured image P1 (t), the resolution is reduced to r times to reduce the size to 2N _x × 2N _y. You may make it output as image P2 (t).

  Therefore, the “B” -shaped region “P1 (t) \ P2 (t)” as shown in FIG. 4 as a region painted in gray is captured in the captured image P1 (t). Thus, it is excluded from the processed image P2 (t), and becomes an area invisible to the user who views the preview display on the display unit 5. For this reason, the region “P1 (t) \ P2 (t)” that cannot be seen by the user viewing the preview display can be referred to in the recognition unit 3 as a part of the captured image P1 (t). When an enlargement process is applied as an embodiment of the process, the present invention can be effective.

  The outline of the effect of processing by enlargement is as follows. Since the user assumes that the captured image P1 (t) captured by the imaging unit 2 is previewed on the display unit 5 when the imaging unit 2 is held over the imaging target, the enlarged processed image P2 (t ) Is displayed on the display unit 5, it is not assumed that it has been processed, and it is mistaken for the imaging unit 2 being too close to the imaging target. At the time of the misconception, (Relationship 1) holds. Then, since the user naturally moves the imaging target and the imaging unit 2 away from each other, the imaging information includes a large amount of features, and the recognition accuracy is improved. In other words, the state naturally shifts to a state in which (Relationship 2) is established, and such a state is maintained.

  With reference to the example shown in FIG. 5, a specific example of the effect of the processing by the enlargement described above will be described. In FIG. 5, as shown in [1], the imaging object Ob1 is the page paper (the whole) in a shopping catalog or the like, and includes a television (image) Ob2 and a sofa (image) Ob3 as partial objects. Take the case as an example. In FIG. 5, [2] and [3] indicate the captured image P1 (t) and the processed image in which (Relation 1) and (Relation 2) are satisfied when the processing content of the processing unit 4 is conceptually described. This is an example of P2 (t).

  That is, in [2] of FIG. 5, the imaging unit 2 is placed so that only the television Ob2 that the user is interested in is shown without the user being particularly aware of the imaging operation U (t1) at the initial time t1. As a result of the operation, the captured image P1 (t1) captures the region R511 where only the television Ob2 is only reflected, while the enlarged processed image P2 (t1) that appears as a preview display as indicated by the arrow is the television Ob2 is too large and is composed of the region R512 that does not capture the entire TV Ob2.

  Here, in [2] of FIG. 5, the region R511 of the captured image P1 (t1) is the entire recognition target Ob1 set by the administrator or the like (feature information is calculated in advance and stored in the storage unit 32). Since only the vicinity of TV Ob2, which is a part of the entire object), is recognized, it is an area that is inappropriate for recognition by the recognition unit 3. At the same time, the region R512 of the processed image P2 (t1) is in a state where the television Ob2 that the user is interested in is so large that only a part of the television Ob2 can be seen, so the state is not as desired by the user as a preview display. . From the above, [2] in FIG. 5 is a state in which the above (Relationship 1) is established.

  Therefore, the user shifts from the state [2] at the initial time t1 in FIG. 5 to an imaging operation to move the imaging unit 2 away from the imaging target in order to obtain a desired preview display in which the entire television Ob2 is shown. As shown in [3], at the subsequent time t2, the state of the imaging operation U (t2) that achieves a desired preview display is reached. Here, since the imaging unit 2 is moved away, the processed image P2 (t2) that is a preview display is configured as a region R522, and appropriately captures the entire television Ob2 that is an object of interest. At the same time, the captured image P1 (t2) is in a state of capturing the region R521 (without being perceived by the user), is in the vicinity of the television Ob2 that the user is interested in, and is captured at the initial time t1. Since it is in a state that can capture the sofa Ob3 that was not, the administrator assumed (that is, the feature information was stored in the storage unit 32) almost the entire page page Ob1 It is in a state. Accordingly, [3] in FIG. 5 is a state in which the above (Relationship 2) is established.

  As described above, according to the application of the enlargement unit 41 as in the catalog page Ob1 illustrated in FIG. 5, it is preferable to capture a wide area as an imaging target from the viewpoint of recognition processing. As a result, it is possible to realize highly accurate recognition even in the case where imaging is performed while focusing only on the television Ob2 and the sofa Ob3, which are a part thereof. When the relationship between the recognition target area for calculating the feature information for reference and the area that the user will actually pay attention to satisfies the above conditions, the enlargement unit 41 is exactly the same. Can be applied.

  When the catalog sheet is a recognition target (that is, when it is determined that the catalog sheet is recognized as a usage scene of the information terminal device 1), the storage unit 32 stores one or more of one or more catalogs. Are registered as recognition targets, and the processing unit 4 presets the processing of the enlargement unit 41 so that any of the plurality of pages is captured by the imaging unit 2. Can be obtained as a recognition result.

  Furthermore, the processing by enlargement in the present invention can also provide the following additional effects.

  That is, in the example of FIG. 5, the object that the user is interested in is TV Ob2, but the image is mostly composed of a flat area, and therefore, the feature information for recognition (local image feature amount) is to be extracted. In this example, only insufficiently can be extracted (feature points themselves are few). In the prior art, even if a target such as TV Ob2 that can extract feature information only in an insufficient manner is set as an individual recognition target, the accuracy of recognition processing cannot be obtained due to the lack of feature information itself. However, in the present invention, even if the user is paying attention to the TV Ob2 or the like that is inherently difficult to recognize due to the lack of feature information, as another target for supplementing the feature information Since the entire paper surface Ob1 including the sofa Ob3 and the like can be registered as a recognition target, the recognition result of the television Ob2 as a part thereof can be obtained through the recognition result of the entire paper surface Ob1.

  Here, in one embodiment, the processing unit 4 (enlargement unit 41) and the recognition unit 3 cooperate with each other so that the user can further connect to the television Ob2 that is a part thereof through the recognition result of the entire paper Ob1 as described above. Can be obtained in the recognition unit 3. Hereinafter, the case of the enlargement unit 41 will be described, but also in the case of the translation unit 43 described later, since there is a predetermined relationship that the processed image P2 (t) exists as a partial region of the captured image P1 (t), One embodiment can be implemented as well. Specifically, in the collation unit 33, as described above, the recognition result R (t) was obtained as to which recognition target stored in the storage unit 32 was captured in the captured image P1 (t). Thereafter, the following may be further performed.

  That is, as the first process, among the feature information of the recognition target stored in the storage unit 32 in the recognition result R (t), it corresponds to the feature information of the captured image P1 (t) during the matching process by the matching unit 33 Excerpt of the relationship set (corresponding to the feature information actually shown in the captured image P1 (t)), and the captured image P1 (t) occupies from the coordinate distribution of the feature points in the extracted feature information The area where The area estimation can be estimated as a rectangular area that covers the coordinate distribution of the feature points in the extracted feature information, for example. That is, the estimated rectangular area is an area including the coordinate distribution of the feature points in the extracted feature information, and the coordinates defining the area are stored in advance in the storage unit 32. It is given in the feature point coordinates in the target feature information. There are an infinite number of rectangular areas covering the coordinate distribution, but the area may be determined as one area based on some predetermined criterion. For example, an area that has the smallest area among an infinite number of rectangular areas may be used as the estimation result. In addition to the minimum area, a condition that the processed image P2 (t) and the shape information (vertical / horizontal size ratio) match may be imposed.

  Further, as the second process, the area estimated as described above is regarded as the entire area of the captured image P1 (t), and is processed as a partial area by applying the processing process in the processing unit 4 (enlargement unit 41). By applying the relationship when the image P2 (t) is obtained (information on the positional relationship between the captured image P1 (t) and the processed image P2 (t) as shown in FIG. 4), the entire estimated region (the captured image P1 It is possible to obtain a result as to which partial region in (t) region) is a region that the user is paying attention to as processed image P2 (t). Therefore, the identification result of the partial area is also given in the feature point coordinates in the feature information of the recognition target stored in advance in the storage unit 32.

  In the above embodiment, even if sufficient feature information does not exist in the region of the processed image P2 (t), information on the region of the processed image P2 (t) can be obtained as the user's attention region. In another embodiment, it is assumed that there is sufficient feature information in the region of the processed image P2 (t), and after the matching processing by the matching unit 33 is performed, the feature information of the captured image P1 (t) With respect to the feature information in the region of the processed image P2 (t), the feature information of the recognition target as the recognition result in the storage unit 32 is identified, and the correspondence relationship is identified. An area covering the coordinate distribution occupied by the feature information of the recognition target may be estimated as the area of the processed image P2 (t). The area covering the coordinate distribution can be determined as in the above embodiment.

  6 and 7 are diagrams for explaining a reduction process as a processing process by the reduction unit 42 and its effect.

Here, as shown in FIG. 6, the captured image P1 (t) obtained by the imaging unit 2 has a size of 2N _x × 2N _y and takes the coordinate system xy so that the center O is the origin, and the four vertices Is assumed to be (± N _x , ± N _y ) (double sign optional). That is, for the description of the captured image P1 (t) and the processed image P2 (t), the same coordinate system as described in FIG. 4 is used, and the size of the captured image P1 (t) is shown in FIG. Same as the case.

As shown by four dotted lines in FIG. 6, the reduction unit 42 performs a reduction process as a reduction process by reducing the captured image P1 (t) by a predetermined ratio r (0 <r <1). P2 (t) is obtained. That is, the processed image P2 (t) has a size of 2rN _x × 2rN _y , the center is the origin O, and the four vertices are (± rN _x , ± rN _y ) (decoding arbitrary). Note that the reduction ratio r uses the same character r as in FIG. 4 described for the enlargement process, but r for enlargement and r for reduction are not necessarily the same. In addition, the processed image P2 (t) obtained by the reduction process shows the entire range shown in the original captured image P1 (t) (unlike the enlargement process), and the captured image P1 (t ) Is a processed image P2 (t) that has been reduced so that the pixels are uniformly reduced by r times.

  The outline of the effect of the processing by reduction is as follows. When the user holds the imaging unit 2 over the recognition target, it is assumed that the captured image P1 (t) captured by the imaging unit 2 is previewed on the display unit 5, and thus the reduced processed image P2 (t ) Is displayed on the display unit 5, it is not assumed that it has been processed, and it is mistakenly assumed that the imaging unit 2 is too far from the imaging target. Then, since the user naturally moves the imaging target and the imaging unit 2 to reduce the distance, the imaging information includes a large amount of features, and the recognition accuracy is improved.

  With reference to the example shown in FIG. 7, a specific example of the effect of the processing by the enlargement described above will be described. In the example of FIG. 7, the imaging object Ob4 shown in [1] is, for example, a poster, and when captured by the user due to circumstances of the place where the image is attached, the imaging object Ob4 tends to be captured from a distance. Therefore, as shown in [2], at the initial time t1, the user captures the poster Ob4 from a distance, and the captured image P1 (t1) occupies the region R711. In this case, the processed image P2 ( t1) is reduced and configured as a small region R712. Therefore, the user who sees the region R712 as a preview display feels small and far away, and performs an imaging operation U (t) that brings the imaging unit 2 closer to the poster Ob4 after time t1.

  In this way, as shown in [3] of FIG. 7, at the subsequent time t2, the captured image P1 (t2) is obtained as occupying the range R721 closer to the poster Ob4, and the processed image P2 (t2) of the preview display is obtained. Since the range R721 is reduced to occupy the range R722 and the poster Ob4 is no longer too small, the user does not feel that it is too far away.

  The above examples [2] and [3] in FIG. 7 also establish the above-described (Relation 1) and (Relation 2). That is, in [2], the captured image P1 (t1) configured as the range R711 is too small compared to the case where the poster Ob4 to be recognized is pre-registered by the administrator or the like. Some feature points cannot be calculated due to insufficient resolution, and are reflected in a state unsuitable for recognition processing, and the processed image P2 (t1) configured as the range R712 is too small for the poster Ob4, The preview display is not as desired by the user. Therefore, [2] corresponds to (Relationship 1), and [3] as a result of eliminating this is (Relationship 2).

  FIGS. 8 and 9 are diagrams for explaining the translation process as the machining process by the translation unit 43 and its effect.

Here, as shown in FIG. 8, the captured image P1 (t) obtained by the imaging unit 2 has a size of 2N _x × 2N _y and takes the coordinate system xy so that the center O is the origin, and the four vertices Is assumed to be (± N _x , ± N _y ) (double sign optional). That is, for the description of the captured image P1 (t) and the processed image P2 (t), the same coordinate system as described in FIG. 4 is used, and the size of the captured image P1 (t) is shown in FIG. Same as the case.

  As shown in the processed image P2 (t) subjected to the translation processing in FIG. 8, the translation processing moves the center C of the processed image P2 (t) with respect to the center (origin O) of the captured image P1 (t). Configured as a process. That is, by extracting a region where the center C is displaced from the origin O as a rectangular region of a part of the captured image P1 (t), a processed image P2 (t) to which translation processing has been added is obtained.

  Therefore, the translation process is the same as the enlargement process described in FIG. 4 (a process in which the processed image P2 (t) is limited only to a region that occupies a portion r of the captured image P1 (t)). Further, it can be regarded as a process in which the center position is moved. As in the case of the enlargement process, the region “P1 (t) \ P2 (t)” contributes to the improvement of the recognition accuracy by being captured in the captured image P1 (t), but the processed image P2 ( This is excluded from t), and is an area that cannot be seen by the user who views the preview display on the display unit 5.

  Note that, as will be described later with reference to FIG. 9, by displaying the processed image P2 (t) as a preview, the user can correct the imaging operation U (t) in order to obtain an appropriate preview display for the user. This is realized in the form of motivating to move the imaging unit 2 in the direction of the vector V shown in FIG. Here, as shown in FIG. 8, the vector V is a vector from the center C of the processed image P2 (t) to the center O of the captured image P1 (t).

  The outline of the effect of the processing by translation (movement of the image center position) is as follows. When the imaging unit 2 is installed off the center of the display unit 5 and the user holds the display unit 5 over the front of the imaging target, the captured image P1 (t) captured by the imaging unit 2 is displayed on the display unit. 5 is assumed to be previewed on the display unit 5. Therefore, when the processed image P2 (t) translated in the direction of emphasizing the shift of the imaging target (the direction of the vector V in FIG. 8) is displayed on the display unit 5 Therefore, it is assumed that the imaging unit 2 is greatly deviated from the imaging target without assuming that it has been processed. Then, the user naturally moves the object so that the positional relationship between the imaging target and the imaging unit 2 faces each other (moves in the direction of the vector V in FIG. 8), and thus the feature amount is present in the captured image P1 (t). Many are included and the recognition accuracy is improved. For example, if the imaging unit 2 is installed at the right end of the back of the display unit 5 on a smartphone or the like, if the display unit 5 itself is held in front of the imaging target, the imaging target is imaged to the left in the imaging information. It is desirable to translate to emphasize the shift to the right.

  With reference to the example shown in FIG. 9, the specific example of the effect of the process by the translation outlined above will be described. In FIG. 9, as shown in [1], the imaging target is the entire poster Ob1 as in [1] in FIG. 4, and the television Ob2 and sofa Ob3 are included as its partial configuration. Similarly to the example of FIG. 4, since the user is interested in the television Ob2 in FIG. 9, the user should be captured even though the entire poster Ob1 should be imaged from the viewpoint of recognition accuracy. It is assumed that there is a tendency to capture only TV Ob2.

  Accordingly, as shown in [2] of FIG. 9, at the initial time t1, the user performs imaging so that the captured image P1 (t1) occupies a range R911 in which only the television Ob2 is captured. As a result, the translation processing shown in FIG. 8 (transverse processing from O to C in the lower left direction as opposed to the vector V) is applied, and the processed image P2 (t1) as a preview occupies the region R912. Is a region in which the TV Ob2 is shifted to the upper right (as opposed to the lower left translation process) and protrudes outside the image.

  Therefore, after time t1, the user moves the imaging unit 2 to the upper right so as to correct the preview that protrudes to the upper right, and as a result, as shown in [3] of FIG. 9 at time t2, the captured image P1 (t2 ) TV Ob2 is captured in the lower left and sofa Ob3 is captured in the upper right, and occupies the area R921 that almost captures the entire poster Ob1. At this time, the processed image P2 (t2), which is a preview display, occupies the region R922, and is in a state where the television Ob2 as desired by the user is captured at the center.

  As described above, the examples of [2] and [3] in FIG. 9 are also obtained by satisfying (Relation 1) and (Relation 2) in FIG. 4 explaining the poster Ob1 which is the same object. This is the same as described for [2] and [3]. Further, as is clear from the examples of [2] and [3] in FIG. 9, the direction of translation processing (the direction of the reverse vector of the vector V in FIG. 8) is the direction in which the user is assumed to image in a biased manner. It is preferable to set the same.

The lightness conversion unit 44 performs processing so that the gradation of the processed image P2 (t) is emphasized or reduced with respect to the gradation of the captured image P1 (t) by lightness conversion. In the lightness conversion unit 44, it is not necessary to change the area occupied by the processed image P2 (t) or the size of the captured image P1 (t) as in the above-described units 41 to 43. . If the captured image P1 (t) has a size of 2N _x × 2N _y , the brightness-converted processed image may have a similar size of 2N _x × 2N _y .

  When the processed image P2 (t) is obtained by the lightness conversion unit 44, the following effects are obtained. That is, when the user holds the imaging unit 2 over the imaging target in a situation where the gradation of the captured image P1 (t) is impaired due to specular reflection, excess or insufficient light quantity, and the feature points of the region are lost, Since it is assumed that the captured image P1 (t) captured by the imaging unit 2 is previewed on the display unit 5, the brightness-converted processed image P2 (t) that emphasizes the gradation is displayed on the display unit 5. If it is, it is not assumed that it has been processed, and it misidentifies that the amount of light and the light source are inappropriate. Then, since the user naturally corrects the imaging operation U (t) by moving it so as to correct the light amount and the light source, the captured image P1 (t) includes a lot of features, and the recognition accuracy Will improve.

  If a situation such as overexposure occurs due to the light source being too strong, the tone is emphasized (that is, the overexposure is further emphasized). When it is assumed that the image is blurred because it is weak and dim, the lightness conversion unit 44 performs lightness conversion processing with a predetermined parameter so as to reduce the gradation (that is, to further blur). It is preferable to do. In any case, when imaging is performed in an environment in which the gradation is impaired (although the cause of the damage is reversed), the user perceives that the gradation is impaired with more emphasis on the user. By doing so, it is possible to prompt the user to re-image the image so as not to impair the gradation.

  As described above, also in lightness conversion, (Relation 1) and (Relation 2) are established before and after correction of the user's imaging operation.

  As described above, according to the present invention, the imaging target can be recognized with high accuracy by imaging the imaging target with the imaging unit 2. In particular, by using the processed image P2 (t) instead of the captured image P1 (t) for the preview display to the user, the captured image P1 (t) suitable for the recognition process is acquired voluntarily for the user. Since it is possible to prompt the user to perform imaging using the imaging unit 2 in a state, highly accurate recognition is possible.

  Hereinafter, supplementary matters such as other embodiments of the present invention will be described.

  (1) Regarding the parameter setting in the processing process (for example, the enlargement ratio 1 / r in the enlargement process in FIG. 4), what is set in advance by the administrator or the like may be used. In order for an administrator or the like to determine appropriate parameters in advance, for example, the following may be performed.

  Specifically, the number, size, and type of recognition target feature points registered in the storage unit 32 can be used. When using the number of feature points, it is desirable to make the scaling factor inversely proportional to the number of feature points. For example, as the number of feature points is smaller, the imaging target is greatly enlarged. When using the size of the object to be imaged, it is desirable to make the scaling factor proportional to the size. For example, the small imaging target is guided so that the imaging target is reflected in the imaging information by being reduced, and the large imaging target is guided so that the entire imaging target is reflected in the imaging information by being enlarged. When the type of the imaging target is used, it is desirable to make the scaling factor proportional to the assumed imaging distance. For example, an imaging target that is assumed to have a long imaging distance (a poster or the like that cannot be moved because it is fixed) is reduced, and an imaging target that is assumed to be a short imaging distance (a product that can be taken by hand such as a catalog) is It is desirable to enlarge.

  In addition, the number, size, and type of feature points of the recognition target as described above are the same or substantially the same in each of the series of recognition targets stored in the storage unit 32, and feature points as typical values The number, size, and type of each may be used in a fixed manner. That is, for example, it is preferable not to register the poster group that is supposed to be photographed at a distance and the catalog page group that is supposed to be photographed at the same time in the storage unit 32.

  Similarly, when the objects to be stored in the storage unit 32 for recognition by the recognition unit 3 are grouped in advance based on the commonality of how the images are captured, and the user performs image capture and recognition May specify in advance which group is to be recognized, and perform common processing according to the group. That is, in the above example, which group of the poster group and the catalog page group is recognized is given to the information terminal device 1 as prior knowledge, and a predetermined processing suitable for the group is performed. You may make it perform.

  Alternatively, the processing instruction I (t) as described above may be generated in the recognition unit 3 so that the parameters of the image conversion process are changed according to the tendency of the user's imaging method. Specifically, the original recognition target that is assumed to be recognized by the recognition unit 3 is compared with the imaging target included in the imaging information after recognition, and a parameter corresponding to the imaging tendency of the user at the initial time t1 is set. It is desirable to set parameters so as to correct the difference in the next and subsequent imaging.

  (2) With regard to the hardware configuration for realizing each part (each part described with reference to FIGS. 1, 2, and 3) by the information terminal device 1, a hardware configuration in a normal computer can be employed.

  That is, the hardware configuration of the information terminal device 1 that implements each unit of FIGS. 1 to 3 includes the configuration of a desktop computer, a laptop computer, or another general computer in addition to a mobile terminal such as a smartphone or a tablet terminal. Can be adopted. That is, a CPU (Central Processing Unit), a temporary storage device that provides a work area to the CPU, a secondary storage device that stores data such as programs, various input / output devices, and data communication between them It is possible to adopt a general computer hardware configuration including a bus to be carried. Each part of FIGS. 1 to 3 is realized by the CPU reading and executing the program stored in the secondary storage device. The present invention can also be provided as such a program. Various input / output devices include a camera for image acquisition, a display for display, a touch panel and keyboard for receiving user input, a microphone / speaker for inputting / outputting audio, a communication interface for communicating with the outside by wire / wirelessly, You can use the one according to the required function.

  (3) The recognition unit 3 has been described with respect to the case of performing recognition based on the local image feature amount by the element configuration of FIG. 2, but other types such as character recognition, code recognition such as QR code (registered trademark), template matching, etc. In the case of performing various types of known recognition, the recognition can be performed in the same manner as the element configuration in FIG. That is, the feature information F (t) obtained from the captured image P1 (t) by the calculation unit 31 may be replaced with information used in the various recognition processes. Further, the feature information stored in the storage unit 32 may be replaced with information used in the various recognition processes. Also in the collation unit 33, the collation process may be replaced with one in the collation process performed in the various recognition processes.

  (4) The processing of each unit in FIG. 1 and the like has been described as being performed at each time t. Each time t may be any time that is processed in real time at a predetermined rate. Further, all or part of the processing of each unit shown in FIG. 1 or the like may be performed only at time t designated by the user. For example, the processing of the recognition unit 3 may be performed when the user himself / herself determines that an appropriate preview display has been obtained and designates that in the information terminal device 1 or after the designated time. .

(5) In the above description, the imaging size of the imaging unit 2 (the number of vertical and horizontal arrays of imaging elements) and the display size of the display unit 5 (the number of vertical and horizontal arrays of display elements) are, for example, FIGS. As described with reference to 8 etc. as the size N _x × N _y etc., it was assumed that they were common to each other. Here, if the imaging size of the imaging unit 2 and the display size of the display unit 5 are different, the resolution of the captured image P1 (t) obtained by the imaging unit 2 is converted to the display size of the display unit 5 May be regarded as “captured image P1 (t)” in the above description, and the present invention may be applied.

  That is, in FIG. 1, a resolution conversion unit, which is a functional unit (not shown), is further provided between the imaging unit 2 and the processing unit 4, and the resolution conversion unit adapted to the size of the display unit 5 has been described above. The processing target in the processing unit 4 may be used. When the preview function is used in the information terminal device 1 (regardless of the present invention), when such a resolution conversion unit is applied as a standard due to a difference in imaging size and display size, When applying the present invention, the above processing of the processing unit 4 may be applied to the captured image P1 (t) whose resolution has been converted by the standard application.

  For example, when the imaging size of the imaging unit 2 is 800 × 600 and the display size of the display unit 5 is 400 × 300, when applying the embodiment in which the reduction unit 42 reduces to 1/2 times, as follows: It may be. That is, the captured image P1 (t) obtained at a size of 800 × 600 is subjected to the reduction processing of the reduction unit 42 by reducing the resolution to 1/2 to match the display size of the display unit 5 to 400 × 300, Further, a size 200 × 150 may be obtained as the processed image P2 (t).

  Conversely, when the image pickup size is 800 × 600 and the display size is 400 × 300, the enlarged portion 41 has a half partial region (partial region P2 when r = 1/2 in FIG. 4). (t)) In the case of applying the embodiment in which only the image is displayed, the one obtained by cutting out a partial area of the imaging size 800 × 600 is a size 400 × 300, which matches the display size, and therefore undergoes resolution conversion. Alternatively, the processing of the enlargement unit 41 can be applied.

  (7) As another embodiment of the enlargement unit 41, as described with reference to FIG. 4, an image obtained by cutting out only a part of the captured image P1 (t) is not the processed image P2 (t), but the captured image P1 ( An image obtained by enlarging the entire t) with a predetermined magnification may be used as the processed image P2 (t). The other embodiment corresponds to applying the reduction unit 42 described with reference to FIG. 6 at a magnification r such that r> 1. As an effect, similar to the above-described embodiment of the enlargement unit 41, the captured image P1 (t) is misidentified as being too large for the user and too close to the imaging target, and the imaging target is more It is possible to promote an action to take an image at a distance.

  Note that when another embodiment described above is applied to the enlargement unit 41, the imaging size is smaller than the display size, and resolution conversion is not applied to the standard preview display, and only a part of the entire display size is applied. Preferably, the captured image is in a state of being arranged, and the user can be encouraged to take the above-described action by arranging the captured image that has been arranged only in the part larger by the processing of the enlargement unit 41. .

  (8) As another embodiment of the reduction unit 42, the captured image P1 (t) is not reduced to obtain the processed image P2 (t) as described in FIG. No reduction processing is added, and a processed image P2 (t) may be obtained by adding a pincushion type distortion with a predetermined parameter. For example, by contracting by a predetermined rate toward the center of the image, it is possible to cause the user who captures an image to misidentify that the imaging target is far away, as in the case of the embodiment of the reduction process. it can. Further, a reduction process and a process for adding a pincushion type distortion may be combined.

  1 ... information terminal device, 2 ... imaging unit, 3 ... recognition unit, 4 ... processing unit, 5 ... display unit

Claims (10)

An imaging unit that captures an imaging target and obtains a captured image;
A recognition unit that analyzes the captured image and recognizes the imaging target;
A processing unit that performs a predetermined processing on the captured image to obtain a processed image;
A display unit that displays the processed image as a preview,
The predetermined processing process is a processing process determined in advance according to a candidate for an imaging target and a candidate for how to perform the imaging,
When the imaging unit is imaging the imaging target and the obtained captured image is inappropriate for recognition by the recognition unit, the obtained processed image is inappropriate as a preview in the display unit. State, and
When the captured image is in a state suitable for recognition by the recognizing unit when the captured image is captured by the imaging unit, the processed image obtained is in a suitable state as a preview in the display unit. An information terminal device characterized by that. The predetermined processing process is a process in which a part of the captured image is extracted as a processed image;
The state inappropriate for recognition is an inappropriate state due to the imaging unit being too close to the recognition target, and the state inappropriate as a preview in the display unit is that the recognition target is too large. The information terminal device according to claim 1, wherein the information terminal device is in an inappropriate state. The predetermined processing is an enlargement process,
The state inappropriate for recognition is an inappropriate state due to the imaging unit being too close to the recognition target, and the state inappropriate as a preview in the display unit is that the recognition target is too large. The information terminal device according to claim 1, wherein the information terminal device is in an inappropriate state. The predetermined processing is a reduction process;
The state inappropriate for recognition is an inappropriate state due to the imaging unit being too far away from the recognition target, and the state inappropriate as a preview in the display unit is due to the recognition target being too small. The information terminal device according to claim 1, wherein the information terminal device is in an inappropriate state. The predetermined processing is a process of adding a pincushion type distortion,
The state inappropriate for recognition is an inappropriate state due to the imaging unit being too far away from the recognition target, and the state inappropriate as a preview in the display unit is due to the recognition target being too small. The information terminal device according to claim 1, wherein the information terminal device is in an inappropriate state. The predetermined processing is processing for moving the center position of the processed image so as to be different from the center position of the captured image;
The state inappropriate for recognition is an inappropriate state due to the imaging unit imaging the recognition target deviating from its center, and the state inappropriate as a preview in the display unit is the recognition target. The information terminal device according to claim 1, wherein the information terminal is in a state of being shifted from the center. The predetermined processing is processing for enhancing or reducing the gradation of the processed image more than the gradation of the captured image by brightness conversion,
The state inappropriate for recognition is an inappropriate state due to the imaging unit imaging the recognition target with its gradation being impaired, and the state inappropriate as a preview in the display unit is the recognition The information terminal device according to claim 1, wherein the object is in a state where the gradation is lost.   The recognizing unit calculates a local image feature amount from the captured image, and determines that they are similar by comparing with a local image feature amount calculated in advance for each of a plurality of predetermined recognition targets for reference. The information terminal device according to claim 1, wherein the recognition target to be recognized is recognized as corresponding to the imaging target in the captured image. The recognizing unit calculates a local image feature amount from the captured image, and determines that they are similar by comparing with a local image feature amount calculated in advance for each of a plurality of predetermined recognition targets for reference. A recognition target to be recognized as corresponding to the imaging target in the captured image,
Based on the distribution of the feature point coordinates of the local image feature value for reference in the recognition target determined to be similar, the correspondence obtained with the local image feature value in the captured image, The information terminal device according to claim 2, wherein an area of the captured image and an area of the processed image as the partial area are estimated.   A program for causing a computer to function as the information terminal device according to any one of claims 1 to 9.