JP2013012105A - Object recognition support device, program, and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object recognition support device for determining image amount adequate for object recognition.SOLUTION: An object recognition support device 1 includes: a first feature extraction unit 11 for extracting a feature of an acquired image; a first feature recording unit 12 for recording the feature; a dispersion calculation unit 13 for calculating dispersion of a feature group recorded in the first feature recording unit 12; a dispersion recording unit 14 for recording the dispersion; and a convergence determination unit 15 for determining whether the dispersion is converged on the basis of a progress of the dispersion recorded in the dispersion recording unit 14 and, when the dispersion is converged, making a notification of the acquisition completion of the image. The object recognition support device 1 supports an object recognition device 3 to perform object recognition with the adequate number of images.

Description

  The present invention relates to a technology that supports object recognition, and relates to a technology that makes it possible to acquire an appropriate number of images for processing when acquiring an image for object recognition.

  In recent years, object recognition technology for estimating the name of an object included in an image is expected to be used in various industries.

  An example of a field of use is the health guidance service field. The health guidance service is a service that provides users with information on health guidance in order to prevent lifestyle-related diseases caused by uneven eating habits. Currently, in health guidance services, experts such as registered dietitians check the contents of meals and write comments based on meal images taken and transmitted by users, so that users can receive information on health guidance. providing.

  However, there is a problem that the cost of providing the service is high and the number of users who can use the service is limited because the work of an expert who checks the contents of the meal by viewing the image is performed.

  Service automation is indispensable in order to keep costs low so that more users can use health guidance services.

  Therefore, the same applies to other service provision fields that use object recognition. However, the above health guidance service uses object recognition technology to automatically recognize meal names from meal images, and to provide meal names. In addition, automatic analysis of eating habits is expected to reduce the work burden on humans (experts) and improve the efficiency of service provision.

  The object recognition process includes a feature extraction stage for extracting feature information of an image and an object recognition stage for estimating the name of an object included in the image based on the extracted feature information. When one still image is input to the object recognition device, the name of the object included in the image is output.

  12 is a diagram showing an example of an image obtained by photographing a certain object (ramen) from a certain direction, and FIG. 13 is a diagram showing an example of an image obtained by photographing the object (ramen) of FIG. 12 from another direction.

  When the image shown in FIG. 12 is input to the object recognition apparatus, the object recognition apparatus recognizes the object of the image as “ramen” based on the features extracted from the image, and outputs the object name.

  However, when the image of FIG. 13 is input, the object recognition apparatus may erroneously estimate the object of the image as another “fried rice” and may not be able to recognize the correct object “ramen”.

  In this way, when object recognition is performed depending on information obtained from only one image, the features necessary for proper object recognition are not extracted depending on the shooting conditions of the input image. There is a problem that it is estimated as an object. In order to solve this problem, there is a method in which the same object is shot a plurality of times while changing conditions such as the shooting direction and the estimation results of these image groups are used in an integrated manner.

  An object recognition apparatus using a plurality of image groups further includes a recognition integration stage in a feature extraction stage and an object recognition stage. Such an object recognition device extracts features for each image in the feature extraction stage when a plurality of images taken while changing conditions such as the shooting direction are input to the same object, and each image in the object recognition stage. The object name is estimated based on the characteristics of. Then, in the recognition integration stage, the object recognition device integrally determines the recognition result (object name) for each image and determines one object name. In the recognition integration stage, for example, one object name that is highly likely to be correct is determined from the object names output in the object recognition stage by a majority vote or the like.

  Therefore, the object recognition apparatus using a plurality of images performs object recognition by using images under various shooting conditions in an integrated manner, and thus can correctly recognize an object name even if an image that is easily misrecognized is included.

  By the way, in the object recognition processing, it is known that the feature points to be extracted are narrowed down at the feature extraction stage in order not to increase the processing amount.

JP 2010-218051 A US Pat. No. 6,711,293 International Publication No. 2007/128452

Churka et al. (Csurka, G., Bray, C., Dance, C. and Fan, L.), "Visual Categorization with Bags of Keypoints", (Czech Republic), Computer Vision In Proc. Of ECCV Workshop on Statistical Learning in Computer Vision, EECV, 2004, p. 59-74, Taichi Joutou et al. (Taichi Joutou, Keiji Yanai), “A Food Image Recognition System with Multiple Kernel Learning”, (Egypt), 16th International Conference on IEEE Image Processing ( Proceedings of the 16th IEEE International Conference on Image Processing), November 7-10, 2009, p. 285-288 David G. Lowe, “Object Recognition from Local Scale-Invariant Features”, International Conference on Computer Vision, (Greece), September 1999, p. 1150-1157 Herbert Bay et al. (Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool), "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding Understanding (CVIU)), (Netherlands), 2008, Vol. 110, no. 3, p. 346-359

  The object recognition apparatus using a plurality of input images in the above-described conventional method can perform object recognition robustly with respect to shooting conditions such as the shooting direction of an object by using the plurality of input images. However, it has not been clarified how many images are used for object recognition to be necessary and sufficient for recognition. Therefore, in object recognition processing, processing is performed using all input images.

  Furthermore, when the input image used for object recognition is an image taken by the user, it is left to the user to determine the timing for finishing the shooting. Therefore, in the actual usage situation, the user inputs an image obtained by photographing the target to the object recognition device, and if the recognition result is incorrect, the user performs further photographing and adds the photographed image (input image). Is being repeated.

  If the image used for object recognition is insufficient, there is a possibility of erroneous recognition, but if it is excessive, the calculation burden in processing increases and becomes inefficient. Therefore, it is desirable that necessary and sufficient images are used for object recognition.

  However, a user who captures or inputs an image cannot recognize how much image is captured or input and is sufficient for object recognition, and knows where to end the operation. I feel the stress of not being able to. Furthermore, there is a problem that the user will continue to perform useless input operations.

  In addition, the object recognition processing with an excessive amount of images also has a problem that the calculation load increases and is wasteful and inefficient for the object recognition apparatus.

  As described above, in the conventional method, there is a problem that it is unclear how many images should be acquired for the object recognition apparatus.

  An object of the present invention is made in view of the above problems, and automatically determines whether or not a reasonable number of images have been acquired for performing object recognition, and notifies the completion of acquisition when the number of images is sufficient. An object of the present invention is to provide an apparatus that supports object recognition processing.

  Another object of the present invention is to provide a program for causing a computer to execute processing realized by the object recognition support apparatus. Furthermore, another object of the present invention is to provide a processing method in which a computer executes a processing process realized by the object recognition support apparatus.

  An object recognition support apparatus disclosed as one aspect of the present invention includes: 1) a first feature extraction unit that extracts features of an image acquired by an image input unit; and 2) the extracted information to support object recognition processing. A first feature recording unit that records the calculated feature, 3) a variance calculation unit that calculates the variance of the feature group recorded in the first feature recording unit, and 4) a dispersion recording unit that records the calculated variance. 5) determining whether the dispersion has converged based on the dispersion transition recorded in the dispersion recording section, and, if the dispersion has converged, a convergence determining section for notifying completion of image acquisition; Prepare.

  According to the disclosed object recognition support apparatus, it is possible to prevent the user from performing unnecessary image input operations by notifying completion of acquisition when an image amount appropriate for object recognition is acquired. In addition, it is possible to prevent the object recognition apparatus from processing a useless image.

It is a figure which shows the structural example of the health guidance service system in one Embodiment. It is a figure which shows the structural example of the object recognition assistance apparatus in one Embodiment. It is a figure which shows the structural example of the image use determination apparatus in one Embodiment. It is a figure which shows the structural example of the object recognition apparatus in one Embodiment. It is a figure which shows the example of a processing flow of an image use determination apparatus. It is a figure which shows the example of the use image employ | adopted by the image use determination apparatus. It is a figure which shows the example of dispersion | distribution about the feature vector group of an image. It is a figure which shows the example of a processing flow of the object recognition assistance apparatus. It is a figure which shows the example of calculation of dispersion | distribution of the feature vector group performed by the process of step S23. It is a figure which shows the flow of a process of the health guidance service system in one Embodiment. It is a figure which shows the example of an estimation of the object name by object recognition. It is a figure which shows the example of an image which image | photographed the object from a certain direction. It is a figure which shows the example of an image which image | photographed the object of FIG. 12 from another direction.

  Hereinafter, an object recognition support apparatus disclosed as one aspect of the present invention will be described.

  As an embodiment of the disclosed object recognition support apparatus, an example in which a health guidance service system that is one of service providing systems having an object recognition support apparatus and an object recognition apparatus will be described.

  FIG. 1 is a diagram illustrating a configuration example of a health guidance service system according to an embodiment.

  The health guidance service system includes an object recognition support device 1, an image use determination device 2, an object recognition device 3, a service providing device 4, and a user device 5. In the present embodiment, the object recognition support device 1 and the image use determination device 2 are provided in the user device 5.

  The object recognition support apparatus 1 notifies completion of acquisition when an amount (number) of images appropriate for object recognition can be acquired. The object recognition support device 1 extracts the features of the acquired image group, determines whether or not the change has converged from the change of the feature group of the acquired image, and if the change has converged, a reasonable number It is determined that the image has been acquired, and an acquisition completion indicating that the image has been acquired is notified.

  The image use determination device 2 determines whether the image acquired by the image input unit 51 of the user device 5 is used for object recognition. When the image is acquired, the image use determination device 2 extracts the feature from the acquired image, and uses the image acquired this time when the extracted feature is different from the feature of the image acquired in the past to a certain degree. Adopt as an image.

  The object recognition device 3 recognizes an object included in the image based on the features extracted from the plurality of images, and estimates the corresponding object name.

  The service providing device 4 is a server device that provides predetermined service information to the user based on the estimated object name. The service providing device 4 provides the user with information regarding health guidance using the object name estimated by image recognition.

  The service providing device 4 includes a calorie database 41 and a calorie calculation unit 42.

  The calorie database 41 stores meal names and calories in association with each other.

  The calorie calculation unit 42 refers to the calorie database 41, acquires the calorie corresponding to the object name, and transmits the acquired calorie to the information output unit 52 of the user device 5.

  The user device 5 is a client device used by the user. The user device 5 includes an image input unit 51 and an information output unit 52.

  The image input unit 51 acquires a moving image or a still image in which a target to be recognized is captured. In the present embodiment, the image input unit 51 is implemented by a camera that captures a moving image or a still image. In the following description, the image data acquired by the image input unit 51 is referred to as an input image.

  The information output unit 52 acquires information (calories) transmitted from the service providing apparatus 4 and outputs the information (calorie) to a display or the like.

  FIG. 2 is a diagram illustrating a configuration example of the object recognition support apparatus 1 according to an embodiment.

  The object recognition assisting apparatus 1 determines whether a number of image data appropriate for object recognition has been collected. For such a determination function, the object recognition support apparatus 1 includes a first feature extraction unit 11, a first feature recording unit 12, a variance calculation unit 13, a variance recording unit 14, and a convergence determination unit 15.

  The first feature extraction unit 11 extracts a feature vector as a feature from an image adopted as an image used for object recognition (hereinafter referred to as a used image).

  The first feature recording unit 12 records the feature vector extracted by the first feature extraction unit 11.

  The variance calculating unit 13 extracts the feature vector of the acquired used image extracted by the first feature extracting unit 11 and the feature vector of the used image group extracted by the previous process recorded in the first feature recording unit 12. Calculate the variance for each dimension from all feature vectors with the group.

  The variance recording unit 14 records the variances calculated by the variance calculating unit 13 in the order of processing.

  The convergence determination unit 15 obtains a transition of the variance based on the variance calculated in the past recorded in the variance recording unit 14 and the variance calculated this time, and notifies the acquisition completion when the variance has converged. The completion of acquisition is notified to the image input unit 51, the image use determination device 2, the object recognition device 3, and the like.

  FIG. 3 is a diagram illustrating a configuration example of the image use determination device 2 according to an embodiment.

  If there is an interval in the shooting timing when the input image is shot, all shot images (input images) may be used for object recognition. However, when a moving image is shot, if all the shot images (frames) are used for object recognition, the image is redundant and the calculation load for object recognition increases. Therefore, it is necessary to thin out the input image. On the other hand, the thinning out of the input image reduces the information input to the object recognition, which may reduce the recognition rate. Therefore, it is necessary to be able to maintain variations of features obtained from the input image.

  The image use determination device 2 determines a use image while omitting only redundant input images having similar features while maintaining variations of features obtained from the input images. For such a determination function, the image use determination device 2 includes a second feature extraction unit 21, a feature writing unit 22, a second feature recording unit 23, a distance calculation unit 24, and a use determination unit 25.

  The second feature extraction unit 21 extracts features from the input image.

  The feature writing unit 22 records the feature of the input image adopted as the use image by the use determination unit 25 in the second feature recording unit 23.

  The second feature recording unit 23 holds the features extracted from the use image.

  The distance calculation unit 24 calculates the difference between the feature of the input image extracted by the second feature extraction unit 21 and the feature of the use image recorded in the second feature recording unit 23.

  The use determination unit 25 selects the input image as the use image when the feature of the input image and the feature of the use image recorded in the second feature recording unit 23 are different from each other by a predetermined threshold or more. To do.

  In the present embodiment, the image use determination device 2 calculates a feature vector as a feature of an image, and uses the distance of the feature vector as a feature difference.

  In addition, when the moving image is acquired by the image input unit 51, the image use determination device 2 determines the use of the image for each frame extracted at a preset interval, and when the still image is acquired. , Image usage is determined in units of still image data.

  FIG. 4 is a diagram illustrating a configuration example of the object recognition device 3 according to an embodiment.

  The object recognition device 3 includes a feature extraction unit 31, an object recognition unit 32, and a recognition integration unit 33.

  The feature extraction unit 31 extracts features from each of the plurality of use images 1, 2,..., N. The object recognition unit 32 refers to an object recognition storage unit (not shown) that stores the feature of the image of each object and the object name in association with each other and refers to the object name based on the extracted feature for each used image. Estimate candidates. The recognition integration unit 33 integrates the estimation results of the object recognition unit 32 and determines an object name for the object included in the image.

  In the present embodiment, the object recognition device 3 executes a known object recognition process using a plurality of images, and a detailed description of the operation is omitted.

  First, the image use determination device 2 will be described in more detail.

  FIG. 5 is a diagram illustrating an example of a processing flow of the image use determination device 2.

  In the second feature recording unit 23 of the image use determination device 2, the feature writing unit 22 records the feature vector v ′ of the input image adopted as the use image in the past determination processing.

  When the image input unit 51 inputs an image obtained by photographing the object recognition target, the second feature extraction unit 21 of the image use determination device 2 acquires an input image from the image input unit 51 (step S11).

  Then, the second feature extraction unit 21 extracts features of the input image. Specifically, the second feature extraction unit 21 converts the input image into an N-dimensional feature vector v (step S12).

  The distance calculation unit 24 extracts the feature vector v ′ of the used image from the second feature recording unit 23. The distance calculation unit 24 calculates the distance d between the feature vector v of the input image and the feature vector v ′ of the used image (step S13).

  The use determination unit 25 compares the calculated distance d with the threshold value θ. If the distance d is larger than the threshold value θ, that is, if “distance d> threshold value θ” is satisfied (Y in step S14), the use determination unit 25 selects and outputs the input image as a use image (step S15). Further, the feature writing unit 22 stores the feature vector v of the input image selected as the use image in the second feature recording unit 23 as the feature vector v ′, and ends (step S16).

  On the other hand, if the distance d is smaller than the threshold value θ, that is, if “distance d> threshold value θ” is not satisfied (N in step S14), the use determining unit 25 discards the input image, and the process returns to step S11. It is.

  In the process of step S13, the distance calculation unit 24 calculates the distance d of the feature vectors v and v ′, for example, as in the following equation (1).

In Equation (1), N is the number of dimensions of the feature vector. v _i represents the value of the i-th dimension in the feature vector v. Similarly, v ′ _i represents the i-th value in the feature vector v ′.

  Alternatively, the distance calculation unit 24 may calculate the distance dn for each dimension of the feature vector by the following equation (2).

  In Expression (2), dn represents the distance related to the nth dimension of the feature vector.

  When the distance calculation unit 24 calculates the distance dn using the equation (2), the use determination unit 25 performs the condition determination of “distance dn> threshold θ” in each dimension n in the process of step S14. If one of the conditions is satisfied, the input image is selected and output as a use image.

  The image features extracted by the second feature extraction unit 21 are preferably the same as those of the object recognition support device 1 and the object recognition device 3. In object recognition, in general, a bag-of-words expression based on local feature amounts such as SIFT feature amounts and SURF feature amounts is often used. In the present embodiment, the object recognition support device 1, the image use determination measure 2, and the object recognition device 3 are assumed to use SIFT feature values.

Therefore, an appropriate value of the threshold θ varies depending on the feature amount to be used. For example, when a 1000-dimensional bag-of-words expression normalized so that the sum of each dimension by SIFT becomes 1 is used as the feature amount. The threshold value θ in the above formula (1) is set to about 10 ⁻⁴ to 10 ⁻³ , and the threshold value θ in the above formula (2) is set to about 10 ⁻² to 10 ^−1. .

  FIG. 6 is a diagram illustrating an example of a use image adopted by the image use determination device 2.

  FIG. 6 shows a plurality of image frames (first image, second image, third image,...) Adopted from the input image, which is a moving image, to the use image.

  The first frame (first frame) of the moving image data is always used as the used image because the feature of the used image to be compared is not retained. However, even if an image is taken every few frames from a moving image, it may be redundant. According to the image use determination device 2, for the image after the first frame of the moving image, the feature of the image of the extracted frame is compared with the feature of the image adopted as the use image, and the feature is different from the threshold value or more. Only the frames having “” are selected as use images as the determination condition is satisfied. Therefore, even if the input image is redundant, an image having a different feature is selected as the use image, and the redundant image can be thinned out while maintaining the variation of the feature.

  Next, the object recognition support apparatus 1 will be described in more detail.

  The object recognition support apparatus 1 calculates a variance for a feature vector group of input images acquired in the past in order to determine whether or not an appropriate number of images for object recognition has been acquired.

  In the present embodiment, the image use determination device 2 selects a use image to be used for object recognition from the input image. Therefore, the object recognition support device 1 calculates a variance for a feature vector group of the use image. In the embodiment in which the image use determination device 2 is not provided, the variance is calculated for the feature vector group of the input image acquired by the image input unit 51.

  FIG. 7 is a diagram showing an example of dispersion for a feature vector group of an image.

  The graph shown in FIG. 7 is an example of the variance for the feature vector group of the used image, and expresses the transition of the variance S with respect to the number t of used images acquired by the object recognition support apparatus 1.

  When using images (input images) that were taken while changing conditions such as the shooting direction are acquired, the variance of the variance calculated from the characteristics of the acquired images is initially large, but images with various shooting conditions As the value accumulates, the fluctuation range gradually decreases.

  Therefore, if the variance variation is large, it is likely that images captured under new conditions will be acquired in the future. If the variance variation converges, images captured under the new conditions may be input. It can be judged that there are few. That is, if the variance variation has converged, it can be determined that an image group having different features necessary for object recognition has already been acquired, and acquisition of an appropriate number of images has been completed.

  FIG. 8 is a diagram illustrating a processing flow example of the object recognition support apparatus 1.

  The first feature extraction unit 11 of the object recognition support apparatus 1 acquires a use image from the image use determination apparatus 2 (step S21).

  The first feature extraction unit 11 extracts the feature vector of the acquired use image and records it in the first feature recording unit 12 (step S22).

  The variance calculation unit 13 calculates variances for each dimension from all feature vectors based on feature vectors extracted in the past held in the first feature recording unit 12, and records the calculated variances in a distributed manner. It records in the part 14 (step S23).

  The convergence determination unit 15 determines whether or not the variance has converged based on the variance transition calculated in the past based on the variance held in the variance recording unit 14 (step S24). When it is determined from the transition of dispersion that the dispersion fluctuation has converged (“convergence” in step S24), the convergence determination unit 15 confirms that an appropriate number of images have been collected for object recognition. “Acquisition completion” is notified, and the process ends (step S25). When it is determined that the variance has not yet converged (“unconverged” in step S24), the convergence determination unit 15 displays “acquired” indicating that an appropriate number of images have not yet been collected for object recognition. “Not completed” is notified and the process ends (step S26). The convergence determination unit 15 may be terminated without performing the process of step S26.

  FIG. 9 is a diagram illustrating a calculation example of the variance of the feature vector group, which is executed in the process of step S23.

The variance calculation unit 13 extracts feature vector groups (X ₁ ,..., X, extracted from t use images (images 1,..., Image t) acquired so far and held in the first feature recording unit 12. _{From t 1} ), elements are extracted for the nth dimension, and an element vector N _tn is generated. When extracting from a use image, you may normalize so that the range which each element of a feature vector can take falls in 0-1.

Then, the variance calculation unit 13 calculates the variance S _nt for the nth dimension in the feature vector group of t images according to the following equation (3).

However, in Expression (3), the variable represented by “Bar over N _tn ” is the average of the element vectors N _tn and is calculated by the following Expression (4).

Every time the first feature extraction unit 11 of the object recognition support apparatus 1 acquires a use image and the feature vector is stored in the first feature recording unit 12, the variance calculation unit 13 performs the variance S _nt by the above processing. Are calculated for each dimension.

  In the process of step S24, the convergence determination unit 15 determines convergence using one of the following conditions based on the transition of the variance calculated by the variance calculation unit 13.

Condition 1: Variance variation has become sufficiently small Condition 2: Variance variation has continued to decrease. For example, the condition shown in the following equation (5) can be detected by L times This is done by confirming whether it has been established continuously.

S _nt −S _{n (t−1)} ≦ ε Equation (5)
However, in Formula (5), ε is a threshold value. If the possible range of each element in the feature vector is normalized to 0 to 1, the threshold ε is preferably set to about 0.001 to 0.1.

  The detection of convergence under the above condition 2 is performed, for example, by confirming whether the condition shown in the following equation (6) is satisfied L times continuously.

  In both cases of condition 1 and condition 2, L represents a period until the determination of convergence is finalized. If L is small, convergence determination can be made quickly, but the risk of erroneous determination increases. If L is large, the convergence determination takes time, but the risk of erroneous determination is reduced. As a tendency of variance transition, when variance is changing greatly, it often takes time to converge. Therefore, it is better to make a decision over time. On the other hand, when the variance is small, the convergence is often fast. Therefore, the determination may be made without taking time.

  Therefore, L is set to a value proportional to the variance change as shown in the following equation (7). Thereby, the convergence determination part 15 performs efficient convergence determination.

  In the above equation (7), K is a determination speed adjustment parameter. In general, K is preferably set to a value of about 100 to 1000.

  Note that L is calculated only at the first time when the conditions are continuously satisfied.

  The convergence determination unit 15 performs the above processing on the element vectors N1 to Nn. When the convergence determination unit 15 determines that more than half of the variances of the elements have converged, the convergence determination unit 15 determines that an appropriate number of used images have been collected.

  In the process of step S45, the convergence determination unit 15 notifies the image use determination device 2 of the completion of acquisition based on the determination that an appropriate number of use images have been collected. The image use determination process ends, and the recording of the use image ends.

  The convergence determination unit 15 notifies the user of acquisition completion. For example, the convergence determination unit 15 sounds an alarm at the information output unit 52 and notifies the user that the imaging phase has been completed. The alarm allows the user to know the timing of the end of image capturing.

  Furthermore, the convergence determination unit 15 may notify the object recognition device 3 of acquisition completion. The object recognition device 3 outputs the object name estimated from the use image acquired from the image use determination device 2 to the service providing device 4 when the acquisition completion notification is obtained.

  Next, the processing flow of the health guidance service system in this embodiment will be described.

  In the health guidance service system shown in FIG. 1, as a health guidance service, a meal name is automatically recognized from a meal image taken by a user using a camera such as a mobile terminal by using an object recognition device and an object recognition support device. Assume a service that outputs health guidance information including calorie intake.

  The user device 5 is a user's portable terminal, the image input unit 51 is implemented by a camera, and the information output unit 52 is implemented by a processing unit that displays information on the display of the portable terminal. The input image is an image of a meal photographed by the user, an object included in the input image represents a meal photographed by the user, and the object name represents a meal name.

  FIG. 10 is a diagram showing the flow of processing of the health guidance service system.

  The image input unit 51 of the user device 5 acquires an input image obtained by photographing the recognition target (step S31). The image use determination device 2 determines whether to use the input image as a use image (image use determination) (step S32). When it is determined that the input image is adopted as the use image (Y in step S33), the image use determination device 2 advances the process to step S34. If it is determined that the input image is not adopted as the use image (N in step S33), the process returns to step S31.

  The object recognition assisting apparatus 1 determines whether an appropriate number of images are collected for the used images acquired from the image use determining apparatus 2 (image number satisfaction determination) (step S34). When the object recognition assisting apparatus 1 determines that the variation has converged and the number of used image groups has already reached a reasonable number from the transition of the variation in the characteristics of the used image group to be acquired (Y in step S35), The process proceeds to step S36. When it is determined that the variation has not converged and the number of used images has not reached the appropriate number (N in step S35), the object recognition support apparatus 1 returns the process to step S31.

  The object recognition device 3 acquires a use image from the image use determination device 2 and performs object recognition until receiving a notification of acquisition completion from the object recognition support device 1, and performs object recognition (object) corresponding to the object (meal) of the use image. (Meal name) is output (step S36).

  The calorie calculation unit 42 of the service providing apparatus 4 collates with the calorie database 41 to obtain calories corresponding to “ramen” of the estimated object name (meal name). And the calorie calculation part 42 transmits the information containing the acquired calorie intake to the information output part 52 of the user apparatus 5 and displays it on the user apparatus 5 (step S37).

  FIG. 11 is a diagram illustrating an example of estimating an object name by object recognition.

  In the process of step S36, the feature extraction unit 31 of the object recognition device 3 sequentially uses use image groups (use images 1 to 4) in which the same object (meal) is taken from different directions as shown in FIG. The object recognition unit 32 acquires the object name candidate estimated by object recognition on the acquired use image. After the completion of the object recognition, the recognition integration unit 33 determines an object name by majority from the held object name candidates. In the case of FIG. 11, object name candidates “katsudon” and “ramen” are held, and “ramen” is determined as the object name by majority vote.

  Through the above processing, the health guidance service system can realize a health guidance service for the user.

  Next, another configuration example of each device of the health guidance service system will be described.

  In the configuration example illustrated in FIG. 1, the object recognition support device 1 and the image use determination device 2 are described as being provided in the user device 5. However, the object recognition support device 1 and the image use determination device 2 may be implemented as a preprocessing function of the object recognition device 3 and are configured between the image input unit 51 of the user device 5 and the object recognition device 3. It may be. For example, the object recognition support device 1 and the image use determination device 2 may be configured as devices external to the user device 5 or may be configured as an integrated device with the object recognition device 3.

  Further, the image use determination device 2 may be configured as one processing unit of the object recognition support device 1.

  Each device of the health guidance service system is realized by a computer system including hardware having a CPU and a memory and a software program, or dedicated hardware. That is, the object recognition support device 1, the image use determination device 2, the object recognition device 3, the service providing device 4, and the user device 5 are respectively a computing device (CPU), a temporary storage device (DRAM, flash memory, etc.) and a persistence. It can be implemented by a computer having a storage device (HDD, flash memory, etc.) and inputting / outputting data from / to the outside.

  Further, the object recognition support device 1 and the image use determination device 2 can each be implemented by a program executable by a computer. For example, in the case of the configuration shown in FIG. 1, a program describing the processing contents of functions that each of the object recognition support device 1 and the image use determination device 2 should have is provided to the user device 5. By executing the provided program by the user device 5 that is a computer, the processing functions of the object recognition support device 1 and the image use determination device 2 described above are realized on the computer.

  Furthermore, when the object recognition support apparatus 1 and the image use determination apparatus 2 are each implemented as a computer, the object recognition support apparatus 1 and the image use determination apparatus 2 can also read a program directly from a portable recording medium and execute processing according to the program. The program can be recorded on a computer-readable recording medium.

  As described above, according to the object recognition support apparatus 1 and the image use determination apparatus 2 disclosed as one aspect of the present invention, the following effects can be obtained.

  The object recognition support apparatus 1 can prevent wasteful acquisition (photographing or input) of images for object recognition.

  -According to the object recognition support apparatus 1, it is possible to suppress an increase in calculation burden by suppressing an image used for object recognition to an appropriate amount.

  According to the object recognition support apparatus 1, it is possible to notify a user who captures or inputs an image to be recognized of the operation end timing, and to eliminate the user's psychological stress with respect to the operation end timing.

  The image use determination device 2 can thin out redundant images while maintaining variations in image features used for object recognition.

  As described above, in the embodiment of the present invention, the case where the invention made by the inventor is applied to the health guidance service field has been described. However, the present invention is not limited to application in such a field, but can be applied to various fields that require assistive technology for object recognition, and various modifications are possible within the scope of the description. Of course it is possible.

DESCRIPTION OF SYMBOLS 1 Object recognition assistance apparatus 11 1st characteristic extraction part 12 1st characteristic recording part 13 Variance calculation part 14 Dispersion recording part 15 Convergence determination part 2 Image use determination apparatus 21 2nd feature extraction part 22 Feature writing part 23 2nd characteristic recording part 24 distance calculation unit 25 use determination unit 3 object recognition device 4 service providing device 5 user device

Claims (4)

An object recognition support device for supporting object recognition processing,
A first feature extraction unit for extracting features of the image acquired by the image input means;
A first feature recording unit for recording the extracted features;
A variance calculating unit that calculates the variance of the feature group recorded in the first feature recording unit;
A dispersion recording unit for recording the calculated dispersion;
A convergence determining unit that determines whether the variance has converged based on a variance transition recorded in the variance recording unit, and notifies the completion of image acquisition when the variance has converged. Characteristic object recognition support device. A second feature recording unit for recording features of a used image used for object recognition;
A second feature extraction unit for extracting features of the image acquired by the image input means;
A distance calculation unit for calculating a difference between the acquired image feature and the image feature recorded in the second feature recording unit;
When the calculated feature difference is greater than or equal to a certain value, the acquired image is selected as a use image acquired by the first feature extraction unit, and the feature of the use image is stored in the second feature recording unit The object recognition support apparatus according to claim 1, further comprising: To support the object recognition process,
Processing for extracting features of the image acquired by the image input means;
A process of recording the extracted features in a first feature recording unit;
Processing for calculating the variance of the feature group recorded in the first feature recording unit;
A process of recording the calculated variance in a variance recording unit;
An object for determining whether or not the dispersion has converged based on the transition of the dispersion recorded in the dispersion recording unit, and executing processing for notifying completion of image acquisition when the dispersion has converged Recognition support program. To support the object recognition process, the computer
Extract image features acquired by image input means,
Recording the extracted features in a first feature recording unit;
Calculating a variance of the feature group recorded in the first feature recording unit;
Record the calculated variance in a variance recording unit;
Determining whether the variance has converged based on the transition of the variance recorded in the variance recording unit;
An object recognition support method comprising notifying completion of image acquisition when the variance has converged.

Images