JP4664047B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method for learning or determining a shooting target included in a shot image from a shot image.

2. Description of the Related Art Conventionally, there are various feature amounts used when determining a shooting target included in a captured image from a captured image. Examples include color feature values and shape (composition) feature values. In recent years, a method has been proposed in which a plurality of image feature amounts are combined to create a high-dimensional feature amount and a subject to be imaged is determined (see, for example, Patent Document 1). In these methods, when extracting a feature amount from an image, the feature amount is extracted using a certain number of dimensions for the entire screen or a specific recognition region in the screen.
Japanese Patent Laying-Open No. 2003-289551 (paragraphs “0089” to “0091”, FIG. 10)

  However, when the subject is photographed so that the subject is located in the center of the image, the background other than the subject often appears in the edge portion of the image, and noise included in the feature amount due to the influence of the background of the edge portion. Will increase. For this reason, the correct answer rate of the discrimination using the feature amount is lowered.

  SUMMARY OF THE INVENTION In view of the above problems, an object of the present invention is to provide an image processing apparatus and an image processing method that extract a feature amount with less noise and correctly determine a shooting target included in a shot image from a shot image. .

  In order to achieve the above object, a first feature of the present invention is an image processing apparatus that performs at least one of learning or discrimination of a shooting target included in a shot image from a shot image, and (a) an arbitrary number of In each area of the captured image divided into areas, a probability estimation unit that estimates the probability that the imaging target is captured, and (b) the image representation method in each area is changed based on the probability estimated by the probability estimation unit. (C) a feature amount extracting unit that extracts a feature vector using the expression method changed by the expression changing unit, and (d) a feature vector in each region extracted by the feature amount extracting unit. And (e) at least learning or discrimination of the captured image based on the feature vector of the entire captured image. And summarized in that an image processing apparatus comprising means for performing one.

  According to the image processing device according to the first feature, by changing the expression for each region, it is possible to extract a feature amount with less noise and correctly determine a shooting target included in the shot image from the shot image.

  The probability estimating means of the image processing apparatus according to the first feature estimates the probability by obtaining a statistic of each region where the shooting target appears in the shot image based on prior information on the size of the shooting target. May be.

  According to this image processing apparatus, the probability can be estimated according to the size of the subject to be photographed.

  In addition, the expression changing unit of the image processing apparatus according to the first feature may change the image expressing method using the frequency of the feature vector obtained from each region or the number of dimensions of the feature vector.

  According to this image processing apparatus, it is possible to change an image representation method using feature vector information.

  In addition, in the image processing apparatus according to the first feature, the photographing target to be learned or discriminated may be selected based on the position information of the terminal device that has performed the photographing and the position information of the photographing target.

  According to this image processing apparatus, it is possible to narrow down the shooting target that needs to be processed from the position information of the terminal device and the position information of the shooting target.

  A second feature of the present invention is an image processing method that performs at least one of learning or discrimination of a shooting target included in a shot image from a shot image, and (a) a shot image divided into an arbitrary number of regions (B) using the estimated probability, (b) changing the image representation method in each region based on the estimated probability, and (c) using the modified expression method. Extracting a feature vector; (d) creating a feature vector of the entire captured image by combining the extracted feature vectors in each region; and (e) based on the feature vector of the entire captured image. The gist of the present invention is an image processing method including a step of performing at least one of learning or discrimination of an imaging target.

  According to the image processing method according to the second feature, it is possible to extract a feature amount with less noise by changing the expression for each region, and to correctly determine the shooting target included in the shot image from the shot image.

  According to the present invention, it is possible to provide an image processing apparatus and an image processing method that extract feature amounts with less noise and correctly determine a shooting target included in a shot image from a shot image.

  Next, embodiments of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic.

(Image processing system)
As illustrated in FIG. 1, the image processing system according to the present embodiment receives a user input, requests processing from the image processing server 1 (image processing apparatus), and outputs a processing result in the image processing server 1. Receives processing requests from the terminal devices 2a, 2b, and 2c via the devices 2a, 2b, and 2c and the communication network (mobile communication network, LAN, Internet, etc.) 3, and performs processing. And an image processing server 1 for transmitting the processing result.

  The user can use the terminal devices 2a, 2b, and 2c by switching between two modes, a learning mode and a discrimination mode. The “learning mode” is a mode in which the image processing server 1 learns a photographed image by photographing a known photographing target and transmitting the photographed image and information specifying the photographing target to the image processing server 1. . On the other hand, in the “discrimination mode”, an unknown shooting target is shot, and this shot image is transmitted to the image processing server 1, whereby the image processing server 1 determines the shooting target, and the determination result is displayed as the terminal device 2a. 2b and 2c.

  When in the learning mode, the image processing server 1 performs grouping of shooting targets from the registration target information, the registration target related information, and the shot images obtained by shooting the shooting targets received from the terminal devices 2a, 2b, and 2c. Store target information and images. Here, “registration target information” is information for specifying a shooting target in a shot image. Further, “registration target related information” includes position information of a shooting target, information (name, URL, etc.) related to the shooting target, and the like.

  When it is in the discrimination mode, the image processing server 1 is registered in advance as a photographing target that is close to the position information of the terminal devices 2a, 2b, and 2c and the position where the captured images received from the terminal devices 2a, 2b, and 2c are photographed. The registration target information, registration target related information, and the like are extracted, and the shooting target included in the shot image is determined probabilistically. In addition, the image processing server 1 transmits to the terminal devices 2a, 2b, and 2c a plurality of candidates for shooting targets that are the determination results, the probability that each candidate is a shooting target, and information related to these shooting targets.

  As shown in FIG. 2, the image processing server 1 includes a communication unit 11, a determination unit 12, a feature amount extraction unit 13, a registration unit 14, a learning unit 15, a calculation device 16, a storage device 17, Weighting means 18, probability estimating means 30, expression changing means 31, and feature vector creating means 32 are provided.

  The communication unit 11 receives photographed images and information of photographing objects from the terminal devices 2a, 2b, and 2c via the communication network 3 (mobile communication network, LAN, Internet, etc.). When the communication unit 11 is in the discrimination mode, the communication unit 11 transmits information about the imaging target and the discrimination result to the terminal devices 2a, 2b, and 2c via the communication network 3 (mobile communication network, LAN, Internet, etc.).

  The probability estimating means 30 estimates the probability that a shooting target is captured in each area of the captured image divided into an arbitrary number of areas. The probability in each area will be described in detail later, and is, for example, a value as shown in FIG. 13 or FIG. When estimating such a probability, the probability estimating means 30 calculates, for example, the statistic of each region in which the shooting target appears in the shot image based on the prior information on the size of the shooting target as shown in FIG. Find the probability. Note that prior information regarding the size of the subject to be photographed may be registered in the storage device 17 in advance. Even if the prior information regarding the size of the object to be photographed is not registered in advance, the probability estimating means 30 estimates the probability using a method described in detail later as shown in FIGS. Can do.

  The expression changing unit 31 changes the image expressing method in each region based on the probability estimated by the probability estimating unit 30. In addition, the expression changing unit 31 uses, for example, the frequency of the feature vector obtained from each region or the dimension of the feature vector (or both the frequency of the feature vector and the dimension of the feature vector) to change the image expression method. change. This changing method will be described in detail later.

  The feature amount extraction unit 13 extracts a feature amount (feature vector) that serves as an index for determining a shooting target in each region of the captured image divided into an arbitrary number of regions. At this time, the feature quantity extraction unit 13 extracts a feature vector using the expression method changed by the expression change unit 31.

  The feature vector creating unit 32 creates a feature vector of the entire photographed image by combining the feature vectors in the respective regions extracted by the feature amount extracting unit 13.

The feature amount extraction unit 13 quantizes, for example, the signal of the captured image expressed in the uniform color space for each arbitrary region of the captured image, and calculates the frequency of the quantization level value of each axis in the uniform color space. Extract as a color histogram. Here, the “color space” includes uniform color space (L ^* , a ^* , b ^* space), RGB color space, CMYK color space, L ^* , u ^* , v ^* space, YUV space, XYZ space, etc. Various color spaces are included. Then, the feature amount extraction means 13 combines the color histograms for each arbitrary region, and extracts the feature amount of the entire captured image. That is, the feature amount extraction unit 13 extracts the feature amount (x) of the entire captured image having elements for the number of dimensions (V) for each known captured image. The feature quantity (x) is a vector having a certain number of elements. In the following description, x and x ′ are vectors. Further, if l is a number of a known photographing target, n is a number assigned to each photographed image, and v is a dimension number of a feature amount, the feature amount X of the photographed image belonging to a plurality of known photographing targets is represented by each element. It is represented by a matrix consisting of x ^(l) _{(n, v)} .

The learning unit 15 (parameter calculation unit) calculates the learning parameter Θ based on the feature quantity extracted by the feature quantity extraction unit 13 using the equation (1). The learning parameter Θ is a matrix in which V × L elements θ _{(v, l)} are arranged, where V is the maximum dimension of the feature quantity and L is the number of known shooting targets.

In Equation (1), l is the number of a known object, L is the number of known objects, N ^(l) is the number of images taken for the l-th known object, and v is the feature dimension number. , V is the maximum number of dimensions of the feature quantity, x ^(l) _{(n, v)} is the feature quantity of the vth dimension of the nth captured image belonging to the lth known imaging target, γ is an arbitrary value, θ _{(v, l)} is a learning parameter in the v-th dimension of the feature quantity belonging to the l-th known imaging target.

  In the above description, v, l, and n (lowercase) are variables, and V, L, and N (uppercase) are fixed values. X and x '(lower case) are vectors, and X and Θ (upper case) are matrices.

Specifically, the learning unit 15 (parameter calculation unit) calculates the v-th dimension learning parameter θ _{(v, l)} of the feature quantity belonging to the l-th known imaging target using the equation (1). To do.

The discriminating unit 12 discriminates an unknown imaging target by using the learning parameter calculated by the learning unit 15 (parameter calculating unit) using Expression (2) for the captured image received in the discrimination mode. .

In Expression (2), x ′ _(v) is the feature quantity of the v th dimension of the captured image including the unknown imaging target, and F _(l) is the degree to which the unknown imaging target belongs to the l th known imaging target. Is a discriminating value.

Specifically, the determination unit 12 belongs to a photographed image including an unknown photographing target using the learning parameter θ _{(v, l)} of the v th dimension of the feature amount belonging to the l th known photographing target. A value of F _(l) indicating the degree to which the feature quantity x ′ belongs to the l-th known imaging target is calculated. Here, it is determined that the larger the value of F _(l), the higher the probability that the unknown imaging target is the l-th known imaging target.

In this way, if F _(l) is calculated L times according to equation (2), according to the value of F _(l) , which of the L types of imaging targets the unknown imaging target belongs to is ranked. Can be determined.

  The registration unit 14 registers the feature amount and learning parameter for each captured image in the storage device 17 via the arithmetic device 16.

  The weighting means 18 multiplies the color histogram for each arbitrary region by an arbitrary weight. The feature amount extraction unit 13 combines the color histograms for each arbitrary region multiplied by the weight by the weighting unit 18. The arbitrary weight may be, for example, the reciprocal of the total value of the frequency values of the color histogram for each arbitrary region.

  The arithmetic device 16 controls the operations of the communication unit 11, the determination unit 12, the feature amount extraction unit 13, the registration unit 14, the learning unit 15, the storage device 17, and the weighting unit 18.

  The image processing server 1 according to the embodiment of the present invention includes a processing control device (CPU), and includes a communication unit 11, a determination unit 12, a feature amount extraction unit 13, a registration unit 14, a learning unit 15, and a weighting unit. 18, the probability estimating means 30, the expression changing means 31, the feature vector creating means 32, and the like may be built in the CPU as modules. These modules can be realized by executing a dedicated program for using a predetermined program language in a general-purpose computer such as a personal computer.

  In addition, the storage device 17 stores a plurality of predetermined learning parameters, photographed image data, registration target information, registration target related information, determination values, and photographing values calculated based on the feature amounts of the plurality of captured images. It is a recording medium for storing prior information related to the size of an object. Examples of the recording medium include RAM, ROM, hard disk, flexible disk, compact disk, IC chip, and cassette tape. According to such a recording medium, it is possible to easily store, transport, and sell captured image data, learning parameters, registration target information, and the like.

  The terminal devices 2a, 2b, and 2c can switch between two types of modes, a learning mode and a discrimination mode.

  In the learning mode, the terminal devices 2a, 2b, and 2c photograph a known subject to be photographed using the mounted camera. The terminal devices 2a, 2b, and 2c transmit the captured image to the image processing server 1 together with registration target information and registration target related information registered in advance by the user. Further, the image processing server 1 selects the shooting target for which the learning process is performed based on the position information of the terminal device 2 that has performed the shooting and the position information of the shooting target included in the registration target related information.

On the other hand, in the discrimination mode, the terminal devices 2a, 2b, and 2c take an image of an unknown shooting target with the mounted camera. The terminal devices 2a, 2b, and 2c transmit the captured image to the image processing server 1 together with the registration target related information. Further, the image processing server 1 selects a shooting target to be subjected to the discrimination process based on the position information of the shooting target terminal device 2 and the position information of the shooting target included in the registration target related information. Then, the terminal devices 2a, 2b, and 2c receive a plurality of shooting target candidates that are the determination results from the image processing server 1, the probability that each candidate is a shooting target, and information related to the shooting targets, and Based on the value of F _(l) , “registration target information” and “registration target related information” are ranked and presented to the user. The user can easily extract desired data from the ranked candidates.

  As shown in FIG. 3, the terminal device 2 includes an input unit 21, a communication unit 22, an output unit 23, a photographing unit 24, a positioning unit 25, a calculation device 26, and a storage device 27.

  The communication unit 22 transmits the photographed image and the photographing target information to the image processing server 1 via the communication network 3 (mobile communication network, LAN, Internet, etc.). When the communication unit 22 is in the determination mode, the communication unit 22 receives information about the imaging target and the determination result from the image processing server 1 via the communication network 3 (mobile communication network, LAN, Internet, etc.).

  Specifically, the imaging unit 24 refers to a mounted camera or the like, captures an object, and acquires a captured image.

  The positioning means 25 measures the position of the terminal device 2 and the position of the photographing target. The positioning means 25 may perform positioning by combining, for example, GPS, linkage with a base station, ultrasonic waves, an electronic compass, and the like.

  The input means 21 refers to devices such as a touch panel, a keyboard, a mouse, and a cellular phone button. When an input operation is performed from the input means 21, corresponding key information is transmitted to the arithmetic device 26. The output means 23 refers to a screen such as a monitor, and a liquid crystal display (LCD), a light emitting diode (LED) panel, an electroluminescence (EL) panel, or the like can be used.

  The arithmetic device 26 controls operations of the input means 21, the communication means 22, the output means 23, the photographing means 24, the positioning means 25, and the storage device 27. The arithmetic unit 26 operates as a switching unit that switches between the learning mode and the discrimination mode based on key information input from the input unit 21.

  The storage device 27 is a recording medium that stores captured images, registration target information, registration target related information, and the like.

(Image processing method)
Next, an image processing method according to the present embodiment will be described with reference to FIGS.

  First, a method for registering captured images will be described with reference to FIG.

  (A) First, in step S101, the terminal device 2 captures a known registration target in the learning mode, and acquires the captured image.

  (B) Next, in step S102, the terminal device 2 inputs registration target information, and in step S103, the terminal device 2 inputs registration target related information. The registration target information and registration target related information may be input in advance before shooting. For example, if the photographed image is a confectionery store as shown in FIG. 9, “cake shop” or the like is input as “registration target information”, and “AAA confectionery store” or AAA confectionery store as “registration target related information”. And the URL of the AAA pastry store.

  (C) Next, in step S104, the terminal device 2 acquires the position information, the positioning error, the shooting time, and, if possible, the distance and direction to the shooting target of the point where the registration target is shot, and the registration target related information. Remember to include.

  (D) Next, in step S <b> 105, the terminal device 2 transmits registration target information, registration target related information, and captured image data to the image processing server 1.

  (E) Next, in step S106, the image processing server 1 receives registration target information, registration target related information, and photographed image data. In step S107, the image processing server 1 extracts the feature amount of the registered image. This feature amount extraction method will be described in detail later.

  (F) Next, in step S108, the image processing server 1 stores the registration target information, the registration target related information, the registration target image, the feature amount, and the registration time in the image processing server 1 in the storage device 17.

  Next, a method for learning an imaging target will be described with reference to FIG.

  (A) First, in step S201, the image processing server 1 reads registration target information, registration target related information, captured image data, and feature amount from the storage device 17.

  (B) Next, in step S202, the image processing server 1 performs grouping of objects based on position information included in the registration object related information to narrow down the imaging objects. In the case where learning is performed as needed at the time of determination of an imaging target, which will be described later, position information of the terminal device 2 is received from the terminal device 2, and the imaging target within the search range is learned. Here, the “search range” refers to an area having a radius within (positioning error) + (distance to the object) with the discrimination target as the center. The position information used here can accept any position. For example, learning may be performed using a position where the determination is likely to be performed in advance, or learning may be performed as needed when performing determination using position information of the terminal.

  (C) Next, in step S203, the image processing server 1 learns the shooting target. Specifically, the learning parameter is calculated using the above-described equation (1).

  (D) Next, in step S204, the image processing server 1 stores the target learning result (learning parameter).

  Next, a method for determining an imaging target will be described with reference to FIG.

  (A) First, in step S301, the terminal device 2 captures an unknown capturing target in the discrimination mode and acquires a captured image. Next, in step S302, the terminal device 2 acquires registration target related information regarding an unknown imaging target.

  (B) Next, in step S <b> 303, the terminal device 2 transmits registration target related information and captured image data to the image processing server 1. Next, in step S304, the image processing server 1 receives registration target related information and captured image data. Next, in step S305, the image processing server 1 extracts a feature amount of the captured image. This feature amount extraction method will be described in detail later.

  (C) Next, in step S306, the image processing server 1 narrows down the discrimination target. Next, in step S307, the image processing server 1 determines whether learning of the search range is completed. If completed, the process proceeds to step S308. If not completed, the process proceeds to step S309.

  (D) In step S309, the image processing server 1 performs the learning described in step S203 in FIG. In step S308, the image processing server 1 probabilistically performs target determination using the feature amount of the captured image and the learning parameter. Specifically, a determination value is obtained using the above-described equation (2), and a captured image that is a discrimination candidate is obtained.

  (E) Next, in step S310, the image processing server 1 reads the registration target information and registration target related information related to the determination result candidate from the storage device 17, and stores the registration target information, registration target related information, and the determination result in the terminal device. 2 to send. In step S311, the terminal device 2 receives the registration target information, the registration target related information, and the determination result.

  (F) Next, in step S312, the terminal device 2 gives priority to the registration target information and the registration target related information based on the probabilistic discrimination result (for example, displays a high-probability candidate at the top of the screen). Present it to the user. Thereby, the user can further extract information related to the imaging target from the network based on the address such as the URL related to the unknown imaging target.

  According to the determination processing shown in FIG. 6, for example, when the user images a store A that is an unknown image capturing object by the terminal device 2 and transmits the captured image to the image processing server 1, the image processing server 1 stores the store A. The name of the store A, the URL, etc. can be transmitted to the terminal device 2. For this reason, the terminal device 2 can easily obtain information regarding the subject to be photographed from the photographed image.

  Next, details of the feature amount extraction method in step S107 in FIG. 4 and step S305 in FIG. 6 will be described with reference to FIG.

  (A) First, in step S401, the image processing server 1 acquires a captured image to be captured by a camera mounted on the terminal device 2. Here, the case where the image shown in FIG. 9 is acquired will be described as an example. The acquired image may be subjected to general image correction such as white balance by the function of the camera or the terminal device 2.

  (B) Next, in step S402, the image processing server 1 performs noise correction image correction processing on the acquired image. Note that the image correction process is not necessarily required, and may be performed according to the situation.

  (C) Next, in step S403, the image processing server 1 extracts feature amounts.

  Details of the feature amount extraction method in step S403 will be described with reference to FIG.

  (A) First, in step S501, the image processing server 1 divides the captured image into I × J in a grid pattern. As a method for dividing the region, FIG. 9 shows a region divided into a lattice shape, but it may be a radial shape or a circular shape, and the divided shape is not limited to this. Further, the area may not be equally divided.

  (B) Next, in step S502, the image processing server 1 estimates the probability that the object is captured in each area. In order to estimate the probability that an object will appear in each area of the image, the probability that the object will appear in each area of the image can be estimated based on the shooting conditions and the size data of the object. For example, when a shooting condition of “shooting the shooting target in the center of the shooting image and the entire area of the shooting image” is set and there is data of the size of the three shooting targets, the size of the three shooting targets is set. The frequency at which the object appears in the image is determined from the aspect ratio. By dividing the frequency at which the photographic subject appears in each area of the photographic image by the number of data of the size of the photographic object, the probability that the object appears in each area of the image is obtained.

  A specific calculation example is shown below. As shown in FIG. 10, it is assumed that sizes regarding three objects are given. Then, it is assumed that each object appears in the image as shown in FIG. 11 (here, in order to simplify the description, the case where the aspect ratio of the captured image is 1 will be described).

  Moreover, the frequency in each area | region of each object becomes as shown in FIG. Here, as shown in FIG. 13, a value obtained by dividing the frequency by the total number of objects can be regarded as the probability that the object appears in each region.

  The above describes an example in which the size of the target is known. For example, the probability of the range in which the target is captured can be obtained by the same calculation method based on the area captured in the image obtained by capturing the target. good.

Furthermore, when there is no data regarding the object, weighting may be performed using some kind of distribution. 14 to 16, using a standard two-dimensional normal distribution (even with other distributions such as a two-dimensional normal distribution) so that the probability of the area in the center of the screen is high when divided into 3 × 3 areas. It does not matter) how to determine the number of dimensions of each region and adjust the value. First, FIG. 14 shows the center area as 0 and the distance from the center area for each area. FIG. 15 shows the result of converting the values in each region using a predetermined function. Here, the cumulative distribution function value used is

  It is. Here, r was calculated with the distance matrix shown in FIG. 14, μ = 0, and σ = 1.

  Then, the normalized cumulative distribution function value shown in FIG. 16 is regarded as the probability that the object appears in each area of the image. FIG. 16 shows an example in which normalization is performed by setting the total value (entire screen) of each region as 1, but normalization may be performed by setting the central region of the screen as 1 as shown in FIG. Since there is a very high probability that an imaging target exists in the center of the screen, normalization shown in FIG. 13 is also useful.

  (C) Next, in step S503, weights for changing the number of dimensions and frequency are determined based on the probability. At this time, the probability estimated in step S502 may be used as it is as a weight. In step S504, it is determined whether or not the number of dimensions is to be changed. If so, the number of dimensions is changed based on the weight in step S505. If not, the number of dimensions is fixed in step S506.

  Here, a method for changing the number of dimensions will be described. When changing the number of dimensions, the number of dimensions of the feature vector extracted from each area of the image is changed by changing the number of quantization bits according to the probability that the object appears in each area of the image. In FIG. 17, the number of quantization bits is calculated based on the probability shown in FIG. 16, and the number of dimensions is calculated by multiplying the color space and the number of quantization bits. In accordance with the number of dimensions calculated in this manner, the number of dimensions of the color histogram is changed as shown in FIG. For example, as shown in FIG. 19, a case will be described in which the dimensionality of the color histogram is changed between the center portion and the periphery of the captured image, and between the center portion and the periphery. In FIG. 19, the number of dimensions is 9 dimensions at the center, 3 dimensions at the periphery, and 6 dimensions between the center and the periphery. Since there is a high probability that an object to be photographed exists in the center, the number of dimensions in the center is increased.

(D) Next, in step S507, the image processing server 1 represents the image in a uniform color space (L ^* , a ^* , b ^* ) that is a visually uniform space for each region. Here, when the captured image is divided into I × J in a grid pattern, the number of dimensions in the i, j-th region is V _{i, j} . For example, the number of dimensions in each divided region is shown as in FIG. Then, feature quantities (the feature quantities may be color histograms, frequencies, pixel values, etc.) are extracted in the V _{i, j} dimensions of each area in the i, j-th area. In the present embodiment, the discussion proceeds assuming that the color histogram is extracted. The color histogram of each region is represented as a V _{i, j-} dimensional feature quantity (vector value having V _{i, j} elements). In step S508, the image processing server 1 performs quantization based on the number of dimensions independently for each axis in the uniform color space. In step S509, the image processing server 1 calculates the frequency of the quantization level values of L ^* , a ^* , and b ^* .

  (E) Next, in step S510, the image processing server 1 determines whether or not to adjust the frequency, and if so, the process proceeds to step S511 to adjust the frequency based on the weight and perform the adjustment. If not, the process proceeds to step S512.

  In step S511, when the frequency is adjusted, the value of the feature vector extracted from each area of the image according to the number of dimensions of each area of the image is multiplied by the value of the probability that the object appears in each area of the image. For example, as shown in FIG. 20, the frequency of the color histogram is adjusted between the center portion and the periphery of the captured image and between the center portion and the periphery. In FIG. 20, the center part, the periphery, and between the center part and the periphery are all six-dimensional, but the frequency is increased or decreased in each region. Since there is a high probability that an object to be photographed exists in the center, the frequency increases as the distance from the center increases.

  Note that either one of the dimensionality change and the frequency adjustment may be performed, or both may be performed. When both are performed, as shown in FIG. 21, the number of dimensions changes for each region, and the frequency is adjusted for each region.

(F) Next, in step S513, the image processing server 1 combines the histograms of the respective regions of the image with the number of dimensions and the frequency adjusted, and extracts a feature vector of the entire photographed image. For example, when the captured image is divided into I × J in a grid pattern in step S501, the number of dimensions in the i, j-th region is V _{i, j} . The dimension V of the feature value obtained by combining the feature values of each region and combining the feature values of all the regions is

  It is represented by

  In the case of the result shown in FIG. 18, the entire histogram is 45 dimensions + 27 dimensions × 4 blocks + 15 dimensions × 4 blocks = 213 dimensions vector.

  (G) Next, in step S514, the entire histogram (213-dimensional vector) is normalized. The normalization process is not necessarily required, and may be performed according to the situation.

(Action and effect)
According to the image processing server 1 (image processing apparatus) and the image processing method according to the present embodiment, the shooting target is usually located in the center of the image, and the center of the image where the target is likely to be reflected is used. Image features that contain more information of interest and are less noisy by increasing the representation (number of dimensions and frequency) and reducing the surrounding representation (number of dimensions and frequency) of images that are less likely to show the background The amount can be extracted. For this reason, it is possible to correctly determine the shooting target included in the shot image from the shot image.

  Further, the probability estimating means 30 of the image processing server 1 can estimate the probability by obtaining the statistic of each region where the shooting target appears in the shot image based on the prior information regarding the size of the shooting target.

  The expression changing unit 31 of the image processing server 1 can change the image expression method using the frequency of feature vectors obtained from each region or the number of dimensions of feature vectors.

  Further, in the image processing server 1, the photographing target for which at least one of learning or discrimination can be performed can be selected based on the positional information of the terminal device 2 that has performed the photographing and the positional information of the photographing target. For this reason, it is possible to narrow down the shooting target that needs to be processed from the position information of the shooting target.

  Further, the above-described formulas (1) and (2) are so-called NB (Naive Bayes) techniques, and the probability that an unknown shooting target corresponds to a known shooting target is assigned to each of a plurality of known shooting targets. It has the advantage that it can be calculated and the processing speed is fast.

(Other embodiments)
Although the present invention has been described according to the above-described embodiments, it should not be understood that the descriptions and drawings constituting a part of this disclosure limit the present invention. From this disclosure, various alternative embodiments, examples, and operational techniques will be apparent to those skilled in the art.

  For example, in the above-described embodiment, an example has been shown in which learning parameters are acquired using an NB (Naive Bayes) method to determine an unknown imaging target. However, not only NB, but also other Bayesian methods and other algorithms such as SVM (Support Vector Machine), kNN (k Nearest Neighbor), LVQ (Learning Vector Quantization), etc. May be obtained to determine an unknown imaging target.

  The image processing server 1 according to the embodiment includes a communication unit 11, a determination unit 12, a feature amount extraction unit 13, a registration unit 14, a learning unit 15, a weighting unit 18, a probability estimation unit 30, Although it has been described that the expression changing unit 31 and the feature vector creating unit 32 can be built in the CPU as modules, they may be divided into two or more CPUs. In this case, it is assumed that the devices are connected by a bus or the like so that data can be exchanged between the CPUs.

  As described above, the present invention naturally includes various embodiments not described herein. Therefore, the technical scope of the present invention is defined only by the invention specifying matters according to the scope of claims reasonable from the above description.

1 is a configuration block diagram of an image processing system according to an embodiment of the present invention. It is a block diagram of the configuration of the image processing server according to the embodiment of the present invention. It is a block diagram of the configuration of a terminal device according to an embodiment of the present invention. It is a flowchart which shows the registration process which concerns on embodiment of this invention. It is a flowchart which shows the learning process which concerns on embodiment of this invention. It is a flowchart which shows the discrimination | determination process which concerns on embodiment of this invention. It is a flowchart which shows the feature-value extraction process which concerns on embodiment of this invention. It is a flowchart which shows the detail of step S403 of FIG. It is an example of the picked-up image which concerns on embodiment of this invention. It is an example of the magnitude | size of the imaging | photography object which concerns on embodiment of this invention. It is a figure which shows the range of the imaging | photography object which occupies for the captured image which concerns on embodiment of this invention. It is a figure which shows the frequency in each area | region of the imaging | photography object which concerns on embodiment of this invention. In FIG. 12, it is a figure which shows the probability normalized by the number of data. It is a figure which shows the distance matrix of each area | region of the picked-up image which concerns on embodiment of this invention. It is a figure which shows the cumulative distribution function value of each area | region of the picked-up image which concerns on embodiment of this invention. It is a figure which shows the normalized cumulative distribution function value of each area | region of the picked-up image which concerns on embodiment of this invention. It is a figure which shows the number of dimensions of each area | region of the picked-up image which concerns on embodiment of this invention. It is a figure which shows the dimension number of the feature vector in each area | region of the picked-up image which concerns on embodiment of this invention. It is a figure for demonstrating the dimension number change of the color histogram in each area | region of the picked-up image which concerns on embodiment of this invention. It is a figure for demonstrating the frequency adjustment of the color histogram in each area | region of the picked-up image which concerns on embodiment of this invention. It is a figure for demonstrating the dimensionality change and frequency adjustment of the color histogram in each area | region of the picked-up image which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing server 2 Terminal apparatus 3 Communication network 11 Communication means 12 Discriminating means 13 Feature-value extraction means 14 Registration means 15 Learning means 16 Arithmetic device 17 Storage device 18 Weighting means 21 Input means 22 Communication means 23 Output means 24 Imaging means 25 Positioning Means 26 Computing device 27 Storage device 30 Probability estimating means 31 Expression changing means 32 Feature vector creating means

Claims (5)

An image processing device that performs at least one of learning or discrimination of a shooting target included in a captured image from a captured image,
Probability estimation means for estimating the probability that the subject is captured in each region of the captured image divided into an arbitrary number of regions;
Based on the probability estimated by the probability estimating means, the expression changing means for changing the expression method of the image in each region;
Feature amount extraction means for extracting feature vectors using the expression method changed by the expression change means;
Feature vector creation means for creating a feature vector of the entire captured image by combining feature vectors in the respective regions extracted by the feature amount extraction means;
Means for performing at least one of learning or discrimination of an imaging target based on the feature vector of the entire captured image;
The expression changing means changes the image expression method using the dimensionality of the feature vector obtained from each region ,
The change of the expression method by the expression changing means is characterized in that the number of dimensions is changed so that the number increases in accordance with a high probability estimated by the probability estimating means in each region. Image processing device.   2. The probability estimating unit estimates a probability by obtaining a statistic of each region in which the imaging target appears in the captured image based on prior information on the size of the imaging target. An image processing apparatus according to 1.   The image processing apparatus according to claim 1, wherein the expression changing unit changes an image expression method using a frequency of a feature vector obtained from each region.   4. The apparatus according to claim 1, wherein the imaging target for performing at least one of learning and discrimination is selected based on position information of a terminal device that has performed imaging and position information of the imaging target. Image processing apparatus. An image processing method in an image processing apparatus that performs at least one of learning or discrimination of a shooting target included in a captured image from a captured image,
The image processing apparatus estimating a probability that the photographing object appears in each region of the photographed image divided into an arbitrary number of regions;
The image processing apparatus, based on the estimated probability, changing a representation method of the image in each region;
The image processing apparatus using the modified representation method to extract a feature vector;
The image processing device creating a feature vector of the entire captured image by combining the extracted feature vectors in each of the regions;
The image processing apparatus includes a step of performing at least one of learning or discrimination of an imaging target based on a feature vector of the entire captured image,
In the step of expressing, using the number of dimensions of the feature vector obtained from each region, the method of expressing the image is changed ,
In the image processing method, the number of dimensions is changed so that the number increases in accordance with a high probability estimated by the probability estimation unit in each region .

Images