JP6472504B1 - Information processing apparatus, information processing program, and information processing method - Google Patents

Abstract

A vehicle is detected from an image with higher accuracy. A learning device includes a learning image including a vehicle, information indicating positions of a plurality of feature points related to the vehicle included in the learning image, and a location of the vehicle indicated by each feature point. Learning is performed using learning data including information. The learned learning device 24 that has been learned outputs a heat map 50 that indicates the positions of a plurality of feature points 52 relating to the vehicle included in the input image 40 and a vector map 60 that indicates the direction between each feature point for each vehicle. The vehicle detection unit 32 detects a vehicle from the input image 40 when the feature point 52 is detected from the input image 40. Further, the direction and type of the vehicle are detected based on the positional relationship between the plurality of feature points 52 where the location of the vehicle is identified. When a plurality of vehicles are included in the input image 40, the vehicle detection unit 32 distinguishes the plurality of feature points 52 for each vehicle based on the vector map 60, and then determines the orientation, type, And the number of units is detected. [Selection] Figure 2

Description

  The present invention relates to an information processing apparatus, an information processing program, and an information processing method.

  Conventionally, a technique for automatically detecting a vehicle from an image has been proposed. In particular, research on machine learning techniques such as convolutional neural networks has been actively conducted recently, and a technique for automatically detecting a vehicle from an image using machine learning has been proposed.

  For example, in Patent Document 1, an image captured by a rear camera of the host vehicle is input to a discriminator for detecting the front of the vehicle, learned from the front image of the vehicle. Techniques for detecting other vehicles are disclosed.

Japanese Unexamined Patent Publication No. 2017-41132

  The conventional technology for automatically detecting a vehicle from an image has room for improvement from the viewpoint of detection accuracy. For example, in the technique of Patent Document 1, a vehicle can be detected only when a vehicle facing the front is included in the image, and the vehicle is suitably detected from an image obtained by photographing the vehicle from other angles. I couldn't. Further, when a part of the vehicle is not included in the image due to being hidden by another object or being out of view, it is difficult to detect the vehicle from the image. That is, a vehicle present in an image can be detected only in a limited scene.

  An object of the present invention is to detect a vehicle from an image with higher accuracy.

The present invention relates to learning data including a learning image including a vehicle, information indicating positions of a plurality of feature points related to the vehicle included in the learning image, and information indicating a location of the vehicle indicated by each feature point. A learning device that learns using the input device, and an input image is input to the learned learning device, and the vehicle included in the input image is detected based on a feature point related to the vehicle detected by the learning device from the input image. and, further, on the basis of the output of the learned of the learning device for the input image, and the vehicle detection unit for identifying the type of the vehicle is a taxi included in the input image, by analyzing the input image A person detection unit for detecting a person included in the input image;
A taxi demand determination unit that determines a taxi demand based on the number of taxis and the number of persons included in the input image .

  Preferably, the vehicle detection unit detects the orientation of the vehicle included in the input image based on the positional relationship of a plurality of feature points that are detected by the learning device from the input image and the corresponding vehicle location is identified. Is detected.

  Preferably, the learning device learns directions between the plurality of feature points of the vehicle included in the learning image, and the vehicle detection unit detects the plurality of feature points detected from the input image by the learning device. The plurality of vehicles included in the input image are distinguished based on the direction between them, and at least one of the direction and the number of the plurality of vehicles is detected.

  Preferably, the learning data includes information indicating the weather when the learning image is captured, and the learning device learns considering the weather when the learning image is captured. It is characterized by.

  Preferably, the learning data includes information indicating whether or not a vehicle light included in the learning image is turned on, and the learning device considers whether or not the vehicle light included in the learning image is turned on. It is characterized by learning.

  Preferably, the vehicle detection unit determines the type of the vehicle included in the input image based on the positional relationship of a plurality of feature points, which are detected from the input image by the learning device, and the location of the corresponding vehicle is identified. It is characterized by detecting.

Further, the present invention provides a computer with a learning image including a vehicle, information indicating the positions of a plurality of feature points related to the vehicle included in the learning image, and information indicating the location of the vehicle indicated by each feature point. A learning device that learns using learning data including: an input image is input to the learned learning device, and the learning image is input to the input image based on a feature point related to the vehicle detected from the input image. detecting a vehicle that contains, further, on the basis of the output of the learned of the learning device for the input image, and the vehicle detection unit for identifying the type of the vehicle included in the input image is a taxi, the input image By analyzing the above, a person detection unit that detects a person included in the input image, and a task for determining the demand for a taxi based on the number of taxis and the number of persons included in the input image. An information processing program for causing to function as an over demand determining unit.

Further, the present invention provides a learning image including a vehicle, information indicating the positions of a plurality of feature points related to the vehicle included in the learning image, and information indicating the location of the vehicle indicated by each feature point. A learning step for learning the learning device using the data for the operation; an input image is input to the learned learning device; and the input image is input to the input image based on a feature point regarding the vehicle detected by the learning device from the input image detecting a vehicle that contains, further, on the basis of the output of the learned of the learning device for the input image, and the vehicle detection step of identifying the type of the vehicle included in the input image is a taxi, the input image The demand for taxis is calculated based on the person detection step for detecting a person included in the input image and the number of taxis and the number of persons included in the input image. An information processing method which comprises the taxi demand determination step of constant, the.

  According to the present invention, a vehicle can be detected from an image with higher accuracy.

1 is a schematic configuration diagram of an information processing system according to an embodiment. 1 is a schematic configuration diagram of a server according to the present embodiment. It is a figure which shows the example of the image for learning. It is a figure which shows the example of an input image. It is a figure which shows the example of the heat map which a learning device outputs. It is a figure which shows the example of the vector map which a learning device outputs. It is a figure which shows the other example of an input image. It is a figure which shows the example of the color difference and similarity of each feature point in a 1st input image and a 2nd input image. It is a flowchart which shows the flow of a process of the server which concerns on this embodiment.

  Hereinafter, embodiments of the present invention will be described.

  FIG. 1 is a schematic configuration diagram of an information processing system 10 according to the present embodiment. The information processing system 10 includes a camera 12, a user terminal 14, and a server 16 as an information processing apparatus. The camera 12 and the server 16 and the user terminal 14 and the server 16 are communicably connected via a communication line 18 such as a LAN or the Internet.

  The camera 12 is a camera for photographing a vehicle such as an automobile. The camera 12 is, for example, a surveillance camera installed in a parking lot or a highway. Alternatively, the camera 12 may be an in-vehicle camera. Further, the camera 12 may be a camera that captures a still image or a video camera that captures a moving image. An image taken by the camera 12 is sent to the server 16 via the communication line 18.

  The user terminal 14 is a terminal used by a user (a person who confirms the result of the vehicle detection process from the image executed by the server 16). The user terminal 14 may be a personal computer or a tablet terminal, for example. The user terminal 14 includes a processing unit such as a CPU (Central Processing Unit), a hard disk, a storage unit such as a ROM (Read Only Memory) or a RAM (Random Access Memory), a display unit such as a liquid crystal panel, a mouse, a keyboard, and a touch panel. An input unit and a communication unit such as a network adapter are included. The user terminal 14 can receive the result of the vehicle detection process in the server 16 by accessing the server 16.

  FIG. 2 shows a schematic configuration diagram of the server 16. In the present embodiment, the server 16 is configured by a computer, but the information processing apparatus may be any apparatus as long as it has the following functions.

  The communication unit 20 includes a network adapter, for example. The communication unit 20 exhibits a function of communicating with the camera 12 and the user terminal 14 via the communication line 18. Specifically, the communication unit 20 receives an image from the camera 12. Further, the communication unit 20 transmits the result of the vehicle detection process by the control unit 30 described later to the user terminal 14 in response to a request from the user terminal 14.

  The storage unit 22 is composed of, for example, a ROM, a RAM, or a hard disk. The storage unit 22 stores an information processing program for operating each unit of the server 16. The storage unit 22 stores a learning device 24.

  As will be described in detail later, the learning device 24 uses an image captured by the camera 12 as an input image, and detects a plurality of feature points related to the vehicle included in the input image. The learning device 24 in the present embodiment includes, for example, a convolutional neural network.

  Note that the convolutional neural network generally includes a plurality of layers. Each layer of the convolutional neural network is composed of a group of neurons arranged (for example, a two-dimensional array). In the convolutional neural network, a filter including a plurality of elements arranged two-dimensionally is used like a neuron group. A filter is provided for each layer, and usually the size of the filter (the number of arranged elements) is smaller than the size of the group of neurons to which it is applied (the number of arranged neurons). A weight is set for each element of the filter.

  A filter is applied to each layer of neurons. Conceptually, a filter is superimposed on a part of a group of neurons. The product sum of each weight set for each element of the filter and the activity of a plurality of neurons (part of the neuron group) on which the filters overlap is calculated. That is, the inner product of a weight vector having the weight set for each element of the filter as a vector element and an activity vector having the activities of a plurality of neurons as vector elements is calculated. The inner product becomes one activity of the output of the layer.

  The filter is applied while shifting one by one (one neuron) or several neurons in the vertical and horizontal directions on the neuron group. Thereby, the arranged active group is output as the output of the layer. This active group becomes the activity of the neuron group in the next layer. By applying filters to a plurality of layers of neuron groups, a plurality of feature points related to the vehicle are finally detected from the input image.

  As described above, in the present embodiment, the actual state of the learning device 24 includes various parameters related to the convolutional neural network (layer structure, neuron structure of each layer, the number of filters in each layer, the filter size, the weight of each element of each filter, etc. ), And a processing execution program for processing the input image. Therefore, storing the learning device 24 in the storage unit 22 means storing the various parameters and the processing execution program in the storage unit 22. Note that the learning device 24 may include a plurality of neural networks.

  A learning method of the learning device 24 will be described. The learning device 24 performs learning using a learning image including a vehicle and learning data including information related thereto. Incidentally, learning by the learning device 24 means adjusting the weight of each element of the filter if the learning device 24 is a convolutional neural network.

  FIG. 3 shows an example of the learning image 26. The learning image 26 may be any image as long as it includes a vehicle, and the acquisition method may be any method. In the example of FIG. 3, the learning image 26 includes one vehicle, but the learning image 26 may include a plurality of vehicles.

  In the learning data, a plurality of feature points relating to the vehicle included in the learning image 26 (28a to 28k, hereinafter, a plurality of feature points relating to the vehicle included in the learning image 26 are collectively referred to as feature points 28). The information indicating the position of is included. In the present embodiment, the feature point 28 related to the vehicle is the intersection of the four corners (feature points 28a to 28d) of the vehicle roof, the line connecting the feature points 28a and 28d, and the line connecting the feature points 28b and 28c. Position of 13 feature points 28 of a certain feature point 28e, two headlights (feature points 28f and 28g), two tail lights (feature points 28h and 28i), and four tires (feature points 28j and 28k) Is known. In addition, as in the image illustrated in FIG. 3, all 13 feature points may not be included in the image. Further, the feature points 28 related to the vehicle are not limited to the above-described positions, and may include other positions (for example, positions of front and rear license plates). The positions of the plurality of feature points are set so that as many feature points as possible appear in the image regardless of the orientation of the vehicle in the image.

  The information indicating the positions of the feature points 28 is, for example, coordinates in the learning image 26.

  The learning data includes information indicating the location of the vehicle indicated by each feature point 28. The vehicle part is a specific part of the vehicle, such as the center of the roof or the right front wheel. In the example of FIG. 3, the feature point 28a is the roof right front corner, the feature point 28b is the roof left front corner, the feature point 28c is the roof right rear corner, the feature point 28d is the roof left rear corner, the feature point 28e is the roof center, and the feature point 28f. Is the right headlight, the feature point 28g is the left headlight, the feature point 28h is the right taillight, the feature point 28i is the left taillight, the feature point 28j is the left front wheel, and the information indicating that the feature point 28k is the left rear wheel Included in learning data. When a plurality of vehicles are included in the learning image 26, the learning data includes information indicating the location of each feature point 28 that is distinguished for each vehicle (for example, the feature point 28a is “ Vehicle A's right front corner of the roof, etc.).

  The learning data includes information indicating the direction (vector in the present embodiment) from each feature point 28 toward another feature point. As the vector connecting the feature points, a vector automatically generated by a predetermined program based on information indicating the position of each feature point 28 can be used.

  Preferably, the learning data may include information indicating the weather when the learning image 26 is taken. The weather is, for example, sunny, cloudy, rainy or snowy. Preferably, the learning data may include information indicating whether or not each light (including a headlight and a taillight) of the vehicle included in the learning image 26 is turned on.

  It should be noted that the position of the plurality of feature points 28 of the vehicle, the location of the vehicle indicated by each feature point 28, the weather when the learning image 26 is photographed, and each light of the vehicle included in the learning image 26 are described above. Each piece of information indicating the presence / absence of lighting may be added as metadata of the learning image 26, for example.

  The learning device 24 can detect a plurality of feature points related to the vehicle from the input image by learning using a large number of learning data. In particular, the learning device 24 learns using learning data in which the positions of a plurality of feature points 28 related to the vehicle and the location of the vehicle indicated by each feature point 28 are known. Thereby, the learning device 24 can identify and detect which part of the vehicle the detected feature point corresponds to.

  The fact that a plurality of feature points can be detected after identifying the corresponding vehicle location means that the positional relationship between the vehicle locations included in the input image can be detected. In other words, the learning device 24 learns the direction (vector in this embodiment) from a certain location (feature point) of the vehicle to another location (feature point) from the learning data, and has already learned the learning device 24. It can be said that the direction (vector) between each feature point detected from the input image can be detected. In addition, even when the learning image 24 includes a plurality of vehicles in the learning image 26, the learning data 24 is distinguished from each vehicle in the learning data, and the vehicle location indicated by each feature point 28. Is known, it is possible to learn the direction between each feature point by distinguishing each vehicle.

  Preferably, the learning device 24 learns to output the weather when the learning image 26 is captured in addition to the feature points and the directions between the feature points. For example, when the weather is snow, the four corners of the vehicle roof and the center of the roof may be hidden by snow accumulation. Even in such a case, by learning to output the weather of the input image using the learning image 26 including the information indicating the weather, the learning device 24 can determine the weather when the learning image 26 is captured. It is considered that feature points will be detected in consideration of snow. Thereby, the learning accuracy can be improved when the learning device 24 detects a plurality of feature points related to the vehicle from the input image.

  Similarly, the learning device 24 preferably learns to output whether or not each light of the vehicle included in the learning image 26 is turned on in addition to the feature points and the directions between the feature points. When an image taken in the daytime and an image taken at night are compared, the appearance of the image is greatly different, for example, the luminance value near the light position of the input image changes depending on whether or not the light is turned on. Even in such a case, the learning device 24 learns to output whether or not each light of the vehicle included in the input image is output using the learning image 26 including information indicating whether or not each light of the vehicle is turned on. It is considered that the feature points are detected in consideration of whether or not each light of the vehicle is turned on when the learning image 26 is taken. Thereby, the learning accuracy can be improved when the learning device 24 detects a plurality of feature points related to the vehicle from the input image.

  Returning to FIG. 2, the control unit 30 includes, for example, a CPU or a microcontroller. The control unit 30 controls each unit of the server 16 according to the information processing program stored in the storage unit 22. 2, the control unit 30 also functions as a vehicle detection unit 32, a person detection unit 34, a processing calculation unit 36, and an identity determination unit 38.

  The vehicle detection unit 32 executes processing for detecting a vehicle from an input image that is an image captured by the camera 12. Specifically, the vehicle detection unit 32 inputs the input image to the learned learning device 24, and detects the vehicle from the input image based on the detection results of a plurality of feature points by the learning device 24 for the input image. .

  FIG. 4 shows an example of the input image. The details of the output of the learning device 24 and the vehicle detection process by the vehicle detection unit 32 for the input image 40 shown in FIG. 4 will be described below. As shown in FIG. 4, in the present embodiment, the input image 40 includes a plurality of vehicles 42. Note that the input image to be processed by the vehicle detection unit 32 may include only one vehicle 42 or an image that does not include the vehicle 42.

  When the vehicle detection unit 32 inputs the input image 40 to the sufficiently learned learning device 24, the learning device 24 outputs a heat map and a vector map.

  FIG. 5 shows a heat map 50 for the input image 40. As described above, the learning device 24 detects a plurality of feature points related to each vehicle from the input image 40. The heat map 50 is a map indicating the positions of the detected feature points 52. The detected positions of the plurality of feature points 52 are represented by coordinates in the input image 40.

  FIG. 6 shows a vector map 60 for the input image 40. As described above, the learning device 24 detects the direction between the feature points 52 detected from the input image. In particular, when a plurality of vehicles are included in the input image, the learning device 24 detects the direction between the feature points 52 by distinguishing the vehicles for each vehicle. The vector map 60 is a map indicating the position and orientation of a vector 62 that indicates the orientation between the feature points 52 for each vehicle. Note that feature points 52 are shown in the vector map 60 shown in FIG. 6, but the vector map 60 may not include the feature points 52.

  Each vector 62 is shown between one feature point 52 and another feature point 52, but its direction (whether it is a direction from one feature point 52 to another feature point 52 or another feature point). The direction from the point 52 toward one feature point 52) may be defined in advance. For example, in the present embodiment, each vector 62 is in principle a front-to-back direction (for example, a direction from the roof right front corner to the roof right rear corner) and a right to left (for example, a direction from the roof right front corner to the roof left front corner). ) Direction.

  In the vector map 60, only the vector 62 between the adjacent (nearby) feature points 52 may be shown. However, from the feature point 52, all other points corresponding to the same vehicle as the feature point 52 are displayed. A vector 62 may be shown between the feature point 52 and the feature point 52. In the vector map 60 shown in FIG. 6, only a part of the vectors 62 is shown.

  The vehicle detection unit 32 detects a vehicle from the input image 40 based on the feature points 52 detected in the heat map 50 and the vector map 60. Details of the detection process will be described below.

  The vehicle detection unit 32 performs a filtering process on the heat map 50 and extracts feature points having a probability equal to or greater than a predetermined value from the feature point group included in the heat map 50 as a plurality of effective feature points. Next, the vector map 60 is integrated on a line segment connecting the feature points with respect to all combinations between the extracted effective feature points. Then, a set of two feature points whose average of the integral values of the vectors is a predetermined value or more is specified as a correct combination. In this way, a plurality of effective feature points are grouped into a plurality of groups. Then, for example, group A having three feature points, group B having five feature points, group C having six feature points, or a single feature point not combined with other feature points, etc. Is done. Then, a group including a predetermined number or more of feature points is specified as a group representing the vehicle. That is, it detects as a vehicle.

  Thus, according to the present embodiment, since the vehicle is detected based on the feature point 52 detected from the input image 40, a part of the vehicle is hidden or overlooked in the input image 40, or the vehicle The vehicle can be detected from the input image 40 regardless of the angle. On the other hand, when the feature point 52 is not detected in the heat map 50, the vehicle detection unit 32 can determine that the vehicle is not included in the input image 40.

  Further, as described above, the learning device 24 can detect a plurality of feature points 52 in which corresponding vehicle locations are identified. For example, in the learning device 24, the feature point 52a is the roof right front corner, the feature point 52b is the roof left front corner, the feature point 52c is the roof right rear corner, the feature point 52d is the roof left rear corner, and the feature point 52e is the roof center. Each feature point 52 is detected after identifying that 52f is the right headlight, the feature point 52g is the left headlight, the feature point 52h is the right front wheel, and the feature point 52i is the right rear wheel.

  The vehicle detection unit 32 can detect the orientation of the vehicle included in the input image 40 based on the positional relationship between the plurality of feature points 52 where the location of the vehicle is identified. For example, in the example of the heat map 50 shown in FIG. 5, the feature point 52c corresponding to the roof right rear corner is located on the upper left side of the feature point 52a corresponding to the roof right front corner, and the roof right front corner is located. A feature point 52b corresponding to the left front corner of the roof is located on the upper right side of the corresponding feature point 52a. From such a positional relationship, the vehicle detection unit 32 can detect that the vehicle represented by the plurality of feature points 52a to 52i faces the front side in the lower right direction in the input image 40.

  Moreover, the vehicle detection part 32 can detect the classification of a vehicle based on the positional relationship of the some feature point 52 by which the location of the vehicle was identified. The type of vehicle is distinguished by the difference in its outer shape, for example, a type such as a sedan, a van, a light vehicle, a truck, or a taxi.

  For example, a van has a longer distance from a tire position to a roof position than a sedan. In addition, the distance between the tire position and the roof position of the truck is relatively longer than that of the sedan, and the length of the roof in the front-rear direction is longer than that of the sedan or van. Since each type of vehicle has such a difference in appearance, the vehicle detection unit 32 can detect the type of vehicle included in the input image 40 based on the positions of the plurality of feature points 52. Since the distance between the plurality of feature points 52 may vary depending on the vehicle angle in the input image 40, the angle of the vehicle is estimated based on, for example, the positional relationship between the feature points at the four corners of the vehicle roof. It is preferable to detect the type of vehicle.

  As shown in FIG. 4, when the input image 40 includes a plurality of vehicles 42, particularly when the plurality of vehicles 42 are close to each other or overlap, a plurality of feature points 52 corresponding to the plurality of vehicles are displayed in the heat map 50. It will be detected densely. For example, the feature point group 64 located in the lower part of the heat map 50 in FIG. 5 is an example. In this case, although the location of the vehicle to which each feature point 52 corresponds is identified, it is difficult to appropriately distinguish the feature point group 64 into groups corresponding to each vehicle only from the heat map 50. There is a case.

  In the present embodiment, by using the vector map 60, even if the feature point group 64 in which the feature points 52 corresponding to a plurality of vehicles are densely formed, the feature point group 64 is assigned to each vehicle group. Can be properly distinguished. Specifically, each vector 62 included in the vector map 60 is shown between a feature point 52 of a certain vehicle and another feature point 52 of the vehicle. Therefore, based on the direction between the feature points 52 included in the feature point group 64, the feature point group 64 can be distinguished into a feature point group 64a and a feature point group 64b corresponding to each vehicle.

  By appropriately distinguishing the plurality of feature points 52 detected from the input image 40 for each vehicle, the vehicle detection unit 32 can detect the orientation of each vehicle included in the input image 40. At the same time, the vehicle detection unit 32 can detect the number of vehicles included in the input image 40.

  Returning to FIG. 2, the person detection unit 34 detects a person included in the input image 40. As a method for detecting a person from the input image 40, another learning device having the same configuration as the learning device 24 for detecting a vehicle and learning to detect a person may be used. Alternatively, a known image analysis process or the like may be used. In particular, the person detection unit 34 detects the number of persons included in the input image 40.

  The processing calculation unit 36 performs calculation based on the vehicle detection result from the input image by the vehicle detection unit 32 and the person detection result from the input image by the person detection unit 34. In the present embodiment, the vehicle detection unit 32 detects the number of taxis 70 and the person detection unit 34 detects a person from an input image 40-2 obtained by bird's-eye view of the taxi stand as shown in FIG. The number 72 is detected. And the process calculating part 36 calculates based on the number of the taxis 70 in the input image 40-2, and the number of the persons 72, and determines the demand of a taxi. In this way, the processing calculation unit 36 functions as a taxi demand determination unit.

  As a taxi detection method by the vehicle detection unit 32, various methods can be employed. First, after the heat map 50 (see FIG. 5) is output by the learning device 24, image analysis is performed on a certain region including the feature point 52 corresponding to the center of the roof to detect a row lamp. Good. Of course, if a row lamp is detected, it can be determined that the vehicle is a taxi, and if a row lamp is not detected, it can be determined that the vehicle is not a taxi.

  Further, information indicating that the vehicle included in the learning image 26 (see FIG. 3) is a taxi is added to the learning data, so that the learning device 24 can identify the taxi from the input image 40-2. You may learn.

  Alternatively, all the vehicles included in the input image 40-2 may be regarded as taxis depending on the shooting target location of the camera 12.

  The processing calculation unit 36 detects the taxi 70 and the persons 72 included in the input image 40-2 by the above-described method. When the number of persons 72 is larger than the number of taxis 70, the processing calculation unit 36 determines that there is a taxi demand. For example, information indicating that there is demand for a taxi at the taxi stand is transmitted to the user terminal 14 used by a taxi dispatcher. Further, the processing calculation unit 36 calculates a difference between the number of taxis 70 and the number of persons 72 in the input image 40-2, and transmits information indicating the difference (that is, the number of necessary taxis) to the user terminal 14. You may do it.

  Further, the processing calculation unit 36 calculates the future demand for taxis at the taxi stand based on the statistical data of the number of taxis 70 and the number of persons 72 included in the plurality of past input images 40-2 at the taxi stand. You may make it predict.

  Returning to FIG. 2, the identity determination unit 38 performs a process of determining the identity of a vehicle (that is, identifying the same vehicle) in a plurality of input images taken at different timings by the same camera 12. According to the identity determination unit 38, it is possible to follow the same vehicle within a shooting range of a certain camera 12, and it is possible to detect how the vehicle moves. In the following description, the identity of the vehicle A and the vehicle B is determined between the first input image including the vehicle A and the second input image including the vehicle B, which are taken with the same camera 12 at different timings. Processing to be performed will be described.

  First, the identity determination unit 38 includes color information of each pixel of a plurality of feature points related to the vehicle A detected by the learning device 24 from the first input image and a plurality of information related to the vehicle B detected by the learning device 24 from the second input image. The color information of each pixel of the feature point is acquired. The color information is a concept including hue, brightness (luminance), and saturation.

  In the present embodiment, the learning device 24 detects each feature point as the coordinates of one pixel of the first input image or the second input image, but the identity determination unit 38 is detected by the learning device 24. A region having a certain area centered on one pixel is defined as a region for acquiring color information (hereinafter referred to as a pixel region). Further, the area of the pixel region may be changed according to the size of the vehicle in the first input image or the second input image. For example, the area of the pixel region may be increased as the vehicle in the input image is larger. Note that the size of the vehicle in the input image can be detected based on the positions of a plurality of feature points where the location of the vehicle is identified.

  In the present embodiment, the identity determination unit 38 expresses the color of a plurality of pixels included in each pixel region in an L * a * b * color space, and then represents a representative value (for example, an average value) of the colors of the plurality of pixels. , Median, mode). Then, the obtained representative value of the color is used as the color information of the pixel area. The color of each pixel is expressed in the L * a * b * color space because the L * a * b * color space is designed to approximate human vision.

  In addition, you may employ | adopt the feature-value other than the above as color information of each pixel area. For example, HOG (Histograms of Oriented Gradients) feature values in a plurality of pixels included in the pixel region may be calculated. Here, the HOG feature value is a feature value in which the luminance gradient direction of each pixel in the pixel region is histogrammed.

  As described above, the pixel color information corresponding to each feature point of the vehicle A included in the first input image and the color information of the pixel corresponding to each feature point of the vehicle B included in the second input image are obtained. To be acquired.

  Next, the identity determination unit 38 compares the color information of each feature point of the vehicle A with the color information of each feature point of the vehicle B. Specifically, the color information of the feature points of the vehicle A and the color information of the feature points of the vehicle B are compared for each vehicle location. Thereby, the identity determination unit 38 determines the identity between the vehicle A and the vehicle B. Specifically, a difference (color difference) in color information between a certain feature point of the vehicle A (here, the roof right front corner) and a feature point of the roof right front corner which is a corresponding portion of the vehicle B is calculated. In this way, color differences are calculated for all locations.

  In this embodiment, the color difference of the color information of the pixel at each feature point is calculated by the CIE2000 color difference formula. This is because the CIE2000 color difference formula is a calculation formula that takes into account the hue dependency, lightness dependency, and saturation dependency, which are the characteristics of the color discrimination range of the human eye. That is, the color difference based on the CIE2000 color difference formula is a value close to the color difference by human vision.

ここで、ｄは色差であり、α及びβは定数である。α及びβを適宜設定することにより、色差の値に対する類似度の値を調整することができる。 FIG. 8 shows the difference (color difference) between the color information of each feature point of the vehicle A and the color information of each feature point of the vehicle B. The identity determination unit 38 calculates the similarity for each feature point based on the calculated color difference. Specifically, the identity determination unit 38 calculates such that the degree of similarity increases as the color difference decreases. In the present embodiment, the similarity is calculated from the color difference using the following equation.

Here, d is a color difference, and α and β are constants. By appropriately setting α and β, the similarity value with respect to the color difference value can be adjusted.

  FIG. 8 shows the similarity calculated for each feature point. In this way, a plurality of similarities are calculated between a plurality of (13 in this embodiment) feature points between the vehicle A and the vehicle B. The identity determination unit 38 obtains representative values (for example, an average value, a median value, and a mode value) of the calculated plurality of similarities. The representative value of the similarity obtained in this way is the similarity between the vehicle A and the vehicle B. In addition, when there is a feature point that has not been detected among a plurality of feature points of the vehicle A or the vehicle B due to overrun or hiding in the first input image or the second input image, the similarity degree regarding the feature point is It is set to a predetermined value (50% in this embodiment).

  As described above, the similarity between the vehicle A included in the first input image and the vehicle B included in the second input image is calculated. The identity determination unit 38 determines that the vehicle A and the vehicle B are the same vehicle when the calculated similarity is greater than or equal to a predetermined similarity threshold, and the calculated similarity is less than the similarity threshold. In some cases, it is determined that the vehicle A and the vehicle B are not the same vehicle.

  As described above, the identity determination unit 38 determines the identity of a vehicle among a plurality of input images taken at different timings. Preferably, the same vehicle can be tracked in the moving image by performing the above processing for each adjacent frame image included in the moving image captured by the camera 12.

  Hereinafter, the processing flow of the server 16 according to the present embodiment will be described with reference to the flowchart shown in FIG.

  In step S10, the control unit 30 of the server 16 includes the learning image 26 including the vehicle, information indicating the positions of the plurality of feature points 28 related to the vehicle included in the learning image 26, and the vehicle indicated by each feature point 28. The learning device 24 is made to learn by repeating inputting learning data including information indicating the location to the learning device 24. Step S10 corresponds to a learning step.

  In step S <b> 12, the vehicle detection unit 32 inputs the input image 40 captured by the camera 12 to the learned learning device 24.

  In step S14, the learning device 24 outputs a heat map 50 and a vector map 60 for the input image 40 input in step S12.

  In step S16, the vehicle detection unit 32 determines whether or not a vehicle is included in the input image 40 based on the heat map 50 output in step S14. Specifically, when the learning device 24 detects a feature point 52 related to a vehicle from the input image 40, the vehicle detection unit 32 determines that the vehicle is included in the input image 40. If it is determined that the input image 40 includes a vehicle, the process proceeds to step S18. If it is determined that the input image 40 does not include a vehicle, the process ends. Absent. Steps S12 to S16 correspond to a vehicle detection step.

  In step S <b> 18, the vehicle detection unit 32 detects the direction and type of the vehicle based on the positional relationship between the plurality of feature points 52 where the location of the vehicle is identified. When the input image 40 includes a plurality of vehicles, the vehicle detection unit 32 distinguishes the plurality of feature points 52 for each vehicle using the vector map output in step S14, and then determines the orientations of the plurality of vehicles. , Type or number is detected.

  As mentioned above, although embodiment which concerns on this invention was described, this invention is not limited to the said embodiment, A various change is possible unless it deviates from the meaning of this invention.

  For example, in this embodiment, the server 16 as the information processing apparatus executes both the learning process of the learning device 24 and the vehicle detection process using the learning device 24. However, the learning process and the vehicle detection process are separated. It may be executed by the main body (apparatus). For example, the learned learning device 24 learned in another device may be stored in the storage unit 22 of the server 16, and the server 16 may execute the vehicle detection process using the learning device 24.

  Further, the present invention can be applied to vehicles such as ships and airplanes in addition to vehicles.

  DESCRIPTION OF SYMBOLS 10 Information processing system, 12 Cameras, 14 User terminal, 16 Server, 18 Communication line, 20 Communication part, 22 Storage part, 24 Learning device, 30 Control part, 32 Vehicle detection part, 34 Person detection part, 36 Process calculation part, 38 Identity determining unit.

Claims (8)

Learning using learning data including a learning image including a vehicle, information indicating positions of a plurality of feature points related to the vehicle included in the learning image, and information indicating a location of the vehicle indicated by each feature point Learner to
An input image is input to the learned learner, a vehicle included in the input image is detected based on a feature point related to the vehicle detected by the learner from the input image, and learning is performed on the input image. a vehicle detection unit for identifying the already on the basis of the output of the learner, the type of vehicle that is included in the input image is a taxi,
Analyzing the input image to detect a person included in the input image;
Based on the number of taxis and the number of persons included in the input image, a taxi demand determination unit that determines the demand for taxis,
An information processing apparatus comprising: The vehicle detection unit detects a direction of a vehicle included in the input image based on a positional relationship of a plurality of feature points, which are detected from the input image by the learning device, and corresponding portions of the vehicle are identified. ,
The information processing apparatus according to claim 1. The learning device learns a direction between the plurality of feature points of the vehicle included in the learning image,
The vehicle detection unit distinguishes the plurality of vehicles included in the input image based on the direction between the plurality of feature points detected by the learning device from the input image, and then determines the direction and number of the plurality of vehicles. Detect at least one,
The information processing apparatus according to claim 1. The learning data includes information indicating the weather when the learning image is taken,
The learning device learns in consideration of the weather when the learning image is taken,
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. The learning data includes information indicating whether or not a vehicle light included in the learning image is turned on,
The learning device learns in consideration of whether or not a vehicle light included in the learning image is turned on,
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. The vehicle detection unit detects a type of a vehicle included in the input image based on a positional relationship between a plurality of feature points, which are detected from the input image by the learning device, and a corresponding vehicle location is identified.
The information processing apparatus according to claim 1. Computer
Learning using learning data including a learning image including a vehicle, information indicating positions of a plurality of feature points related to the vehicle included in the learning image, and information indicating a location of the vehicle indicated by each feature point Learner to
An input image is input to the learned learner, a vehicle included in the input image is detected based on a feature point related to the vehicle detected by the learner from the input image, and learning is performed on the input image. a vehicle detection unit for identifying the already on the basis of the output of the learner, the type of the vehicle included in the input image is a taxi,
Analyzing the input image to detect a person included in the input image;
Based on the number of taxis and the number of persons included in the input image, a taxi demand determination unit that determines the demand for taxis,
An information processing program that functions as a computer program. Learning using learning data including a learning image including a vehicle, information indicating positions of a plurality of feature points related to the vehicle included in the learning image, and information indicating a location of the vehicle indicated by each feature point Learning step to learn the vessel,
An input image is input to the learned learner, a vehicle included in the input image is detected based on a feature point related to the vehicle detected by the learner from the input image, and learning is performed on the input image. a vehicle detection step of identifying that already on the basis of the output of the learner, the type of the vehicle included in the input image is a taxi,
Analyzing the input image to detect a person included in the input image; and
Taxi demand determination step for determining taxi demand based on the number of taxis and the number of persons included in the input image;
An information processing method comprising:

Images