JP2019106193A - Information processing device, information processing program and information processing method - Google Patents

Abstract

To detect a vehicle from an image with improved accuracy.SOLUTION: A learning device 24 leans using learning data comprising: a learning image 26 including a vehicle; information showing positions of a plurality of characteristic points 28 with respect to the vehicle included in the learning image 26; and information showing portions of the vehicle expressed by respective characteristic points 28. After learning, the leaning device 24 outputs a heat map 50 showing the positions of the plurality of characteristic points 52 with respect to the vehicle included in an input image 40 and a vector map 60 showing directions of respective characteristic points of each vehicle. A vehicle detection section 32 detects: the vehicle from the input image 40 when the characteristic points 52 are detected in the input image 40; and a direction and a type of the vehicle on the basis of a positional relationship of the plurality of characteristic points 52 which identify the portions of the vehicle. When a plurality of vehicles are shown in the input image 40, the vehicle detection section 32 classifies the plurality of characteristic points 52 into respective vehicles on the basis of the vector map 60 and detects the directions, the types and the number of vehicles.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an information processing apparatus, an information processing program, and an information processing method.

  Conventionally, techniques for automatically detecting a vehicle from images have been proposed. In particular, research on machine learning methods such as convolutional neural networks, for example, has been promoted recently, and techniques for automatically detecting a vehicle from an image using machine learning have been proposed.

  For example, in Patent Document 1, an image captured by the rear camera of the vehicle is input to a classifier for detecting the front of the vehicle, which is learned using the front image of the vehicle, from the image Techniques for detecting other vehicles are disclosed.

JP, 2017-41132, A

  The conventional technology for automatically detecting a vehicle from an image has room for improvement from the viewpoint of detection accuracy. For example, in the technique of Patent Document 1, a vehicle can be detected only when a vehicle facing the front is included in the image, and the vehicle is suitably detected from an image obtained by photographing the vehicle from other angles. I could not. In addition, when a part of the vehicle is not included in the image, it is difficult to detect the vehicle from the image because the vehicle is hidden or missed. That is, vehicles present in the image could be detected only in limited situations.

  An object of the present invention is to detect a vehicle from an image with higher accuracy.

  The present invention is a learning data including a learning image including a vehicle, information indicating the positions of a plurality of feature points related to the vehicle included in the learning image, and information indicating the location of the vehicle indicated by each feature point And an input image is input to the learning device which has already been learned, and the vehicle detects the vehicle included in the input image based on the feature points related to the vehicle detected from the input image by the learning device And a vehicle detection unit.

  Preferably, the vehicle detection unit detects the orientation of the vehicle included in the input image based on the positional relationship of the plurality of feature points detected by the learning device from the input image and in which the location of the corresponding vehicle is identified. To detect.

  Preferably, the learning device learns the direction between the plurality of feature points of the vehicle included in the learning image, and the vehicle detection unit detects a plurality of feature points detected from the input image by the learning device. After distinguishing the plurality of vehicles included in the input image based on the direction between the two, at least one of the direction and the number of the plurality of vehicles is detected.

  Preferably, the learning data includes information indicating a weather when the learning image is captured, and the learning device performs learning in consideration of the weather when the learning image is captured. It is characterized by

  Preferably, the learning data includes information indicating whether the light of the vehicle included in the image for learning is lit, and the learning device takes into consideration whether the light of the vehicle included in the image for learning is lit It is characterized by learning.

  Preferably, the vehicle detection unit determines the type of the vehicle included in the input image based on the positional relationship of the plurality of feature points detected by the learning device from the input image and in which the location of the corresponding vehicle is identified. It is characterized by detecting.

  Preferably, the vehicle detection unit identifies that the type of the vehicle included in the input image is a taxi based on the output of the learning device with respect to the input image, and the information processing apparatus further performs the input A taxi demand determination unit that determines a taxi demand based on the number of taxis and the number of persons included in the input image by analyzing an image to detect a person included in the input image And.

  Furthermore, the present invention provides a computer with a learning image including a vehicle, information indicating the positions of a plurality of feature points related to the vehicle included in the learning image, and information indicating the location of the vehicle indicated by each feature point The input image is input to the learning device for learning using the learning data including the above, and the learning device which has been learned, and the learning device performs the input image based on the feature points regarding the vehicle detected from the input image. It is an information processing program characterized by making it function as a vehicles detection part which detects vehicles included.

  Further, according to the present invention, a learning image including a learning image including a vehicle, information indicating positions of a plurality of feature points related to the vehicle included in the learning image, and information indicating a location of the vehicle indicated by each feature point Step of learning a learning device using data for learning, and inputting an input image to the learning device which has already been learned, and the computer inputs the image based on the feature points of the vehicle detected from the input image by the learning device. And a vehicle detection step of detecting a vehicle included in the input image.

  According to the present invention, a vehicle can be detected from an image with higher accuracy.

FIG. 1 is a schematic view of an information processing system according to an embodiment of the present invention. It is a structure schematic of a server concerning this embodiment. It is a figure which shows the example of the image for learning. It is a figure which shows the example of an input image. It is a figure which shows the example of the heat map which a learning device outputs. It is a figure which shows the example of the vector map which a learning device outputs. It is a figure which shows the other example of an input image. It is a figure which shows the example of the color difference and similarity of each feature point in a 1st input image and a 2nd input image. It is a flowchart which shows the flow of a process of the server which concerns on this embodiment.

  Hereinafter, embodiments of the present invention will be described.

FIG. 1 shows a schematic configuration of an information processing system 10 according to the present embodiment. The information processing system 10 includes a camera 12, a user terminal 14, and a server 16 as an information processing apparatus. The camera 12 and the server 16 and the user terminal 14 and the server 16 are communicably connected via a communication line 18 such as a LAN or the Internet.

  The camera 12 is a camera for photographing a vehicle such as a car. The camera 12 is, for example, a surveillance camera installed in a parking lot or an expressway. Alternatively, the camera 12 may be an on-vehicle camera. The camera 12 may be a camera that captures a still image, or may be a video camera that captures a moving image. The image taken by the camera 12 is sent to the server 16 via the communication line 18.

The user terminal 14 is a terminal used by a user (a person who confirms the result of the vehicle detection process from the image executed by the server 16). The user terminal 14 may be, for example, a personal computer or a tablet terminal. The user terminal 14 includes a processing unit such as a central processing unit (CPU), a hard disk, a storage unit such as a read only memory (ROM) or a random access memory (RAM), a display unit such as a liquid crystal panel, a mouse, a keyboard, and a touch panel. It is configured to include an input unit, a communication unit such as a network adapter, and the like. The user terminal 14 can receive the result of the vehicle detection process in the server 16 by accessing the server 16.

  A schematic diagram of the server 16 is shown in FIG. In the present embodiment, the server 16 is configured by a computer, but any device may be used as the information processing device as long as it has the following functions.

  The communication unit 20 includes, for example, a network adapter. The communication unit 20 performs a function of communicating with the camera 12 and the user terminal 14 via the communication line 18. Specifically, the communication unit 20 receives an image from the camera 12. Further, the communication unit 20 transmits the result of the vehicle detection process by the control unit 30 described later to the user terminal 14 in response to the request from the user terminal 14.

  The storage unit 22 includes, for example, a ROM, a RAM, a hard disk, and the like. The storage unit 22 stores an information processing program for operating each unit of the server 16. Further, a learning unit 24 is stored in the storage unit 22.

  The learning device 24 uses an image captured by the camera 12 as an input image and detects a plurality of feature points related to a vehicle included in the input image, as described in detail later. The learning device 24 in the present embodiment is configured to include, for example, a convolutional neural network.

  Generally, a convolutional neural network is configured to include a plurality of layers. Each layer of the convolutional neural network is composed of an array of neurons (for example, a two-dimensional array). In addition, in the convolutional neural network, a filter including a plurality of elements arranged two-dimensionally as in the neuron group is used. A filter is provided for each layer, and the size of the filter (the number of arranged elements) is usually smaller than the size of the neuron group to which it is applied (the number of arranged neurons). Each element of the filter is assigned a weight.

  A filter is applied to each group of neurons. Conceptually, filters are superimposed on some of the neurons. A product sum of each weight set to each element of the filter and the activity of a plurality of neurons (parts of neuron groups) overlapping the filter is calculated. That is, an inner product of a weight vector having the weight set to each element of the filter as an element of the vector and an activity vector having activities of a plurality of neurons as an element of the vector is calculated. The inner product is one activity of the output of the layer.

  The filters are applied while being shifted one by one (one neuron) or several neurons at a time in the vertical and horizontal directions on the neuron group. Thus, the arranged active groups are output as the output of the layer. The said active group becomes an activity of the neuron group of the following layer. By applying a filter to a plurality of layer neuron groups, a plurality of feature points of the vehicle are finally detected from the input image.

  As described above, in the present embodiment, the actual state of the learning device 24 includes various parameters related to the convolutional neural network (layer structure, neuron structure of each layer, number of filters in each layer, filter size, weight of each element of each filter, etc. And an execution program for processing an input image. Therefore, that the learning device 24 is stored in the storage unit 22 means that the various parameters and the process execution program are stored in the storage unit 22. The learning unit 24 may include a plurality of neural networks.

  The learning method of the learning device 24 will be described. The learning device 24 performs learning using a learning image including a vehicle and learning data including information related thereto. Incidentally, the learning of the learning device 24 means adjusting the weight of each element of the filter if the learning device 24 is a convolutional neural network.

  An example of the learning image 26 is shown in FIG. The learning image 26 may be any image as long as it includes a vehicle, and the acquisition method may be any method. Further, in the example of FIG. 3, the learning image 26 includes one vehicle, but the learning image 26 may include a plurality of vehicles.

  In the learning data, a plurality of feature points regarding the vehicle included in the learning image 26 (28a to 28k, hereinafter, a plurality of feature points regarding the vehicle included in the learning image 26 will be collectively referred to as feature points 28) It contains information indicating the position of In this embodiment, four feature points 28 related to the vehicle are intersection points of four corners (feature points 28a to 28d) of the roof of the vehicle, a line connecting feature points 28a and 28d, and a line connecting feature points 28b and 28c. The position of 13 feature points 28 of a certain feature point 28e, two headlights (feature points 28f and 28g), two tail lights (feature points 28h and 28i), and four tires (feature points 28j and 28k) Is known. As in the image shown in FIG. 3, there may be a case where all 13 feature points are not included in the image. In addition, the feature points 28 regarding the vehicle are not limited to the above-described positions, and may include other positions (for example, the positions of the front and rear license plates). The positions of the plurality of feature points are set so that as many feature points as possible appear in the image regardless of the orientation of the vehicle in the image.

  The information indicating the position of the feature point 28 is, for example, coordinates in the learning image 26.

  The learning data also includes information indicating the location of the vehicle indicated by each feature point 28. The location of the vehicle is a specific location of the vehicle, such as the center of the roof or the front right wheel. In the example of FIG. 3, the feature point 28a is the roof front right corner, the feature point 28b is the roof front left corner, the feature point 28c is the roof right rear corner, the feature point 28d is the roof left rear corner, the feature point 28e is the roof center, the feature point 28f Is a right headlight, feature point 28g is a left headlight, feature point 28h is a right taillight, feature point 28i is a left taillight, feature point 28j is a front left wheel, and feature point 28k is a rear left wheel Included in training data. When the learning image 26 includes a plurality of vehicles, the learning data includes information indicating the location of each feature point 28 distinguished for each vehicle (for example, the feature points 28 a The vehicle's 'roof right front corner', etc.).

  The learning data also includes information indicating the direction (vector in the present embodiment) from each feature point 28 to another feature point. In addition, as a vector connecting feature points, a vector automatically generated by a predetermined program based on information indicating the position of each feature point 28 can be used.

  Preferably, the learning data may include information indicating the weather when the learning image 26 is captured. The weather is, for example, sunny, cloudy, rain or snow. Preferably, the learning data may include information indicating whether or not each light (including headlights and tail lights) of the vehicle included in the learning image 26 is lit.

  The positions of the plurality of feature points 28 of the vehicle, the location of the vehicle indicated by each feature point 28, the weather when the learning image 26 is captured, and the lights of the vehicle included in the learning image 26 Each information which shows the lighting existence of each may be added as metadata of picture 26 for study, for example.

  The learning device 24 can detect a plurality of feature points related to the vehicle from the input image by learning using a large number of learning data. In particular, the learning device 24 performs learning using learning data in which the positions of the plurality of feature points 28 related to the vehicle and the location of the vehicle indicated by each feature point 28 are known. Thereby, the learning device 24 can identify and detect which part of the vehicle the detected feature point corresponds to.

  Being able to detect a plurality of feature points after identifying the corresponding vehicle location means that it is possible to detect the positional relationship between vehicle locations included in the input image. In other words, the learning device 24 learns from the data for learning the direction (vector in this embodiment) from the location (feature point) of the vehicle to another location (feature point), It can be said that the direction (vector) between each feature point detected from the input image can be detected. In addition, even when the learning image 26 includes a plurality of vehicles, the learning device 24 distinguishes for each vehicle in the data for learning, and then the location of the vehicle indicated by each feature point 28. Because it is known, it is possible to learn the orientation between each feature point, separately for each vehicle.

  Preferably, the learning device 24 performs learning so as to output the weather when the learning image 26 is captured, in addition to the feature points and the directions between the feature points. For example, when the weather is snow, snow may hide four corners and the center of the roof of the vehicle. Even in such a case, by learning so as to output the weather of the input image using the learning image 26 including the information indicating the weather, the learning device 24 detects the weather when the learning image 26 is taken. It is considered that feature points will be detected in consideration of being snow. As a result, it is possible to improve the detection accuracy when the learning device 24 detects a plurality of feature points related to the vehicle from the input image.

  Similarly, preferably, the learning device 24 performs learning so as to output lighting / non-lighting of each light of the vehicle included in the learning image 26 in addition to the feature points and the direction between the feature points. When the image photographed in the daytime and the image photographed in the nighttime are compared, the appearance of the image is largely different, for example, the luminance value near the position of the light of the input image changes according to the lighting status of the light. Even in such a case, the learning device 24 learns by using the learning image 26 including the information indicating the lighting presence / absence of each light of the vehicle to output the lighting presence / absence of each light of the vehicle included in the input image. It is considered that the feature point is to be detected in consideration of the lighting status of each light of the vehicle when the learning image 26 is photographed. As a result, it is possible to improve the detection accuracy when the learning device 24 detects a plurality of feature points related to the vehicle from the input image.

Returning to FIG. 2, the control unit 30 includes, for example, a CPU or a microcontroller. The control unit 30 controls each unit of the server 16 according to an information processing program stored in the storage unit 22. Further, as illustrated in FIG. 2, the control unit 30 also functions as a vehicle detection unit 32, a person detection unit 34, a processing calculation unit 36, and an identity determination unit 38.

  The vehicle detection unit 32 executes a process of detecting a vehicle from an input image which is an image captured by the camera 12. Specifically, the vehicle detection unit 32 inputs the input image to the learned learning device 24 and detects the vehicle from the input image based on the detection results of the plurality of feature points by the learning device 24 with respect to the input image .

  An example of the input image is shown in FIG. Hereinafter, the output of the learning device 24 and the vehicle detection processing by the vehicle detection unit 32 for the input image 40 illustrated in FIG. 4 will be described in detail. As shown in FIG. 4, in the present embodiment, the input image 40 includes a plurality of vehicles 42. The input image to be processed by the vehicle detection unit 32 may include only one vehicle 42 or may be an image not including the vehicle 42.

  When the vehicle detection unit 32 inputs the input image 40 to the sufficiently learned learning device 24, the learning device 24 outputs a heat map and a vector map.

  A heat map 50 for the input image 40 is shown in FIG. As described above, the learning device 24 detects a plurality of feature points on each vehicle from the input image 40. The heat map 50 is a map that indicates the positions of a plurality of detected feature points 52. The positions of the plurality of detected feature points 52 are represented by coordinates in the input image 40.

  A vector map 60 for the input image 40 is shown in FIG. As described above, the learning device 24 detects the orientation between the feature points 52 detected from the input image. In particular, when a plurality of vehicles are included in the input image, the learning device 24 distinguishes each vehicle and detects the direction between the feature points 52. The vector map 60 is a map that indicates the position and orientation of a vector 62 that indicates the orientation between feature points 52 for each vehicle. Although the feature points 52 are shown in the vector map 60 shown in FIG. 6, the vector map 60 may not include the feature points 52.

  Each vector 62 is shown between one feature point 52 and the other feature point 52, but its orientation (the direction from one feature point 52 to the other feature point 52, another feature) The direction from point 52 to one feature point 52) may be predefined. For example, in the present embodiment, each vector 62 is, in principle, front to back (for example, a direction from the roof right front corner to the roof right rear corner), right to left (for example, a direction from the roof right front corner to the roof left front corner) It is the direction of).

  In the vector map 60, a vector 62 may be shown only between adjacent (nearly) feature points 52. However, from one feature point 52 to all other items corresponding to the same vehicle as the feature point 52. The vector 62 may be shown between the feature point 52 of the image. In the vector map 60 shown in FIG. 6, only a part of the vectors 62 is shown.

  The vehicle detection unit 32 detects a vehicle from the input image 40 based on the feature points 52 detected in the heat map 50 and the vector map 60. Specific contents of the detection process will be described below.

The vehicle detection unit 32 performs a filtering process on the heat map 50, and extracts feature points having a probability equal to or more than a predetermined value from the group of feature points included in the heat map 50 as a plurality of effective feature points. Next, integration of the vector map 60 is performed on the feature points and the line segments connecting the feature points for all combinations between the extracted effective feature points. Then, a set of two feature points whose average of integrated values of vectors is equal to or more than a predetermined value is specified as a combination of correct answers. In this manner, effective feature points are grouped into a plurality of groups. Then, classification is performed, for example, group A having three feature points, group B having five feature points, group C having six feature points, or a single feature point not combined with other feature points. Be done. Then, a group including a predetermined number or more of feature points is specified as a group representing a vehicle. That is, it detects as a vehicle.

  As described above, according to the present embodiment, since the vehicle is detected based on the feature points 52 detected from the input image 40, a part of the vehicle is hidden or missed in the input image 40, or Vehicles can be detected from the input image 40 regardless of the angle. On the other hand, when the feature point 52 is not detected in the heat map 50, the vehicle detection unit 32 can determine that the input image 40 does not include a vehicle.

  In addition, as described above, the learning device 24 can detect the plurality of feature points 52 in which the corresponding vehicle location is identified. For example, in the learning device 24, the feature point 52a is the roof front right corner, the feature point 52b is the roof front left corner, the feature point 52c is the roof right rear corner, the feature point 52d is the roof left rear corner, the feature point 52e is the roof center, the feature point After identifying the right headlight 52f as the right headlight, the left headlight as the feature point 52g, the right front wheel as the feature point 52h and the right rear wheel as the feature point 52i, each feature point 52 is detected.

  The vehicle detection unit 32 can detect the direction of the vehicle included in the input image 40 based on the positional relationship between the plurality of feature points 52 at which the location of the vehicle is identified. For example, in the example of the heat map 50 shown in FIG. 5, the feature point 52c corresponding to the roof right rear corner is located on the upper left side of the feature point 52a corresponding to the roof right front corner. A feature point 52b corresponding to the left front corner of the roof is located on the upper right side of the corresponding feature point 52a. From such a positional relationship, the vehicle detection unit 32 can detect that the vehicle represented by the plurality of feature points 52a to i points the front side in the lower right direction in the input image 40.

  In addition, the vehicle detection unit 32 can detect the type of vehicle based on the positional relationship between the plurality of feature points 52 in which the location of the vehicle is identified. The type of vehicle is distinguished by the difference in its external form, and is, for example, the type of sedan, van, minicar, truck or taxi.

  For example, in the van, the distance from the position of the tire to the position of the roof is longer than that of the sedan. Further, in the truck, the distance from the position of the tire to the position of the roof is relatively long as compared to the sedan, and the length in the front-rear direction of the roof is long as compared to the sedan or van. Since the vehicles of each type have the difference in external shape as described above, the vehicle detection unit 32 can detect the type of the vehicle included in the input image 40 based on the positions of the plurality of feature points 52. Note that the distance between the plurality of feature points 52 may vary depending on the angle of the vehicle in the input image 40, so the angle of the vehicle is estimated based on, for example, the positional relationship of the feature points at the four corners of the roof of the vehicle. Above, it is preferable to detect the type of vehicle.

  As shown in FIG. 4, when the input image 40 includes a plurality of vehicles 42, in particular, when a plurality of vehicles 42 are close or overlapping, a plurality of feature points 52 corresponding to a plurality of vehicles are included in the heat map 50. It will be detected densely. For example, the feature point group 64 located in the lower part of the heat map 50 of FIG. 5 is such an example. In this case, although the location of the vehicle to which each feature point 52 corresponds is identified, it is difficult to properly distinguish the feature point group 64 into groups corresponding to the respective vehicles only from the heat map 50. There is a case.

In the present embodiment, feature points 5 corresponding to a plurality of vehicles are obtained by using the vector map 60.
Even when the feature point group 64 in which 2 is densely formed is formed, the feature point group 64 can be appropriately distinguished into groups corresponding to the respective vehicles. Specifically, each vector 62 included in the vector map 60 is shown between a feature point 52 of a certain vehicle and another feature point 52 of the vehicle. Therefore, based on the direction between the feature points 52 included in the feature point group 64, the feature point group 64 can be distinguished into a feature point group 64a and a feature point group 64b corresponding to each vehicle.

  The vehicle detection unit 32 can detect the direction of each vehicle included in the input image 40 by appropriately distinguishing the plurality of feature points 52 detected from the input image 40 for each vehicle. At the same time, the vehicle detection unit 32 can detect the number of vehicles included in the input image 40.

  Returning to FIG. 2, the person detection unit 34 detects a person included in the input image 40. The method of detecting a person from the input image 40 may be the same as the learning device 24 for detecting a vehicle, and another learning device learned to detect a person may be used, or Alternatively, known image analysis processing may be used. In particular, the person detection unit 34 detects the number of persons included in the input image 40.

  The processing operation unit 36 performs an operation based on the detection result of the vehicle from the input image by the vehicle detection unit 32 and the detection result of the person from the input image by the person detection unit 34. In the present embodiment, the vehicle detection unit 32 detects the number of taxis 70 with respect to the input image 40-2 captured in an overhead view of the taxi stand as shown in FIG. 7, and the person detection unit 34 detects a person Detect the number of 72. Then, the processing operation unit 36 performs an operation based on the number of taxis 70 and the number of persons 72 in the input image 40-2, and determines the demand for the taxi. Thus, the processing operation unit 36 functions as a taxi demand determination unit.

  As a method of detecting a taxi by the vehicle detection unit 32, various methods can be adopted. First, after the heat map 50 (see FIG. 5) is output by the learning device 24, image analysis is performed on a certain area including the feature point 52 corresponding to the center of the roof to detect a streetlight. Good. Of course, if the streetlight is detected, it can be determined that the vehicle is a taxi, and if the streetlight is not detected, it can be determined that the vehicle is not a taxi.

  In addition, information indicating that the vehicle included in the learning image 26 (see FIG. 3) is a taxi is added to the learning data, and the learning device 24 can identify the taxi from the input image 40-2. You may learn

  Alternatively, depending on the shooting target location of the camera 12, all vehicles included in the input image 40-2 may be considered to be taxis.

  The processing operation unit 36 detects the taxi 70 and the persons 72 included in the input image 40-2 by the above-described method, and determines that there is a demand for taxis when the number of persons 72 is larger than the number of taxis 70. For example, information indicating that there is a demand for a taxi at the taxi stand is transmitted to the user terminal 14 used by a taxi dispatch person in charge. In addition, the processing operation unit 36 calculates the difference between the number of taxis 70 and the number of persons 72 in the input image 40-2, and transmits information indicating the difference (that is, the number of required taxis) to the user terminal 14. You may do so.

  The processing operation unit 36 further calculates the future demand for taxis at the taxi stand based on statistical data of the number of taxis 70 and the number of persons 72 included in the plurality of past input images 40-2 at the taxi stand. It may be predicted.

  Returning to FIG. 2, the identity determination unit 38 performs processing of determining the identity of a vehicle (that is, identifying the same vehicle) in a plurality of input images captured at different timings by the same camera 12. According to the identity determination unit 38, it becomes possible to follow the same vehicle within the imaging range of a certain camera 12, and it is possible to detect what kind of movement the vehicle will do. In the following description, the sameness between the vehicle A and the vehicle B is determined between the first input image including the vehicle A and the second input image including the vehicle B captured at different timings by the same camera 12 The process to be performed will be described.

  First, the identity determination unit 38 detects color information of each pixel of a plurality of feature points of the vehicle A detected by the learner 24 from the first input image, and a plurality of vehicles B detected by the learner 24 from the second input image. The color information of each pixel of the feature point is acquired. The color information is a concept including hue, lightness (brightness), and saturation.

  In the present embodiment, while the learning unit 24 detects each feature point as the coordinates of one pixel of the first input image or the second input image, the identity determining unit 38 detects the feature points by the learning unit 24. An area having a certain area centered on one pixel is referred to as an area for acquiring color information (hereinafter referred to as a pixel area). Further, the area of the pixel area may be changed according to the size of the vehicle of the first input image or the second input image. For example, as the vehicle in the input image is larger, the area of the pixel area may be increased. The size of the vehicle in the input image can be detected based on the positions of a plurality of feature points at which the location of the vehicle is identified.

  In the present embodiment, the identity determination unit 38 represents the colors of a plurality of pixels (for example, an average value) after expressing the colors of the plurality of pixels included in each pixel area in the L * a * b * color space. , Median, mode). Then, the representative value of the obtained color is used as color information of the pixel area. The color of each pixel is expressed in the L * a * b * color space because the L * a * b * color space is designed to approximate human vision.

Note that feature amounts other than the above may be adopted as color information of each pixel area. For example, HOG (Histograms of Oriented Gradients) feature quantities of a plurality of pixels included in a pixel area may be calculated. Here, the HOG feature value is a feature value obtained by histogramming the gradient direction of the luminance of each pixel in the pixel area.

  As described above, the color information of the pixel corresponding to each feature point of the vehicle A included in the first input image and the color information of the pixel corresponding to each feature point of the vehicle B included in the second input image It is acquired.

  Next, the identity determination unit 38 compares the color information of each feature point of the vehicle A with the color information of each feature point of the vehicle B. Specifically, the color information of the feature point of the vehicle A and the color information of the feature point of the vehicle B are compared for each location of the vehicle. Thereby, the identity determination unit 38 determines the identity of the vehicle A and the vehicle B. Specifically, the difference (color difference) in color information between a certain feature point of the vehicle A (here, the roof right front corner) and a feature point of the roof right front corner, which is the corresponding location of the vehicle B, is calculated. Thus, color differences are calculated for all the locations.

  In the present embodiment, the color difference of the color information of the pixel of each feature point is calculated by the CIE 2000 color difference formula. This is because the CIE 2000 color difference equation is a calculation equation taking into consideration the hue dependency, the lightness dependency, and the saturation dependency, which are features of the color discrimination region of human eyes. That is, the color difference based on the CIE 2000 color difference formula is a value close to the color difference by human vision.

ここで、ｄは色差であり、α及びβは定数である。α及びβを適宜設定することにより、色差の値に対する類似度の値を調整することができる。 The difference (color difference) between the color information of each feature point of the vehicle A and the color information of each feature point of the vehicle B is shown in FIG. The identity determination unit 38 calculates the similarity for each feature point based on the calculated color difference. Specifically, the identity determination unit 38 calculates the degree of similarity to be larger as the color difference is smaller. In the present embodiment, the similarity is calculated from the color difference according to the following equation.

Here, d is a color difference, and α and β are constants. By setting α and β appropriately, it is possible to adjust the value of similarity to the value of color difference.

  FIG. 8 shows the similarity calculated for each feature point. In this manner, between the vehicle A and the vehicle B, a plurality of (about 13 in the present embodiment) feature points are calculated with respect to a plurality of similarities. The identity determination unit 38 obtains representative values (for example, an average value, a median value, and a mode value) of the calculated plurality of similarities. The representative value of the degree of similarity thus obtained is the degree of similarity between the vehicle A and the vehicle B. When there is a feature point not detected among a plurality of feature points of the vehicle A or the vehicle B due to missing or hiding in the first input image or the second input image, the similarity regarding the feature point is A predetermined value (50% in the present embodiment) is used.

  As described above, the similarity between the vehicle A included in the first input image and the vehicle B included in the second input image is calculated. When the calculated similarity is equal to or higher than a predetermined similarity threshold, the identity determination unit 38 determines that the vehicle A and the vehicle B are the same vehicle, and the calculated similarity is less than the similarity threshold. If there is, it is determined that the vehicle A and the vehicle B are not the same vehicle.

  As described above, the identity determination unit 38 determines the identity of the vehicle among the plurality of input images captured at different timings. Preferably, the same vehicle can be tracked in the moving image by performing the above-described process for each adjacent frame image included in the moving image captured by the camera 12.

  Hereinafter, the flow of processing of the server 16 according to the present embodiment will be described according to the flowchart shown in FIG.

  In step S10, the control unit 30 of the server 16 includes a learning image 26 including a vehicle, information indicating the positions of a plurality of feature points 28 related to the vehicle included in the learning image 26, and the vehicle indicated by each feature point 28. The learning device 24 is made to learn by repeating inputting learning data including information indicating a part to the learning device 24. Step S10 corresponds to a learning step.

  In step S12, the vehicle detection unit 32 inputs the input image 40 captured by the camera 12 to the learned device 24 that has been learned.

  In step S14, the learning device 24 outputs the heat map 50 and the vector map 60 to the input image 40 input in step S12.

  In step S16, the vehicle detection unit 32 determines whether the input image 40 includes a vehicle based on the heat map 50 output in step S14. Specifically, when the learning device 24 detects a feature point 52 related to a vehicle from the input image 40, the vehicle detection unit 32 determines that the input image 40 includes a vehicle. If it is determined that the input image 40 includes a vehicle, the process proceeds to step S18. If it is determined that the input image 40 does not include a vehicle, the process ends. Absent. Steps S12 to S16 correspond to the vehicle detection step.

  In step S18, the vehicle detection unit 32 detects the direction and type of the vehicle based on the positional relationship of the plurality of feature points 52 at which the location of the vehicle is identified. When the input image 40 includes a plurality of vehicles, the vehicle detection unit 32 distinguishes the plurality of feature points 52 for each vehicle using the vector map output in step S14, and then the orientations of the plurality of vehicles. , Type or number of units.

  As mentioned above, although embodiment which concerns on this invention was described, this invention is not limited to the said embodiment, A various change is possible unless it deviates from the meaning of this invention.

  For example, in the present embodiment, the server 16 as the information processing apparatus executes both the learning process of the learning device 24 and the vehicle detection process using the learning device 24. However, the learning process and the vehicle detection process are different It may be executed by the subject (apparatus) of For example, the learning unit 24 which has been learned in another device may be stored in the storage unit 22 of the server 16, and the server 16 may execute the vehicle detection process using the learning unit 24.

  In addition to vehicles, the present invention is also applicable to vehicles such as ships and airplanes.

  Reference Signs List 10 information processing system 12 camera 14 user terminal 16 server 18 communication line 20 communication unit 22 storage unit 24 learning unit 30 control unit 32 vehicle detection unit 34 person detection unit 36 processing operation unit 38 Identity check unit.

Claims (9)

Learning using learning data including a learning image including a vehicle, information indicating positions of a plurality of feature points regarding the vehicle included in the learning image, and information indicating a location of the vehicle indicated by each feature point The learning device to
A vehicle detection unit that inputs an input image to the learned device that has been learned, and detects a vehicle included in the input image based on a feature point related to a vehicle detected by the learner from the input image;
An information processing apparatus comprising: The vehicle detection unit detects the direction of the vehicle included in the input image based on the positional relationship of a plurality of feature points detected by the learning device from the input image and in which the location of the corresponding vehicle is identified. ,
An information processing apparatus according to claim 1, characterized in that. The learning device learns an orientation between the plurality of feature points of the vehicle included in the image for learning.
The vehicle detection unit distinguishes the plurality of vehicles included in the input image based on the orientation between the plurality of feature points detected from the input image by the learning device, and then determines the direction and number of the plurality of vehicles. Detect at least one,
An information processing apparatus according to claim 1, characterized in that. The learning data includes information indicating weather when the learning image is taken,
The learning device performs learning in consideration of the weather when the image for learning is taken.
The information processing apparatus according to any one of claims 1 to 3, characterized in that: The learning data includes information indicating whether or not the light of the vehicle included in the learning image is lit,
The learning device performs learning in consideration of whether the lights of the vehicle included in the image for learning are lit.
The information processing apparatus according to any one of claims 1 to 4, characterized in that: The vehicle detection unit detects the type of the vehicle included in the input image based on the positional relationship of a plurality of feature points detected by the learning device from the input image and in which the location of the corresponding vehicle is identified.
An information processing apparatus according to claim 1, characterized in that. The vehicle detection unit identifies that the type of vehicle included in the input image is a taxi based on the output of the learning device for the input image,
The information processing apparatus further includes:
A person detection unit that detects a person included in the input image by analyzing the input image;
A taxi demand determination unit that determines a taxi demand based on the number of taxis and the number of persons included in the input image;
The information processing apparatus according to claim 6, comprising: Computer,
Learning using learning data including a learning image including a vehicle, information indicating positions of a plurality of feature points regarding the vehicle included in the learning image, and information indicating a location of the vehicle indicated by each feature point The learning device to
A vehicle detection unit that inputs an input image to the learned device that has been learned, and detects a vehicle included in the input image based on a feature point related to a vehicle detected by the learner from the input image;
An information processing program characterized in that it functions as: Learning using learning data including a learning image including a vehicle, information indicating positions of a plurality of feature points regarding the vehicle included in the learning image, and information indicating a location of the vehicle indicated by each feature point Learning steps to learn the
A vehicle detection step of inputting an input image into the learned device, and detecting a vehicle included in the input image based on a feature point of the vehicle detected by the learner from the input image;
An information processing method comprising:

Images