JP2016177757A - Information processing system, information processing method and program for detecting foot position - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing system, information processing method and program capable of suitably detecting a foot position.SOLUTION: An information processing system includes detection means for detecting a foot area by collating an input image with a plurality of templates, management means for managing foot position information about a foot position in each template of the plurality of templates, and output means for outputting the foot position on the basis of the foot position information corresponding to a template used to detect the foot area.SELECTED DRAWING: Figure 1

Description

  Some embodiments according to the present invention relate to an information processing system, an information processing method, and a program.

  2. Description of the Related Art Conventionally, a device that attaches a monocular camera to a car, monitors the front, detects a pedestrian with high risk from the obtained image, and warns is developed. In such a device, it is extremely important to find a pedestrian from the image. In particular, among pedestrians, pedestrians that are closer to the car have a higher risk of collision, so it is desirable to be able to detect the distance from the car to the pedestrian. For example, distance can be easily detected by using a distance measuring sensor such as a millimeter wave radar, but problems such as an increase in product cost arise.

  Here, if the pedestrian's foot position can be accurately detected on the image, the distance between the car and the pedestrian can be measured even from the image of the monocular camera based on the correspondence between the camera and the real world. It is.

Incidentally, for example, there is a method described in Patent Document 1 for image processing on an image in which a person is captured. Patent Document 1 discloses that a plurality of template images are prepared and the degree of matching between the template image and the input image is calculated to identify the state of the arm or foot.
In addition, Patent Documents 2-4 also disclose related technologies.

Japanese Patent Laid-Open No. 08-279044 Japanese Patent Laid-Open No. 10-255057 JP 2007-004721 A JP 2007-188417 A

  However, although the technique described in Patent Document 1 identifies the state of each part such as an arm or a leg, no consideration is given to a technique for detecting the position of a specific part such as a foot.

  Some aspects of the present invention have been made in view of the above-described problems, and an object thereof is to provide an information processing system, an information processing method, and a program capable of suitably detecting a foot position.

  An information processing system according to the present invention manages a foot position information relating to a foot position for each template of the plurality of templates, and a detecting unit that detects a foot region by comparing an input image with a plurality of templates. Management means, and output means for outputting the foot position based on the foot position information corresponding to the template used for detecting the foot area.

  An information processing method according to the present invention includes a step of detecting a foot area by comparing an input image with a plurality of templates, and manages foot position information relating to a foot position for each template of the plurality of templates. The information processing apparatus performs a step and a step of outputting a foot position based on the foot position information corresponding to the template used for detecting the foot area.

  The program according to the present invention includes a process of detecting a foot area by comparing an input image and a plurality of templates, and a process of managing foot position information relating to a foot position for each template of the plurality of templates. And causing the computer to execute a process of outputting the foot position based on the foot position information corresponding to the template used for detecting the foot area.

  In the present invention, “part”, “means”, “apparatus”, and “system” do not simply mean physical means, but “part”, “means”, “apparatus”, “system”. This includes the case where the functions possessed by "are realized by software. Further, even if the functions of one “unit”, “means”, “apparatus”, and “system” are realized by two or more physical means or devices, two or more “parts” or “means”, The functions of “device” and “system” may be realized by a single physical means or device.

  ADVANTAGE OF THE INVENTION According to this invention, the information processing system, the information processing method, and program which can detect a foot position suitably can be provided.

It is a functional block diagram which shows schematic structure of the foot position detection apparatus which concerns on 1st Embodiment. It is a flowchart which shows the flow of a process of the foot position detection apparatus shown in FIG. It is a flowchart which shows the specific example of the flow of a process at the time of producing | generating foot position distribution information. It is a block diagram which shows the structure of the hardware which can mount the foot position detection apparatus shown in FIG. It is a functional block diagram which shows schematic structure of the foot position detection apparatus which concerns on 2nd Embodiment.

  Embodiments of the present invention will be described below. In the following description and the description of the drawings to be referred to, the same or similar components are denoted by the same or similar reference numerals.

(1 First Embodiment)
1 to 4 are diagrams for explaining the first embodiment. Hereinafter, the present embodiment will be described along the following flow with reference to these drawings. First, an overview of the entire system is described in “1.1”, and then an overview of the functional configuration of the system is described in “1.2”. Next, “1.3” shows the flow of processing, and “1.4” shows a specific example of a hardware configuration capable of realizing this system. Finally, the effects and the like according to the present embodiment will be described after “1.5”.

(1.1 Overview)
2. Description of the Related Art Conventionally, there is a device that attaches a monocular camera to a car, monitors the front, and detects and warns a pedestrian with high risk from the obtained image. Thus, it is important to detect a pedestrian from an image. As a method for detecting a pedestrian, for example, a method of collecting a large amount of pedestrian images and non-pedestrian images and generating a statistical classifier based on these images can be considered.

  Here, among the detected pedestrians, the pedestrians approaching the car are considered to have a higher risk of collision. Therefore, it is desirable to be able to detect the distance from the car to the pedestrian. As a method for detecting the distance, for example, a distance measuring sensor such as a millimeter wave radar may be used. However, since the sensor needs to be prepared separately from the camera, the product cost increases.

  In this regard, if the pedestrian's foot position can be accurately detected on the image, the distance between the car and the pedestrian can be measured even from the image of a monocular camera based on the correspondence between the camera and the real world. It is.

  As a method for finding the pedestrian's feet, similar to the method for detecting pedestrians described above, a large amount of images of the feet and non-foot are collected, and a statistical discriminator is generated based on these images. A method can be considered. In particular, as one of the methods for detecting a pedestrian, there is a method in which the pedestrian is decomposed into the head, the lower body, the feet, etc., and each result is detected instead of using an image of the pedestrian, and the results are integrated. is there. When a pedestrian is detected by such a method, the area of the foot in the image can be specified without adding a calculation cost by using the detection result.

  However, even if a foot region is found by such a method, it is difficult to specify the foot position. In the image showing the area of the foot, if only the area below the shin is detected as the area of the foot, if it is detected as the area including all below the thigh, the leg may be open due to circumstances such as walking There can be various states. Because of such abundant variations, even if the foot region is specified, the foot position cannot be detected with high accuracy.

  Furthermore, the amount of computing resources that can be used by an in-vehicle device attached to a car is usually considerably limited as compared with a general PC (Personal Computer) or workstation. This is because there are severe restrictions on the amount of power that can be used and the amount of heat generated by the device. Therefore, it is necessary to improve the estimation accuracy of the foot position with as little calculation cost as possible. The method of detecting the foot position according to the present embodiment takes this point into consideration.

  Hereinafter, an outline of the method according to the present embodiment will be described. The foot position detection apparatus according to the present embodiment collects a large number of foot images and non-foot images as a method for finding the foot of a pedestrian, and finds a foot region by generating a statistical identifier based on these images. Shall.

  In this embodiment, a template classifier such as GLVQ (Generalized Learning Vector Quantization) is used as the statistical classifier. The template type discriminator learns a foot template and a non-foot template on the feature space, and identifies a foot region using the template that is the learning result. In the step detection process using such a discriminator, a template that is closest to the feature amount of the partial image region that is considered to be a step is searched, and if the template is a step template, the partial image region is determined as the step region, If it is a non-foot template, it is determined as a non-foot area.

  Here, if there are 100 foot templates, for example, the 100 foot templates represent various foot shapes. Therefore, the corresponding foot position (coordinate position) is different for each foot template. For example, in the first foot template, the foot position is in the lower right part, and in the second foot template, the foot position is in the lower left part. Variation occurs.

  In the foot position detection method according to the present embodiment, the foot position detection accuracy can be increased with a simple configuration by learning in advance the variation in the foot position for each template.

(1.2 System functional configuration)
Hereinafter, the system configuration of the foot position detection device 100 which is the information processing system according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating a functional configuration of the foot position detection device 100. The foot position detection device 100 includes an input unit 110, a foot classifier 120, a foot position calculation unit 130, an output unit 140, and a storage unit 150.

  The input unit 110 reads images one by one from, for example, a storage medium (not shown), an input interface, or the like, and passes the input partial image that is a partial area to the foot classifier 120.

  The foot classifier 120 collates the feature amount of the input partial image with a plurality of templates 151 indicating foot / non-foot learned in advance. As a result, the footstep discriminator 120 outputs the template 151 having the highest matching score and the matching score to the foot position calculation unit 130.

  The foot position calculation unit 130 uses the template number of the template 151 with the highest probability, that is, the highest matching score input from the foot classifier 120 as a key, and stores foot position distribution information 152 corresponding to the template from the storage unit 150. Search for.

The output unit 140 outputs the foot position distribution information 152 obtained as a result to external software, a display device, or the like. Note that the information output by the output unit 140 may be, for example, the average position of the distribution, the peak position of the distribution, or the like, not the foot position distribution information 152 itself.
The storage unit 150 is a storage device that stores the template 151 and the foot position distribution information 152.

  The template 151 is obtained as a result of learning by collecting a large number of foot images and non-foot images. The template 151 can include a plurality of foot templates corresponding to the feature amounts of the foot partial images having various shapes, and a plurality of non-foot templates corresponding to the feature amounts of the non-foot partial images having various shapes. As described above, the foot classifier 120 collates the input partial image with each template 151 and selects the most likely template 151 to determine whether the input partial image is a foot region or a non-foot region. Can be determined.

  The foot position distribution information 152 is information relating to the foot position distribution corresponding to each of the plurality of templates 151. The foot position distribution information 152 is generated by learning shown in FIG.

(1.3 Process flow)
Next, the processing flow of the foot position detection device 100 will be described with reference to FIGS. Each processing step, which will be described later with reference to FIGS. 2 and 3, can be executed in any order or in parallel as long as there is no contradiction in the processing contents. Other steps may be added. Further, a step described as a single step for convenience can be executed by being divided into a plurality of steps, and a step described as being divided into a plurality of steps for convenience can be executed as one step.

(1.3.1 Flow of foot position detection)
First, the flow of foot position detection will be described with reference to FIG. FIG. 2 is a flowchart showing a flow of processing when the foot position detection device 100 according to the present embodiment detects the foot position.

  First, the input unit 110 reads images one by one from, for example, a storage medium (not shown), an input interface, or the like, and outputs an input partial image that is a partial area to the foot classifier 120 (S201). The foot classifier 120 calculates a feature amount from the input partial image that has been input, and collates the feature amount with each template 151 stored in the storage unit 150 (S203). As a result, the foot classifier 120 outputs the most likely template 151 and its matching score to the foot position calculating unit 130 (S205).

  If the most likely template 151 is a foot template, the foot position calculating unit 130 searches the foot position distribution information 152 using the template number of the template as a key (S207). Thereby, the foot position distribution information 152 corresponding to the most likely template 151 is obtained.

  The output unit 140 outputs information on the foot position from the foot position distribution information 152 obtained in this way (S209). The output information may be the foot position distribution information 152 itself, or may be information on the average position of the distribution indicated by the foot position distribution information 152 or information on the position at which the distribution reaches a peak.

(1.3.2 Flow of generation of foot position distribution information 152)
Next, the flow of processing when generating the foot position distribution information 152 will be described. Note that the foot position distribution information 152 may be generated by the foot position detection device 100, or may be generated by another information processing device and stored in the foot position detection device 100.

  In the following description, it is assumed that there are n step templates, and the variable k is an integer indicating a template number. (Px, py) are variables corresponding to coordinates indicating the foot position in the learning foot image that is the input partial image. It is assumed that there are n learning foot images, and the foot position coordinates are specified in the learning foot images.

P _k (x, y) indicates the probability that the foot position of the template with the template number k exists at the coordinates (x, y) in the input partial image. X and Y indicate the sizes of the input partial images in the x and y directions, respectively. c (k) is a variable in which the number of times the template with the template number k is specified is entered.

First, all the values of P _k (x, y) and c (k) are initialized to 0 for all combinations of k, x, and y (S301). Also, the variable i indicating the number of the learning foot image that is the input partial image is initialized to 1 (S303). Here, it is assumed that there are a total of n learning foot images.

  The input unit 110 receives an input of a learning foot image [i] that is an i-th learning foot image (S305). Also, (px, py) indicating the coordinates in the learning foot image is initialized to (1, 1) (S307). The input unit 110 inputs the learning foot image [i] to the foot classifier 120 such that the foot position is (px, py) (S309).

  The foot classifier 120 compares the feature amount of the learning foot image [i] with the foot template included in the template 151. As a result, the nearest foot template m is specified (S311). In response, Pm (px, py) and c (m) are added one by one (S313).

  In order to move the coordinates of the processing target, px is incremented by 1 (S315). As a result, when px is equal to or smaller than the width X of the learning footstep image [i] (No in S315), the process returns to S309 at (px, py) and continues the process. If px is larger than the width X of the learning image [i] (Yes in S315), it means that the process has been completed for all the current y coordinates, so that 1 is added to py and 1 is added to px. (S319). If py is less than or equal to the height Y of the learning foot image [i] (No in S321), the process returns to S309 to process the (px, py).

  If py is larger than the height Y of the learning image [i] (Yes in S321), it means that the processing has been completed for all coordinates, and processing is performed on a new learning foot image. I is incremented by 1 (S323). As a result, if i is less than or equal to the number n of the learning foot images (No in S325), the processing for the new learning foot images is performed based on S305.

On the other hand, when i is larger than the number n of the footstep images for learning (Yes in S325), it means that the processing for all the footstep images for learning is finished. Therefore, P _k (x, y) = P _k (x, y) / c (k) is calculated for all combinations of (k, x, y) (S327). Accordingly, the foot position distribution information 152 indicating the foot position distribution at each coordinate (x, y) can be generated for each template k.

In order to smooth the distribution shape of Pk (x, y), smoothing processing such as convolution of a Gaussian filter may be further performed. Further, the foot position distribution information 152 may be P _k (x, y) itself, but the average position of the distribution and the peak position of the distribution may be changed or added thereto.

(1.4 Hardware configuration)
Hereinafter, an example of a hardware configuration when the above-described foot position detection device 100 is realized by a computer will be described with reference to FIG. Note that the function of the foot position detection device 100 is not necessarily realized by an information processing device such as a single computer, and can be realized by a plurality of information processing devices.

  As shown in FIG. 4, the foot position detection device 100 includes a processor 401, a memory 403, a storage device 405, an input interface (I / F) 407, a data I / F 409, a communication I / F 411, and a display device 413.

  The processor 401 controls various processes in the foot position detection apparatus 100 by executing a program stored in the memory 403. For example, the processes related to the input unit 110, the foot classifier 120, the foot position calculation unit 130, and the output unit 140 described with reference to FIG. 1 are temporarily stored in the memory 403 and are mainly operated on the processor 401. It is feasible.

  The memory 403 is a storage medium such as a RAM (Random Access Memory). The memory 403 temporarily stores a program code of a program executed by the processor 401 and data necessary for executing the program. For example, a stack area necessary for program execution is secured in the storage area of the memory 403.

  The storage device 405 is a non-volatile storage medium such as a hard disk or a flash memory. The storage device 405 includes an operating system, various programs for realizing the input unit 110, the foot classifier 120, the foot position calculation unit 130, and the output unit 140,

Various data including the template 151 and the foot position distribution information 152 are stored as the storage unit 150. Programs and data stored in the storage device 405 are referred to by the processor 401 by being loaded into the memory 403 as necessary.

  The input I / F 407 is a device for receiving input from the user. Specific examples of the input I / F 407 include a keyboard, a mouse, a touch panel, and various sensors. The input I / F 407 may be connected to the foot position detection device 100 via an interface such as a USB (Universal Serial Bus).

  The data I / F 409 is a device for inputting data from outside the foot position detection apparatus 100. Specific examples of the data I / F 409 include a drive device for reading data stored in various storage media. It is also conceivable that the data I / F 409 is provided outside the foot position detection device 100. In this case, the data I / F 409 is connected to the foot position detection device 100 via an interface such as USB.

  The communication I / F 411 is a device for performing data communication with a device or the like external to the foot position detection device 100 by wire or wireless. It is also conceivable that the communication I / F 411 is provided outside the foot position detection device 100. In this case, the communication I / F 411 is connected to the foot position detection device 100 via an interface such as a USB.

  The display device 413 is a device for displaying various information. Specific examples of the display device 413 include a liquid crystal display and an organic EL (Electro-Luminescence) display. The display device 413 may be provided outside the foot position detection device 100. In that case, the display device 413 is connected to the foot position detection device 100 via, for example, a display cable.

(1.5 Effects according to this embodiment)
As described above, the foot position detection apparatus 100 according to the present embodiment can infer the foot position from the detected foot area by learning in advance the relationship between the template used for foot detection and the foot position. it can. Thereby, it is possible to detect the foot position of the foot position detecting device 100 that detects the foot region using the template with high accuracy. In addition, the method according to the present embodiment can be applied to an inexpensive system because the necessary calculation cost for detecting the foot position is low.

(2 Second Embodiment)
The second embodiment will be described below with reference to FIG. FIG. 5 is a block diagram illustrating a functional configuration of a foot position detection device 500 that is an information processing system. As shown in FIG. 5, the foot position detection device 500 includes a detection unit 510, a management unit 520, and an output unit 530.

  The detection unit 510 detects the foot area by comparing the input image with a plurality of templates. As a method for detecting a foot region using the template, for example, it is conceivable to use an advanced statistical method such as GLVQ.

The management unit 520 manages foot position information regarding the position of the foot for each template of the plurality of templates. The output unit 530 outputs the foot position based on the foot position information corresponding to the template used for detecting the foot area.
By mounting in this way, the foot position detection device 500 according to the present embodiment can preferably detect the foot position.

(3 Additional notes)
Note that the configurations of the above-described embodiments may be combined or some of the components may be replaced. The configuration of the present invention is not limited to the above-described embodiment, and various modifications may be made without departing from the scope of the present invention.

  A part or all of each of the above-described embodiments can be described as in the following supplementary notes, but is not limited thereto. Moreover, the program of this invention should just be a program which makes a computer perform each operation | movement demonstrated in said each embodiment.

(Appendix 1)
Detection means for detecting a foot area by comparing an input image with a plurality of templates, management means for managing foot position information regarding the position of the foot for each template of the plurality of templates, An information processing system comprising: output means for outputting a foot position based on the foot position information corresponding to the template used for detection.

(Appendix 2)
The information processing system according to appendix 1, wherein the foot position information managed by the management means is information indicating a distribution of the foot positions for each template.

(Appendix 3)
The information according to appendix 1 or appendix 2, wherein the foot position output by the output means is at least one of the foot position information, an average value of the foot positions, or a position indicating a peak of the distribution of the foot positions. Processing system.

(Appendix 4)
The information processing system according to any one of appendix 1 to appendix 3, wherein the detection unit detects the foot region by GLVQ (Generalized Learning Vector Quantization).

(Appendix 5)
A step of detecting a foot area by comparing an input image with a plurality of templates, a step of managing foot position information regarding the position of the foot for each template of the plurality of templates, and detection of the foot area An information processing method in which an information processing apparatus performs a step of outputting a foot position based on the foot position information corresponding to the used template.

(Appendix 6)
The information processing method according to appendix 5, wherein the managed foot position information is information indicating a distribution of the foot positions for each template.

(Appendix 7)
The information processing method according to appendix 5 or appendix 6, wherein the output foot position is at least one of the foot position information, an average value of the foot positions, or a position indicating a peak of the distribution of the foot positions.

(Appendix 8)
The information processing method according to any one of appendix 5 to appendix 7, wherein the foot region is detected by GLVQ (Generalized Learning Vector Quantization).

(Appendix 9)
A process of detecting a foot area by matching an input image with a plurality of templates,
For each template of the plurality of templates, a process for managing foot position information related to a foot position, and a process for outputting a foot position based on the foot position information corresponding to the template used for detecting the foot area. A program to be executed by a computer.

(Appendix 10)
The program according to appendix 9, wherein the managed foot position information is information indicating a distribution of the foot positions for each template.

(Appendix 11)
The program according to appendix 9 or appendix 10, wherein the output foot position is at least one of the foot position information, an average value of the foot positions, or a position indicating a peak of the distribution of the foot positions.

(Appendix 12)
The program according to any one of appendix 5 to appendix 7, wherein the foot region is detected by GLVQ (Generalized Learning Vector Quantization).

100: foot position detection device 110: input unit 120: foot identifier 130: foot position calculation unit 140: output unit 150: storage unit 151: template 152: foot position distribution information 401: processor 403: memory 405: storage device 407: Input interface 409: Data interface 411: Communication interface 413: Display device 500: Foot position detection device 510: Detection unit 520: Management unit 530: Output unit

Claims (6)

Detecting means for detecting a step region by comparing an input image with a plurality of templates;
Management means for managing foot position information relating to the position of the foot for each template of the plurality of templates;
An information processing system comprising: output means for outputting a foot position based on the foot position information corresponding to the template used for detecting the foot area. The foot position information managed by the management means is information indicating a distribution of the foot positions for each template.
The information processing system according to claim 1. The foot position output by the output means is at least one of the foot position information, an average value of the foot positions, or a position indicating a peak of the distribution of the foot positions.
The information processing system according to claim 1 or 2. The detection means detects the foot region by GLVQ (Generalized Learning Vector Quantization).
The information processing system according to any one of claims 1 to 3. Detecting a step region by matching an input image with a plurality of templates;
For each template of the plurality of templates, managing step position information regarding the position of the step;
An information processing method in which an information processing apparatus performs a step of outputting a foot position based on the foot position information corresponding to the template used for detecting the foot area. A process of detecting a foot area by matching an input image with a plurality of templates,
For each template of the plurality of templates, a process for managing foot position information regarding the foot position;
A program for causing a computer to execute a process of outputting a foot position based on the foot position information corresponding to the template used for detecting the foot area.

Images