JP2018163096A - Information processing method and information processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To dynamically give correct data related to object identification to a reception signal of a radar.SOLUTION: An information processing method is provided. The information processing method includes: calculating the position of an object in a three-dimensional space on the basis of a distance image; specifying an object area in a first signal intensity map acquired from a radar reception signal on the basis of the position of the object in the three-dimensional space; and associating the type of the recognized object with the object area.SELECTED DRAWING: Figure 8

Description

  The present invention relates to an information processing method and an information processing apparatus.

  In recent years, with the development of technology, various methods for tracking and identifying objects have been developed. For example, Non-Patent Document 1 and Non-Patent Document 2 include techniques for tracking and identifying moving objects based on three-dimensional information collected by LIDAR (Light Detection and Ranging, or Laser Imaging Detection and Ranging). It is disclosed.

  Non-Patent Documents 3 to 5 disclose techniques for identifying the position and type of an object on the image data based on the image data.

Naotaka Hosoo, two others, “Multiple types and multiple moving body tracking methods by LRF using angle-based multiple hypotheses”, Transactions of the Society of Instrument and Control Engineers, Society of Instrument and Control Engineers, May 2015, Vol. 51, no. 5, P297-308 Kiyosumi Shiroto, 3 others, “Pedestrian identification by high resolution laser data”, Journal of the Robotics Society of Japan, Robotics Society of Japan, December 2011, Vol. 29, no. 10, P963-970 Shaoqing Ren, 3 others, “FasterR-CNN: Towards Real-Time Object Detection with Region Proposal Networks”, June 4, 2015, arXiv, [February 10, 2017 search], Internet <URL: http: //tokkyo.shinsakijun.com/information/newtech.html> Joseph Redmon, 3 others, “YouOnly Look Once: Unified, Real-Time Object Detection”, June 8, 2015, arXiv, [Search February 10, 2017], Internet <URL: https: // arxiv .org / pdf / 1506.02640v1.pdf> Evan Shelhamer, two others, “Fully Convolutional Networks for Semantic Segmentation”, Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference, IEEE, June 7, 2015, 10.1109 / CVPR. 2015.7298965 Manuel Dudek, 3 others, “AMillimeter-Wave FMCW Radar System Simulator for Automotive Applications”, RadarConference (EuRAD), 2011 European, IEEE, October 12, 2011 Manuel Dudek, 4 others, “TheImpact of Antenna Characteristics on Target Detection in FMCW-Radar System Simulations for Automotive Applications”, Radio and Wireless Symposium (RWS), 2012 IEEE, IEEE, January 15, 2012, 10.1109 / RWS.2012.6175357

  By the way, in the identification of a moving object, it is expected to use a received signal acquired by a radar. However, when the intensity of the received signal is visualized, a false image appears in addition to the object to be identified on the data, so correct data for the object is manually assigned while checking only the received signal of the radar. It is very difficult to do. It is also possible to manually identify the reaction and type of the object from the radar received signal by comparing the radar received signal with the image data. In this case, however, the workload is high and a large amount of correct data is obtained. It is not practical as a means to obtain.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is a novel and capable of dynamically assigning correct data related to object identification to a received signal of a radar. An object is to provide an improved information processing method and information processing apparatus.

  In order to solve the above problems, according to an aspect of the present invention, a position of an object in a three-dimensional space is calculated based on a distance image, and a radar reception is performed based on the position of the object in the three-dimensional space. An information processing method is provided that includes specifying an object region in a first signal intensity map acquired from a signal and associating the recognized object type with the object region.

  The information processing method further includes recognizing the position of the object on the image data and the type of the object based on the photographed image data, and the association is performed on the object on the image data. May be further converted to coordinates corresponding to the first signal intensity map, and the object type and the object region are associated with each other.

  The specifying includes extracting a first object candidate region from the first signal strength map, and determining the object region from the first object candidate region based on a position of the object in the three-dimensional space. Identifying may further be included.

  The specifying may further include specifying the object area by comparing the center of gravity of the position of the object in the three-dimensional space with the center of gravity of the first object candidate area.

  The information processing method further includes extracting a second object candidate region from a second signal intensity map obtained by a simulation based on the position of the object in the three-dimensional space, and the specifying The method may further include specifying the object area based on the second object candidate area.

  The specifying may further include evaluating an overlap between the first object candidate area and the second object candidate area and specifying the object area.

  The first signal intensity map may be obtained by simulation based on the position of the object.

  The information processing method may further include radiating a radio wave to acquire the radar reception signal and generating a first signal intensity map based on the radar reception signal.

  The distance image may be acquired by LIDAR.

  The association calculates the existence probability of the object on the image data in the object area based on the center of gravity of the position of the object on the image data, and determines the type of the object and the object area. It may further include associating.

  In order to solve the above problem, according to another aspect of the present invention, a sensor unit that radiates radio waves and acquires a radar reception signal, and an identification unit that identifies an object based on the radar reception signal, And the identification unit identifies the object based on a learning result learned by associating an object region in a signal intensity map obtained from the radar reception signal and the type of the object acquired from image data. An information processing apparatus is provided.

  As described above, according to the present invention, it is possible to dynamically give correct data related to object identification to a received signal of a radar.

It is a figure which shows an example of the signal strength map which concerns on the 1st Embodiment of this invention. It is an example of correct data according to the embodiment. It is an example of a functional block diagram of the information processing apparatus according to the embodiment. It is a figure which shows an example of the object candidate area | region extracted by the area | region specific part which concerns on the embodiment. It is a figure for demonstrating the collision with the point group which concerns on the object which concerns on the same embodiment, and an object candidate area | region. It is a figure for demonstrating the identification of the object on the image by the identification part which concerns on the embodiment. It is a figure for demonstrating the conversion of the coordinate by the integration part which concerns on the same embodiment. It is a flowchart which shows the flow of operation | movement of the information processing apparatus which concerns on the same embodiment. It is a flowchart which shows the flow of operation | movement of the information processing apparatus which concerns on the 2nd Embodiment of this invention. It is a hardware structural example of the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the data set used for the learning which concerns on image recognition.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

<1. Background>
First, the background that led to the idea of this technical idea will be described. As described above, in recent years, various methods for identifying an object (hereinafter also referred to as an object) have been developed. As an example of the above-described method, there is image recognition for identifying the type of object on the image data.

  In recent years, research related to image recognition has been very active, and many competitions have been held to compete for the accuracy of image recognition. There are also databases that provide data sets for use in image recognition. For example, about 14 million data sets are disclosed in ImageNet, which is one of the above databases.

  FIG. 11 is a diagram illustrating an example of a data set used for learning related to image recognition. FIG. 11 shows image data IM1 used for image recognition and correct data CL relating to objects on the image data IM1. Here, the correct answer data CL may include, for example, information related to the segmentation of the object and information related to the type of the object, as illustrated. In the example shown in FIG. 11, the correct answer data CL includes information related to the objects O1 and O2, and indicates that the objects O1 and O2 are a person and a passenger car, respectively.

  Such a data set is usually generated manually and registered in a database. In the field of image recognition, it is possible to construct a highly accurate recognizer by using such a large data set for learning related to object recognition.

  On the other hand, as described above, it is expected that a received signal acquired by a radar is used for identifying a moving object. However, with regard to the received signals of the radar, there is a current situation that the number of existing data sets as described above is limited, and data sets that can be used for learning cannot be easily collected.

  This is because it is difficult for a person to confirm only the information that visualizes the received signal of the radar, and to identify the signal area resulting from the reaction of the object, so a data set such as image recognition cannot be generated easily. Because.

  For example, when a multi-element FMCW (Frequency Modulated Continuous Wave) radar is used, it is possible to calculate an angle, a distance, and a speed related to an object based on the received signal. In this case, for example, data (hereinafter referred to as a signal intensity map) in which the above-mentioned angle and distance are taken as two axes and the signal intensity is visualized can be obtained.

  FIG. 1 is a diagram illustrating an example of a signal intensity map according to the present embodiment. On the other hand, FIG. 2 is an example of correct data according to the present embodiment. Here, the correct answer data may be data including an object region and an object type specified in the signal strength map. The object area is a signal area indicating a reaction caused by the object. In the correct answer data according to the present embodiment, the type of the object may be indicated by color coding of the object area, addition of metadata, or the like. For example, the object area OC1 shown in FIG. 2 may be a signal area indicating a reaction caused by a passenger car.

  Here, referring to the signal strength map in FIG. 1, in the signal strength map obtained from the radar received signal, in addition to the reaction caused by the object to be identified, a strong signal strength is detected even at a position where the object does not exist. You can see that This is because a false image is generated due to a Fourier transform side lobe or the like.

  Therefore, in order to create correct answer data as shown in FIG. 2, in the signal intensity map, an object region is defined by eliminating a false image, and further, based on the time and position information at which the received signal is acquired. It is necessary to identify the object type from the captured image data and associate the object area with the object type. However, in order to perform the above-described processes manually, since the work load is large, a method for generating correct data more easily and more accurately has been desired.

  The present technical idea has been conceived by paying attention to the above points, and makes it possible to automatically generate correct data relating to a signal intensity map obtained from a received signal of a radar. Therefore, an information processing method and an information processing apparatus according to an embodiment of the present invention calculate a position of an object in a three-dimensional space based on a distance image, and perform radar based on the position of the object in the three-dimensional space. One of the features is that the object region in the signal intensity map acquired from the received signal is automatically specified. In addition, an information processing method and an information processing apparatus according to an embodiment of the present invention are characterized by automatically associating the identified object area with the recognized object type.

  According to the above feature of the information processing method and information processing apparatus according to an embodiment of the present invention, it is possible to generate a large amount of correct data related to a signal intensity map obtained by a radar, such as a deep neural network. By performing learning according to, it becomes possible to identify an object with high accuracy based on a radar reception signal.

<2. First Embodiment>
<< 2.1. Functional configuration example of information processing apparatus 10 >>
Next, a functional configuration example of the information processing apparatus 10 that realizes the information processing method according to the first embodiment of the present invention will be described. FIG. 3 is an example of a functional block diagram of the information processing apparatus 10 according to the present embodiment. Referring to FIG. 1, the information processing apparatus 10 according to the present embodiment includes a sensor unit 110, a data collection unit 120, a position calculation unit 130, an area specifying unit 140, an identification unit 150, and an integration unit 160.

(Sensor unit 110)
The sensor unit 110 according to the present embodiment includes sensors such as a radar 112, a LIDAR 114, and a camera 116 that receive a signal related to an object. At this time, the radar 112, the LIDAR 114, and the camera 116 are calibrated with each other.

  At this time, since both the radar 112 and the LIDAR 114 can acquire the distance and the angle to the object, it is possible to perform calibration based on the mutual installation relationship. As for the calibration between the camera 116 and the radar 112 or the LIDAR 114, it is possible to perform calibration based on the photographing characteristics of the camera 116 and the mutual installation relationship. Specifically, calculation based on a conversion formula that associates two-dimensional coordinates related to image data captured by the camera 116 with three-dimensional coordinates related to the target space can be performed. Note that the coordinates used in the following description are, for example, coordinates in real space with reference to any sensor such as the radar 112.

(Data collection unit 120)
The data collection unit 120 according to the present embodiment has a function of collecting signals received by the radar 112, the LIDAR 114, and the camera 116 of the sensor unit 110.

(Position calculation unit 130)
The position calculation unit 130 according to the present embodiment has a function of calculating the position of the object in the three-dimensional space based on the distance image. The distance image may be data having coordinate information in a three-dimensional space, and may be data of LIDAR 114 acquired by the data collection unit 120, for example. The position calculation unit 130 according to the present embodiment calculates the coordinates of a predetermined object set in advance, such as a car, a bicycle, or a person.

  At this time, in the distance image obtained by the LIDAR 114, the distance to the reflecting object (object) with respect to the direction of the laser irradiated at a constant angular interval appears as a point cloud. For this reason, the position calculation unit 130 performs processing for classifying the above point cloud for each object and calculating the position of each object. At this time, the position calculation unit 130 may perform the above processing using, for example, a particle filter as described in Non-Patent Document 1. In addition, for example, the position calculation unit 130 can perform the above-described processing using a technique as described in Non-Patent Document 2. As described above, the position calculation unit 130 according to the present embodiment uses only various methods to acquire only the point group caused by the object from the distance image, and classifies it for each object.

(Area specifying unit 140)
The area specifying unit 140 according to the present embodiment has a function of specifying an object area in the signal intensity map acquired from the radar reception signal based on the position of the object in the three-dimensional space calculated by the position calculating unit 130. More specifically, the area specifying unit 140 first extracts an object candidate area from the signal intensity map, and based on the position of the object in the three-dimensional space, the area specifying unit 140 actually originates from the object candidate area. An object area which is a signal area indicating a reaction is specified.

  At this time, for example, the region specifying unit 140 generates a signal intensity map having two axes of an angle and a distance calculated from a radar reception signal collected by the data collecting unit 120. The signal intensity map generated by the region specifying unit 140 is not limited to the above example, and may take two axes of distance and speed, or may take three axes of distance, angle, and speed. It may be. Further, the distance r and the angle θ may be converted into two-dimensional coordinates in the real space by x = r cos θ and y = rsin θ.

  For example, when the radar 112 is a multi-element FMCW radar, the region specifying unit 140 applies a two-dimensional FFT (Fast Fourier Transform) to the received signal, and thereby a signal intensity map with the distance and the angle as two axes. Can be obtained. That is, the region specifying unit 140 calculates the distance by performing FFT on the received signal for one frequency sweep, and calculates the angle by arranging the obtained results in the element (receiving antenna) direction and applying the FFT. can do. The region specifying unit 140 generates a signal intensity map as shown in FIG.

  Next, the area specifying unit 140 extracts an object candidate area from the signal intensity map as shown in FIG. At this time, the region specifying unit 140 may extract a region where the signal strength is equal to or greater than a threshold in the signal strength map. Note that it is not desirable to set the threshold value high in order to reduce the number of object candidate areas to be extracted. This is because, in the subsequent process, the object candidate areas are narrowed down based on the position of the object calculated by the position calculation unit 130, and therefore it is not a big problem that there are many extracted object candidate areas.

In addition, the area specifying unit 140 may extract object candidate areas using a certain threshold, but according to the radar equation shown in the following formula (1), the signal strength is inversely proportional to the square of the distance. Therefore, the threshold value may be changed according to the distance. For example, the region specifying unit 140 can set the threshold value Th (d) to a smaller value as the distance d increases by using the following formula (2). However, P in the following formula (1) represents transmission power [w], G represents antenna gain, σ ₀ represents reflection cross section [m ² ], and α in formula (2) is a constant. Good.

  FIG. 4 is a diagram illustrating an example of the object candidate area extracted by the area specifying unit 140 according to the present embodiment. FIG. 4 shows object candidate areas OC1 to OC4 extracted by the area specifying unit 140 in the signal intensity map. At this time, the object candidate areas OC1 to OC4 extracted by the area specifying unit 140 may include false images due to multiple reflection of radar, side lobes due to Fourier transform, and the like. For this reason, the area specifying unit 140 according to the present embodiment removes false images and the like by specifying the object candidate area by matching the point cloud related to the object acquired by the position calculating unit 130 and specifies the object area. be able to.

  FIG. 5 is a diagram for explaining a match between a point group related to an object and an object candidate area according to the present embodiment. FIG. 5 shows the object candidate regions OC1 to OC4 extracted in the signal intensity map and the point group LP related to the object projected on the signal intensity map.

At this time, the area specifying unit 140 calculates the center of gravity coordinates _{C r} of the object candidate region OC1～OC4, the center of gravity coordinates _{C l} a point cloud LP according to each object, respectively. Further, the region specifying unit 140 sets a set of point groups LP related to each object as Q, and sets the center-of-gravity coordinates of the nth object as C _l (n) _nεQ . Subsequently, the region specifying unit 140 specifies an object region related to the nth object by selecting an object candidate region having a centroid coordinate C _r closest to C _l (n) _nεQ for all Qs. can do. For example, in the example shown in FIG. 5, the area specifying unit 140 specifies the object candidate area OC1 as an object area, and excludes the object areas OC2 to OC4 corresponding to false images as shown in FIG. You may get

(Identification unit 150)
The identification unit 150 according to the present embodiment has a function of identifying the type of object. Specifically, the identification unit 150 according to the present embodiment identifies the position of the object and the type of the object on the image data based on the image data captured by the camera 116.

  FIG. 6 is a diagram for explaining identification of an object on an image by the identification unit 150. In the example of FIG. 6, image data IM2 captured by the camera 116 and an object O1 identified by the identifying unit 150 on the image data IM2 are illustrated. As described above, the identification unit 150 according to the present embodiment may identify an object on the image data and indicate the position and type of the object by a rectangular area.

  At this time, the identification unit 150 may identify the object by a technique using deep learning as described in Non-Patent Document 3 and Non-Patent Document 4, for example. The identifying unit 150 can also identify an object in pixel units using a deep learning method as described in Non-Patent Document 5, for example. Note that the above method is merely an example, and the identification unit 150 according to the present embodiment may identify the position and type of the object on the image data using various methods used in the image recognition field.

  In addition, the identification unit 150 according to the present embodiment may have a function of identifying the type of an object by a recognizer generated by learning using the received signal of the radar 112 and correct data. That is, the identification unit 150 identifies the type of the object based on a learning result learned by associating the object region in the signal intensity map obtained from the radar reception signal with the type of the object acquired from the image data. According to the above-described function of the identification unit 150, it is possible to identify an object using only a radar reception signal, and it is possible to realize highly robust object identification.

(Integration unit 160)
The integration unit 160 according to the present embodiment has a function of associating the object type recognized by the identification unit 150 (hereinafter also referred to as a label) with the object region specified by the region specifying unit 140. At this time, the integration unit 160 according to the present embodiment converts the object position on the image data recognized by the identification unit 150 into coordinates corresponding to the signal intensity map, so that the object type and the object area in the signal intensity map are converted. Is associated with.

  FIG. 7 is a diagram for explaining coordinate conversion by the integration unit 160 according to the present embodiment. On the left side of FIG. 7, image data IM3 in which pedestrians P1 and P2, which are examples of objects, and passenger cars V1 and V2 are photographed are shown. At this time, as shown on the right side of the drawing, the integration unit 160 according to the present embodiment, based on a conversion formula that associates the two-dimensional coordinates related to the image data IM3 with the three-dimensional coordinates related to the target space, pedestrians P1 and P2 The coordinates of the passenger cars V1 and V2 can be converted into coordinates with two axes of angle and distance.

  Subsequently, the integration unit 160 associates the object region specified by the region specification unit 140 with the type of object identified by the identification unit 150. At this time, the measurement accuracy of the position by the camera 116 is lower than that of the radar 112 and the LIDAR 114, and the position measurement by the camera 116 is a property of the position measurement based on the reaction from the reflection surface of the object by the radar 112 and the LIDAR 114. Is different.

Therefore, for example, the integration unit 160 according to the present embodiment sets C as the set of objects on the image data identified by the identification unit 150 and (θ _C , r _c ) as the barycentric coordinates of Q _i (εC). By applying a two-dimensional Gaussian distribution with variance σ _C centered at (θ _C , r _c ), the existence probability P _Qi (θ, r) at each coordinate of the object O _i is obtained. Subsequently, the integration unit 160 determines an object type (label _Rj ) associated with the j-th object region R _j specified by the region specification unit 140 using the following mathematical formula (3). However, in the following mathematical formula (3), P _Rj (θ, r) is 1 when (θ, r) is a coordinate in the object region R _j , and 0 otherwise. . Label (O) represents the object type of the object O.

  According to the above calculation by the integration unit 160, it is possible to select a label having the highest object existence probability obtained from the image data in the object region specified by the region specification unit 140.

<< 2.2. Flow of operation of information processing apparatus 10 >>
Next, an operation flow of the information processing apparatus 10 according to the present embodiment will be described. FIG. 8 is a flowchart showing an operation flow of the information processing apparatus 10 according to the present embodiment.

  Referring to FIG. 8, first, the data collection unit 120 collects sensor information from the sensor unit 110 (S1101). Here, the sensor information includes a reception signal by the radar 112, a distance image by the LIDAR 114, and image data by the camera 116.

  Next, the position calculation unit 130 calculates the position of the object in the three-dimensional space based on the distance image collected in step S1101 (S1102).

  Next, the area specifying unit 140 extracts an area having a signal intensity equal to or higher than a threshold in the generated signal intensity map as an object candidate area (S1103).

  Subsequently, the area specifying unit 140 specifies an object area from the object areas extracted in step S1103 based on the position of the object calculated in step S1102 (S1104).

  Next, the identification unit 150 identifies the position and type of the object on the image data (S1105).

  Next, the integration unit 160 converts the position of the object on the image data identified in step S1105 into coordinates corresponding to the signal intensity map, and associates the object type with the object area (S1106).

  The operation flow of the information processing apparatus 10 according to the present embodiment has been described in detail above. As described above, according to the information processing apparatus 10 according to the present embodiment, it is possible to automatically generate correct data in which a label indicating the type of an object is assigned to a signal area (object area) caused by the object. Become.

<3. Second Embodiment>
<< 3.1. Outline of Second Embodiment >>
Next, a second embodiment of the present invention will be described. In the first embodiment described above, when specifying the object area, the point cloud obtained by the LIDAR 114 and the object candidate area are matched with each other, and the closest coordinates of the center of gravity are associated with each other. However, in this method, when a plurality of objects are present in the vicinity, the possibility of incorrect association cannot be denied.

  Therefore, in the second embodiment of the present invention, when the point cloud obtained by the LIDAR 114 and the object candidate region are matched, the signal intensity map obtained from the simulation result when the point cloud is irradiated with radar is further added. Based on this, the object area may be specified.

  The actual received signal acquired by the radar 112 is closer to the data acquired by the simulation than the data acquired by the LIDAR 114. For this reason, in 2nd Embodiment using the simulation result which concerns on a radar, compared with 1st Embodiment which directly utilizes the point cloud acquired by LIDAR114, it is anticipated that the precision which suppresses a false image etc. increases. Is done.

  In the following description, differences from the first embodiment will be mainly described, and detailed description of configurations, functions, effects, and the like common to the first embodiment will be omitted.

<< 3.2. Identifying object areas based on simulation results >>
As described above, in the present embodiment, the object region is specified based on the simulation result relating to the radar. At this time, the region specifying unit 140 according to the present embodiment can obtain the above simulation result based on a technique described in Non-Patent Document 6 or Non-Patent Document 7, for example.

  Specifically, the region specifying unit 140 performs parameter setting related to the transmission wave by the radar, the incident wave to the object, the reflected wave from the object, and the amplification of the reflected wave, and performs signal processing on the calculated echo. , You can get the distance of the object.

First, the region specifying unit 140 calculates the transmission wave x (t) using, for example, the following mathematical formula (4). Here, in the following equation (4), X (t) is the original signal generated by the FMCW method, P is the peak power [W], G _s is the transmitter gain [dB], and L _s is The transmitter loss [dB] is shown respectively. The area specifying unit 140 can design a transmission wave by setting each parameter as described above. Examples of parameters related to FMCW include a frequency sweep width and a sweep time.

Next, the region specifying unit 140 calculates an incident wave on the object. At this time, the region specifying unit 140 may adopt a free space propagation model for propagation between antennas, and may adopt a round-trip model for setting the propagation distance. For example, the area specifying unit 140 calculates the incident wave y (t) to the object by the following mathematical formulas (5) to (7). Here, τ _rt in the following equation (5) indicates a point time [sec], and P _r and P _c in the following equation (7) are the radar position [m] and the object position [m], respectively. Indicates. The region specifying unit 140 can design the incident wave by setting parameters such as _Pr and _Pc .

  Next, the area specifying unit 140 calculates a reflected wave from the object. At this time, the region specifying unit 140 can calculate the reflected wave Sig (t) from the object by, for example, the following formulas (8) and (9). Here, RCS in Equation (8) below is a parameter indicating the reflection characteristics of the object. The area specifying unit 140 can also set the RCS based on the type of the object identified based on the image data.

Next, the area specifying unit 140 calculates an echo based on the reflected wave. At this time, the area specifying unit 140 can calculate the echo Echo (t) using, for example, the following formulas (10) and (11). In the following equation (10), w represents single dispersion noise, G _r represents receiver gain [dB], and L _r represents receiver loss [dB]. KB in _B ) represents a Boltzmann constant, B represents a sampling rate, T represents a reference temperature, and F represents noise intensity. The region specifying unit 140 can design an echo by setting parameters such as G _r , L _r , B, T, and F.

  Next, based on the echo Echo (t) obtained above, the region specifying unit 140 calculates the object distance Rng [m] using the following mathematical formulas (12) to (15). In Equation (15) below, B represents a frequency sweep width [dB], and T represents a frequency sweep time [sec].

  The area specifying unit 140 can calculate the angle by further arranging the distance Rng calculated as described above in the element (receiving antenna) direction and applying FFT, and obtain a signal intensity map based on the simulation. Hereinafter, a signal strength map obtained from an actual received signal is also referred to as a first signal strength map, and a signal strength map obtained through simulation is also referred to as a second signal strength map.

  At this time, since the false image resulting from the side lobe of the Fourier transform or the like may occur in the second signal intensity map based on the simulation, the region specifying unit 140 determines the object point group obtained by the position calculating unit 130 and Are extracted, and an object candidate area (hereinafter also referred to as a second object candidate area) related to the second signal intensity map is extracted.

Next, the area specifying unit 140, based on the second object candidate area R _s , object candidate area R _r (hereinafter referred to as the first object candidate area, the first signal intensity map based on the actual received signal). Also, the object region _RRs is specified. At this time, the area specifying unit 140 can specify the object area R _Rs by the following mathematical formula (16). In the following equation (16), N _Rr is the area of the first object candidate region R _r , N _Rs is the area of the second object candidate region R _s , and N _{Rr and Rs} are the first object candidate region R _r. And the area where the second object candidate region R _s overlaps.

That is, the area specifying unit 140 according to the present embodiment calculates IoU (Intersection of Unit) with all the first object candidate areas R _r in each second object candidate area R _s , and the value is By extracting the maximum first object candidate region R _r , it is possible to eliminate the false image and specify the object region R _Rs .

<< 3.3. Flow of operation of information processing apparatus 10 >>
Next, an operation flow of the information processing apparatus 10 according to the present embodiment will be described. FIG. 9 is a flowchart showing an operation flow of the information processing apparatus 10 according to the present embodiment. Note that the processing in steps S2101 to S2103, S2107, and S2108 in FIG. 9 may be substantially the same as the processing in steps S1101 to S1103, S1105, and S1106 shown in FIG.

  Referring to FIG. 9, in step S2103, the area specifying unit 140 extracts a first object candidate area from the first signal intensity map, and then performs a simulation related to a radar reception signal (S2104).

  Subsequently, the region specifying unit 140 extracts a second object candidate region from the second signal intensity map obtained from the simulation result in step S2104 (S2105).

  Subsequently, the area specifying unit 140 specifies an object area from the first candidate area based on the second object candidate area (S2106).

  The operation flow of the information processing apparatus 10 according to the present embodiment has been described above. According to the information processing apparatus 10 according to the present embodiment, by evaluating the overlap between the first candidate area based on the actual received signal and the second candidate area based on the simulation result, the false image from the first candidate area. Etc. can be eliminated more effectively.

<4. Third Embodiment>
Next, a third embodiment of the present invention will be described. In the second embodiment, the object region is specified from the first candidate region based on the actual received signal based on the second object candidate region extracted from the second signal intensity map based on the simulation result. Explained.

  On the other hand, in the third embodiment of the present invention, correct data may be generated based only on the simulation result related to the received signal of the radar. That is, the information processing apparatus 10 according to the present embodiment specifies an object area from a simulation result of a radar reception signal based on a distance image obtained by the LIDAR 114, and associates the identified object type with the object type. It is possible to generate correct answer data.

  At this time, in the above simulation result, a false image due to a side lobe of Fourier transform or the like may be generated as in the case of the second embodiment, but these are the same as the processing shown in the first embodiment. This can be eliminated.

  In addition to the data acquired by the LIDAR 114, the distance image used for the simulation of the radar reception signal according to the present embodiment may be, for example, a 3D model related to the object. Furthermore, when information related to the type of object is given in advance to the 3D model, the integration unit 160 can also associate the object region with the object type based on the above information.

<5. Hardware configuration example>
Next, a hardware configuration example of the information processing apparatus 10 according to an embodiment of the present invention will be described. FIG. 10 is a block diagram illustrating a hardware configuration example of the information processing apparatus 10 according to the present invention. Referring to FIG. 10, the information processing apparatus 10 includes, for example, a CPU 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input unit 878, and an output unit 879. A storage unit 880, a drive 881, a connection port 882, and a communication unit 883. Note that the hardware configuration shown here is an example, and some of the components may be omitted. Moreover, you may further include components other than the component shown here.

(CPU 871)
The CPU 871 functions as, for example, an arithmetic processing unit or a control unit, and controls all or part of the operation of each component based on various programs recorded in the ROM 872, RAM 873, storage unit 880, or removable recording medium 901. .

(ROM 872, RAM 873)
The ROM 872 is a means for storing programs read by the CPU 871, data used for calculations, and the like. In the RAM 873, for example, a program read by the CPU 871, various parameters that change as appropriate when the program is executed, and the like are temporarily or permanently stored.

(Host bus 874, bridge 875, external bus 876, interface 877)
The CPU 871, the ROM 872, and the RAM 873 are connected to each other via, for example, a host bus 874 capable of high-speed data transmission. On the other hand, the host bus 874 is connected to an external bus 876 having a relatively low data transmission speed via a bridge 875, for example. The external bus 876 is connected to various components via an interface 877.

(Input unit 878)
For the input unit 878, for example, a mouse, a keyboard, a touch panel, a button, a switch, a microphone, a lever, and the like are used. Furthermore, as the input unit 878, a remote controller (hereinafter referred to as a remote controller) that can transmit a control signal using infrared rays or other radio waves may be used. The input unit 878 includes an image sensor and a receiver element related to radar or LIDAR.

(Output unit 879)
The output unit 879 includes acquired information such as a display device (display device) such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, or a facsimile. Is a device capable of visually or audibly notifying a user. The output unit 879 may include various types of radar, LIDAR, and the like as described above.

(Storage unit 880)
The storage unit 880 is a device for storing various data. As the storage unit 880, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like is used.

(Drive 881)
The drive 881 is a device that reads information recorded on a removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable recording medium 901.

(Removable recording medium 901)
The removable recording medium 901 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, or various semiconductor storage media. Of course, the removable recording medium 901 may be, for example, an IC card on which a non-contact IC chip is mounted, an electronic device, or the like.

(Connection port 882)
The connection port 882 is a port for connecting an external connection device 902 such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal. is there.

(External connection device 902)
The external connection device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.

(Communication unit 883)
The communication unit 883 is a communication device for connecting to the network 903. For example, a communication card for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, ADSL (Asymmetric) A digital subscriber line) or a modem for various types of communication. Moreover, you may connect to telephone networks, such as an extension telephone network and a mobile telephone provider network.

<6. Summary>
As described above, an information processing method and an information processing apparatus according to an embodiment of the present invention calculate the position of an object in a three-dimensional space based on a distance image, and based on the position of the object in the three-dimensional space. One of the features is that the object region in the signal intensity map acquired from the radar reception signal is automatically specified. In addition, an information processing method and an information processing apparatus according to an embodiment of the present invention are characterized by automatically associating the identified object area with the recognized object type. According to such a configuration, it is possible to dynamically give correct data related to object identification to a radar reception signal.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  In addition, each step related to processing of the information processing apparatus 10 according to the embodiment of the present invention does not necessarily have to be processed in time series in the order described in the flowchart. For example, each step related to the processing of the information processing apparatus 10 may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

10 Information processing device 110 Sensor unit 112 Radar 114 LIDAR
116 camera 120 data collection unit 130 position calculation unit 140 region identification unit 150 identification unit 160 integration unit

Claims (11)

Calculating the position of the object in the three-dimensional space based on the distance image;
Identifying an object region in a first signal intensity map obtained from a radar received signal based on the position of the object in the three-dimensional space;
Associating the recognized object type with the object area;
including,
Information processing method. Recognizing the position of the object on the image data and the type of the object based on the captured image data;
Further including
The associating further includes converting the position of the object on the image data into coordinates corresponding to the first signal intensity map, and associating the object type with the object region,
The information processing method according to claim 1. The specifying includes extracting a first object candidate region from the first signal strength map, and determining the object region from the first object candidate region based on a position of the object in the three-dimensional space. Further including,
The information processing method according to claim 1 or 2. The specifying further includes comparing the center of gravity of the position of the object in the three-dimensional space with the center of gravity of the first object candidate region, and specifying the object region.
The information processing method according to claim 3. Extracting a second object candidate region from a second signal intensity map obtained by simulation based on the position of the object in the three-dimensional space;
Further including
The specifying further includes specifying the object region based on the second object candidate region;
The information processing method according to claim 3 or 4. The specifying further includes evaluating an overlap between the first object candidate area and the second object candidate area, and specifying the object area.
The information processing method according to claim 5. The first signal strength map is obtained by simulation based on the position of the object.
The information processing method according to claim 1. Irradiating radio waves to obtain the radar reception signal;
Generating a first signal strength map based on the radar received signal;
Further including
The information processing method according to claim 1. The distance image is acquired by LIDAR.
The information processing method according to claim 1.
The association calculates the existence probability of the object on the image data in the object area based on the center of gravity of the position of the object on the image data, and determines the type of the object and the object area. Further comprising:
The information processing method according to claim 2. A sensor unit that radiates radio waves and acquires radar reception signals;
An identification unit for identifying an object based on the radar reception signal;
With
The identification unit identifies the object based on a learning result learned by associating an object region in a signal intensity map obtained from the radar reception signal and the type of the object acquired from image data.
Information processing device.

Images