JP7224592B2 - Information processing device, information processing method, and program - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus or the like that recognizes objects in the vicinity of a moving object using time-series three-dimensional information acquired by an observation apparatus.

Conventionally, there is a technology called LIDAR (see, for example, Non-Patent Document 1). LIDAR is one of the remote sensing technologies using light, which measures the scattered light from pulsed laser irradiation and analyzes the distance to a distant target and the properties of the target.

Also, conventionally, there is a technique called a distance image camera (see, for example, Non-Patent Document 2). A depth image camera is a camera that can obtain three-dimensional depth information in real time.

Wikipedia, "LIDAR", [online], [searched on August 19, 2018], Internet [URL: https://ja.wikipedia.org/wiki/LIDAR] "Capability of TOF range image camera", [online], [searched on August 19, 2018], Internet [URL: http://www.brainvision.co.jp/wp/support/tech/capability_of_tofcamera.pdf# search=%27TOF+%E8%B7%9D%E9%9B%A2%E7%94%BB%E5%83%8F%E3%82%AB%E3%83%A1%E3%83%A9%E3%81 %AE%E5%AE%9F%E5%8A%9B%27]

However, in the conventional technology, it is difficult to recognize an object using original information at two or more points in time, which is a set of three-dimensional coordinates of a specific portion of the object existing around the moving object.

Specifically, when combining point clouds observed at different points (at different points in time) by a moving observation device, the result inevitably becomes an unorganized point cloud. However, in general, the positional relationship (nearest neighbor points, etc.) between irregular point clouds is not self-explanatory, so object recognition using irregular point clouds requires a large computational cost and is not easy.

In addition, in actual observations, we have to use observation equipment with a limited field of view and resolution. Since only part of the point cloud of the target object can be acquired, it is difficult to recognize objects existing around the observation device.

Furthermore, when observations are made from multiple locations at different times using moving observation equipment, fluctuations in the position acquisition accuracy of the observation equipment and the effects of temporary shielding due to obstacles, etc., can affect the accuracy of observation equipment. Since the absolute coordinates of the point cloud of the object may fluctuate, it is difficult to recognize objects existing in the vicinity of the observation device.

In order to solve the above, in the present invention, an appropriate feature amount of each object is acquired for each observation point, and the objects around the observation device are recognized using the feature amounts acquired at a plurality of points.

The information processing apparatus of the first aspect of the present invention provides information that serves as a basis for specifying the position in a three-dimensional space of an object existing around a moving object, and is a set of three-dimensional coordinates of a specific portion of the object. An original information storage unit that stores an original information group, which is the original information, in association with mobile body position information indicating the position of the mobile body at each of two or more points in time; An object extraction unit for extracting one or more objects using a group of information, and a feature amount of each of the one or more objects extracted by the object extraction unit using a group of original information associated with two or more points in time. a feature amount acquiring unit to acquire, a same object recognizing unit that takes correspondence of the same object at two or more different points in time, and two or more feature amounts of the same object that the same object recognizing unit has taken correspondence; A feature amount synthesizing unit that obtains a feature amount set of the same object using the above feature amounts, and an object that obtains an object identifier that identifies the same object using the feature amount set obtained by the feature amount synthesizing unit. The information processing apparatus includes a recognition unit, and an output unit that outputs the object identifier acquired by the object recognition unit and the absolute position information of the same object.

With such a configuration, it is possible to easily recognize an object using two or more original information groups that are sets of three-dimensional coordinates of specific portions of the object existing around the moving object.

Further, in the information processing apparatus of the second invention, in contrast to the first invention, the feature amount acquisition unit is the original information group at the first time point, and the original information group that can specify the entire one object is to acquire the overall feature amount of the one object, and to obtain the partial feature of the one object using the original information group at the second point in time, from which a part of the one object can be specified The feature amount synthesizing unit acquires the overall feature amount and the partial feature amount of the same object for which the same object recognition unit has corresponded, and using the overall feature amount and the partial feature amount, It is an information processing device that acquires a set of feature amounts of the same object.

With such a configuration, an object is identified by using two or more original information groups, which are a set of three-dimensional coordinates of a specific portion of the object existing around the moving body, and using the entire feature amount and the partial feature amount of the object. easy to recognize.

Further, in the information processing apparatus of the third invention, in contrast to the second invention, the overall feature amount is size information related to the size of the object or the aspect ratio of the object, and the partial feature amount is the shape related to the shape of the object. It is an information processing device that is information.

With this configuration, by using two or more original information groups that are a set of three-dimensional coordinates of a specific portion of an object existing around the moving object, the size information or aspect ratio and shape information of the object can be appropriately obtained. The object can be easily recognized using the size information or aspect ratio and shape information of the object.

Further, the information processing apparatus of the fourth invention is, for any one of the first to third inventions, an original information group associated with two or more points in time and two or more positions of a moving body at each point in time. an absolute position information acquisition unit that acquires one or more absolute position information related to the absolute positions of the one or more objects extracted by the object extraction unit, using the information, and the position information of the moving body and the one or more objects an output diagram constructing unit that constructs an output diagram that is a three-dimensional diagram in which the relative positional relationship between the moving object and the one or more objects can be recognized using the two or more absolute position information; The unit is an information processing device comprising a diagram output unit that outputs an output diagram, and an object identifier output unit that outputs the object identifier acquired by the object recognition unit to a position corresponding to the absolute position information of the same object. .

With such a configuration, an object identifier can be output so that the object in the three-dimensional space drawing can be recognized.

Further, the information processing apparatus of the fifth invention, in contrast to any one of the first to fourth inventions, uses two or more sets of information having an object feature amount set and an object identifier to perform machine learning. The object recognition unit applies the feature amount set acquired by the feature amount synthesizing unit to the learning unit to generate the machine learning algorithm. is an information processing device that acquires an object identifier.

With such a configuration, an object can be recognized using a learning device.

Further, in the information processing apparatus of the sixth invention, in the information processing apparatus of any one of the first to fifth inventions, the object recognition unit recognizes one or more types of objects among a telephone pole, a traffic light, and a post on the road. An information processing device that acquires an object identifier to be identified.

Such a configuration allows recognition of one or more types of objects on the road, such as telephone poles, traffic lights, or posts.

According to the information processing apparatus of the present invention, it is possible to easily recognize an object using two or more original information groups that are a set of three-dimensional coordinates of a specific portion of an object existing around a moving object.

Block diagram of information processing device 1 according to Embodiment 1 Flowchart explaining the operation of the information processing device 1 Flowchart for explaining an example of same-source information acquisition processing Flowchart for explaining an example of the same moving body position information acquisition processing Flowchart explaining same object extraction processing Flowchart explaining the same object recognition processing Flowchart for explaining the same output diagram construction processing Conceptual diagram of the same information processing device 1 Conceptual diagram of observation by the same observation device 802 Diagram showing the same correspondence table Diagram showing an example of the same output diagram Overview of the computer system Block diagram of the same computer system

Hereinafter, embodiments of an information processing apparatus and the like will be described with reference to the drawings. It should be noted that, since components denoted by the same reference numerals in the embodiments perform similar operations, repetitive description may be omitted.

(Embodiment 1)
The information processing apparatus described in the present embodiment uses three-dimensional data acquired from an observation device at each point to recognize an object without determining its attributes, and also recognizes the positional relationship between the observation point and the object (for example, distance, etc.), an appropriate feature amount is acquired (for example, overall dimensions for a long distance, surface shape for a short distance, etc.). Further, the information processing device identifies the same object recognized at each observation point based on the position of each observation point obtained by a device such as GPS or the self-position recognition technology described later, or on the basis of object tracking technology. , and by synthesizing these feature amounts, the identifier (type) of the object is identified.

Further, in the present embodiment, an information processing apparatus that constructs and outputs an output diagram that is a three-dimensional diagram acquired using an original information group will be described.

FIG. 1 is a block diagram of an information processing device 1 according to this embodiment.

The information processing device 1 includes a storage unit 11 , a reception unit 12 , a processing unit 13 and an output unit 14 .

The storage unit 11 includes an original information storage unit 111 , a learning device storage unit 112 and a correspondence table storage unit 113 .

The processing unit 13 includes an original information acquisition unit 131, a moving body position information acquisition unit 132, an original information storage unit 133, an object extraction unit 134, a feature amount acquisition unit 135, an absolute position information acquisition unit 136, a same object recognition unit 137, a feature It comprises a quantity synthesizing unit 138 , an object recognizing unit 139 , and an output diagram constructing unit 140 .

The output unit 14 has a diagram output unit 141 and an object identifier output unit 142 .

Various types of information are stored in the storage unit 11 . Various types of information are, for example, two or more laser transmitting means information. The laser transmission means information is information having a laser identifier for identifying the laser transmission means, transmission means position information for specifying the position of the laser transmission means, and transmission means direction information for specifying the radar transmission direction of the laser transmission means. . Note that the laser transmission means is means for transmitting a laser and receiving its reflected signal. For example, LIDAR comprises a number of laser emitting means.

Further, the various types of information are, for example, a group of original information to be described later. The original information group is information acquired by the original information acquisition unit 131 . Further, the various types of information are, for example, a learning device to be described later and a correspondence table to be described later.

The original information storage unit 111 stores two or more original information groups in association with the mobile body position information. The mobile body position information is information indicating the position of the mobile body at two or more points in time. The mobile body position information is acquired by the mobile body position information acquiring unit 132 that acquires mobile body position information indicating the position of the mobile body at each of two or more points in time. Note that the moving object is an observation device that moves. Details of the original information group will be described later.

The learning device storage unit 112 stores a learning device obtained by performing learning processing using a machine learning algorithm using two or more sets of information having an object feature amount set and an object identifier. Machine learning includes, for example, SVM, decision tree, random forest, deep learning, etc., but any machine learning algorithm is acceptable.

The correspondence table storage unit 113 stores a correspondence table having two or more pieces of correspondence information that are pairs of object feature amount sets and object identifiers. The feature amount set includes, for example, information on the width range of the object, information on the height range of the object, information on the surface shape of one surface of the object, and the aspect ratio of the object. An object identifier is information for identifying an object, such as an object name and an object ID.

Information used when acquiring an object identifier may be a learning device or a correspondence table.

The reception unit 12 receives various instructions, information, and the like. Various instructions, information, and the like are, for example, an operation start instruction, an operation end instruction, and an output instruction of the information processing apparatus 1 . Further, the operation start instruction is an instruction to start the operation of the information processing device 1, but may be an instruction to start moving the moving body. The operation end instruction is an instruction to end the operation of the information processing device 1 . The output instruction is an instruction to output an output diagram, and has, for example, a point-in-time identifier that identifies the point in time when the original information group on which the output diagram is based was obtained.

Here, reception means reception of information input from input devices such as keyboards, mice, and touch panels, reception of information transmitted via wired or wireless communication lines, and recording on optical discs, magnetic discs, semiconductor memories, etc. This is a concept that includes acceptance of information read from a medium.

Any means such as a touch panel, keyboard, mouse, or menu screen may be used as input means for various instructions and information.

The processing unit 13 performs various types of processing. The various processes include, for example, the original information acquisition unit 131, the moving body position information acquisition unit 132, the original information storage unit 133, the object extraction unit 134, the feature amount acquisition unit 135, the absolute position information acquisition unit 136, and the same object recognition unit. 137, a feature amount synthesis unit 138, an object recognition unit 139, and an output diagram construction unit 140. FIG.

The original information acquisition unit 131 acquires an original information group at two or more points in time. The original information group has one or more pieces of original information.

The original information acquisition unit 131 may acquire one or more pieces of original information covering 360 degrees around the moving object at one point in time. In addition, when the original information acquisition unit 131 acquires two or more pieces of original information covering 360 degrees, a time delay may occur depending on the region within the 360 degree range.

The original information is information that serves as a basis for specifying the position in the three-dimensional space of an object that exists around the moving object. The original information is three-dimensional coordinates of a specific portion (specific point) of an object that exists around the moving body. The original information has three-dimensional data of an observation device (not shown). The original information group is a set of points representing the 3D coordinates of specific parts such as people and things existing in the vicinity observed from the observation device, such as point cloud data or range image data. . Note that the observation device is a device capable of continuously acquiring 3D data as observation data. The observation device is, for example, the above-described LIDAR, three-dimensional camera, ToF sensor, stereo camera, or the like. The observation device may be a system configured by combining two or more devices as long as it outputs 3D data in the same coordinate system as a result. Also, the original information acquisition unit 131 may be considered to include an observation device, or may be considered to acquire original information from the observation device. Also, the three-dimensional data is, for example, three-dimensional relative coordinate values with the observation device or the moving body as the origin.

The original information is laser information, for example. The laser information is a set of return time and laser identifier. A laser identifier is information for identifying a laser transmission means. The laser identifier may be information specifying the position of the irradiated laser. For example, the original information acquisition unit 131 receives reflections of lasers transmitted by two or more laser transmission means, acquires the return time from the transmission of the laser to the reception of the reflection, and compares the return time with the laser transmission means. Laser information, which is a pair with a laser identifier for identification, is acquired for each laser transmission means. The original information acquisition unit 131 may be considered to include a LIDAR that emits a laser beam, or may be implemented by software, an MPU, or the like that acquires original information from the LIDAR. Note that LIDAR may also be referred to as LiDAR.

The original information group may be, for example, range images. The original information acquisition unit 131 acquires, for example, one or more distance images captured by a distance image camera. Note that the distance image includes coordinate values of each pixel in the image in a three-dimensional space. Also, the original information acquisition unit 131 may be considered to include a camera (which may be called a distance image sensor) that captures a distance image, and is implemented by software that acquires the distance image captured by the camera, an MPU, and the like. can be

The mobile position information acquisition unit 132 acquires mobile position information. Mobile body position information is usually information that specifies a position in a three-dimensional space (hereinafter sometimes simply referred to as "space"). Mobile position information usually has a structure of (x, y, z).

The mobile position information acquisition unit 132 includes, for example, a GPS receiver, and acquires the coordinate values (latitude, longitude, altitude) of the absolute position from the GPS receiver.

The mobile body position information acquisition unit 132 acquires the mobile body position information of the moving mobile body, for example, using one or more original information groups acquired by the original information acquisition unit 131 . Further, mobile body position information is, for example, information indicating a relative position. When the mobile body position information is information indicating a relative position, for example, the starting point of the mobile body is the origin (0, 0, 0). If the absolute position (usually (latitude, longitude, altitude)) of the starting point of the mobile body is known, the mobile body position information can be converted into information indicating the absolute position. In other words, position information such as mobile body position information and object position information handled in this specification may be information indicating a relative position or information indicating an absolute position. Note that the object position information is information indicating the positions of the points forming the object. The object position information is preferably relative position information with the moving body position information as the origin.

The mobile body position information acquisition unit 132 acquires mobile body position information at two or more points in time, for example, using two or more original information groups.

More specifically, the moving body position information acquiring unit 132, for example, extracts feature points from two pieces of original information at different points in time, and extracts two feature points corresponding to the two pieces of original information (for example, (x ₁₁ , y ₁₁ , z ₁₁ ) and (x ₁₂ , y ₁₂ , z ₁₂ )) are detected, and the amount of movement (for example, (x ₁₂ −x ₁₁ , y ₁₂ −y ₁₁ , z ₁₂ −z ₁₁ ).Then, the mobile body position information acquisition unit 132 calculates, for example, mobile body position information (eg, (x ₀ , y ₀ , z ₀ )) indicating the immediately preceding position of the mobile body and the amount of movement (eg, , (x ₁₂ −x ₁₁ , y ₁₂ −y ₁₁ , z ₁₂ −z ₁₁ ) and the current position information of the mobile unit (for example, (x ₀ +x ₁₂ −x ₁₁ , y ₀ +y ₁₂ −y ₁₁ , z ₀ +z ₁₂ −z ₁₁ )), where the feature points are corner points of the object, specific points on the outline of the object, etc. From the original information, which is a set of three-dimensional data, the feature points Techniques for obtaining points are well known.

The original information storage unit 133 associates the original information acquired by the original information acquisition unit 131 with the mobile object position information acquired by the mobile object position information acquisition unit 132 at the time corresponding to the original information, and stores the original information. Stored in unit 111 .

The object extraction unit 134 extracts one or more objects using the original information group associated with two or more points in time.

The object extraction unit 134 uses the original information group to identify one or more objects (“identification” may also be called “recognition”). In other words, the object extraction unit 134 identifies continuous point groups of two or more pieces of three-dimensional coordinate information included in the original information group. Note that a continuous point group of three-dimensional coordinate information is a set of points at specific positions of the object. Specifying a point group means obtaining a point group for each object, associating the point group with each object, and the like, and it is sufficient that a set of points at specific positions of an object is recognized as a group. Also, a continuous point group of three-dimensional coordinate information is, for example, a set of points whose distance is within a threshold or smaller than a threshold. A continuous point group of three-dimensional coordinate information is a set of specific positions (points) of one object, and a point group is a set of three-dimensional coordinate information.

Here, it is preferable that the object extraction unit 134 associates the specified point group of each object with a flag indicating whether or not it is the point group of the entire object. In other words, it is preferable for the object extraction unit 134 to acquire a flag for each object that indicates whether the original information group at one point in time contains information on the entire object or information on only a part of the object. is. For example, the object extraction unit 134 extracts the three-dimensional coordinate information of any point in the continuous point group, which is a set of specific points of the recognized object, from the original information group at one point in time. If the information is not the three-dimensional coordinate information of the points of the outer frame, it is determined that the information of the entire recognized object exists in the original information at one point in time. For example, the object extraction unit 134 converts the three-dimensional coordinate information of any point in a continuous point group, which is a set of specific points of the recognized object, into the three-dimensional coordinate information of the points of the outer frame of the original information. In a certain case, it is determined that only part of the recognized object exists in the original information at one point in time.

The feature amount acquisition unit 135 acquires the feature amount of each of the one or more objects extracted by the object extraction unit 134 using the original information group associated with two or more points in time. The feature amount of the object is, for example, size information and shape information. The feature amount of the object is, for example, the ratio of the sides of the circumscribed rectangular parallelepiped obtained from the principal component analysis and the aspect ratio.

The feature amount acquisition unit 135 extracts one or more points extracted by the object extraction unit 134 using a point group that is a set of specific points of an object in the original information group associated with each of two or more points in time. Get the features of an object.

It is preferable that the feature amount acquisition unit 135 acquires the entire feature amount of the one object by using the original information group at the first point in time, and by which the whole of the one object can be specified. Here, the overall feature amount is a feature amount for the entire object. The overall feature amount is, for example, size information of the object or aspect ratio of the object.

It is preferable that the feature amount acquisition unit 135 acquires the size information of the one object by using the original information group at the first point in time, which can specify the entire one object.

If the feature amount acquisition unit 135 is able to acquire size information from two or more original information groups at each point in time and from each of the original information groups that can identify the entire object, the size information with the largest size is adopted. is preferred.

If the feature amount acquisition unit 135 is able to acquire size information from each of the original information groups at two or more points in time and from which an entire object can be specified, then the size information is acquired from the original information group at the latest point in time. It is preferable to employ the size information provided. In such a case, the observation device is moving while approaching one object.

If the feature amount acquisition unit 135 is able to acquire size information from each of two or more original information groups at each point in time, and from each of the original information groups from which an entire object can be specified, the proportion of the object in the original information group. It is preferable to adopt the size information obtained from the original information group with the largest (largest occupied by the object).

It is preferable that the feature amount acquisition unit 135 acquires the partial feature amount of the one object using the original information group at the second point in time, in which only a part of the one object can be specified. . A partial feature amount is a feature amount of a portion of an object. A partial feature amount is, for example, shape information of an object.

It is preferable that the feature amount acquisition unit 135 acquires the shape information of the one object using the original information group at the second point in time, in which a part of the one object can be specified.

Note that the second time point is a time point different from the first time point. Also, the second time point is usually a time point after the first time point. Also, the shape information is information on the shape of the surface, for example, information on the surface in front of the observation device.

The absolute position information acquisition unit 136 uses the original information group associated with two or more points in time and the moving body position information at the two or more points in time to extract one or more objects extracted by the object extraction unit 134. Obtain one or more absolute position information about the absolute position. The absolute position information may be (latitude, longitude, altitude) or (x, y, z) in a three-dimensional space with a specific point as the origin (0, 0, 0). Absolute position information is information on absolute three-dimensional coordinates of a specific point of an object.

The absolute position information acquisition unit 136 normally calculates two or more pieces of absolute position information using each three-dimensional coordinate information and mobile body position information of each piece of original information included in the original information group. In other words, the absolute position information acquisition unit 136 uses the three-dimensional coordinate information (e.g., (x _a , _ya , _za )) of the original information as offsets to the mobile position information (e.g., (x _b , y b , y _b , z _b )) is added to calculate object position information (eg, (x _a +x _b , ya ₊ y _b , za ₊ z _b )), which is relative three-dimensional coordinate information.

The absolute position information acquisition unit 136 calculates two or more pieces of absolute position information using the three-dimensional coordinate information of each point forming the point group for each object identified by the object extraction unit 134 and the moving body position information. That is, the absolute position information acquisition unit 136 moves the three-dimensional coordinate information (for example, (x _a , _ya , z _a )) of each point constituting the point group for each object identified by the object extraction unit 134 as an offset. Absolute position information ₍ for _example , ( _{xa + xb} , _ya + _yb , _za +z _b )).

Also, the absolute position information acquisition unit 136 preferably associates a set of acquired absolute position information for each object. A set of absolute position information may be called an absolute position information group.

The same object recognition unit 137 matches two or more identical objects at different points in time. The same object recognition unit 137 may use any algorithm for matching the same object at two or more different points in time.

The same object recognition unit 137 uses the original information group associated with each of two or more points in time to match the same object at two or more points in time.

The same object recognition unit 137 uses, for example, one or more absolute position information for each of one or more objects for each of two or more points in time acquired by the absolute position information acquisition unit 136 to identify two or more different points in time. Take the correspondence of the same object. Here, taking correspondence means, for example, associating a group of absolute position information of the same object. Taking correspondence means, for example, making an absolute position information group including all two or more absolute position information groups at respective points of time of the objects for which correspondence is established as the absolute position information group of the object.

The same object recognizing unit 137 recognizes objects at different points of time that are close to each other by using the positions of the objects obtained from the group of absolute position information of the objects extracted from the group of original information at two or more different points of time, for example. is an object of The closeness of the positions of two objects means, for example, that the spatial distance between figures formed by contours extracted from each absolute position information group is small. Note that instead of the absolute position information group, a set of three-dimensional coordinate information of each point forming a point group for each object may be used. Further, when determining the spatial distance, based on the past history, based on the Kalman filter, etc., the movement vector of the object from the older time point 1 to the newer time point 2 is predicted, and the predicted position of the object and each Object positions may be compared.

The same object recognition unit 137 may, for example, match the same object at two or more different points in time using one or more feature amounts acquired at two or more points in time. That is, for example, if there are an object for which feature amounts A and B are acquired at time 1 and an object for which feature amounts A and C are acquired at time 2, and the common feature amount A is similar , the same object recognition unit 137 determines that the object at time 1 and the object at time 2 are the same object.

Also, the same object recognition unit 137 may match the same object at two or more different points in time, for example, using both the position and the feature amount of the object. For example, if there are multiple objects at time 2 within a certain distance from object 1 at time 1, it is determined that the object at time 1 and the object at time 2 that has the most similar features are the same object. do. Further, for example, the similarity between the feature amount of the object at time 1 and the feature amount of the object at time 2 is weighted by the spatial distance of each object, and then the score is calculated. 2 is determined to be the same object as the object with the highest score.

As described above, the same object recognition unit 137 preferably uses the position information of the object acquired from the original information group at two or more different points in time to match the same object at two or more points at different points in time. be. Note that the positional information of the object here may be the absolute positional information described above or the relative positional information. The relative position information is, for example, position information with the position of the moving body as the origin, that is, three-dimensional coordinate information of the original information.

The feature amount synthesizing unit 138 acquires two or more feature amounts of the same object correlated by the same object recognition unit 137, and acquires a feature amount set of the same object using the two or more feature amounts. Note that the feature amount set is, for example, size information and shape information. The shape information is, for example, information on the surface shape of the object. The information on the surface shape is, for example, information on the surface shape of one surface of the object. Further, the surface shape information is, for example, smoothness.

The object recognition unit 139 uses the feature amount set acquired by the feature amount synthesizing unit 138 to acquire an object identifier that identifies the same object.

The object recognition unit 139 applies, for example, the feature amount set acquired by the feature amount synthesizing unit 138 to a learning device, and acquires an object identifier by a machine learning algorithm. Any machine learning algorithm may be used as described above.

The object recognition unit 139, for example, searches the correspondence table for a feature amount set (for example, feature amount vector) that is most similar to the feature amount set (for example, feature amount vector) acquired by the feature amount synthesizing unit 138, and searches for the feature amount Get the object identifier paired with the set from the correspondence table.

The object recognition unit 139 preferably obtains object identifiers that identify one or more types of objects such as utility poles, traffic lights, or posts on the road.

The object extraction unit 134, the feature amount acquisition unit 135, the absolute position information acquisition unit 136, the identical object recognition unit 137, the feature amount synthesis unit 138, and the object recognition unit 139 are, for example, any of the following (1) to (3) By the processing of , the object identifier is obtained using the original information group at two or more points in time.
(1) First processing example of obtaining an object identifier

A first example of processing is object recognition processing when object recognition and self-position recognition are performed by independent recognition processing. Self-location awareness is the awareness of the position of the observation device.

The processing unit 13 performs the following processing using, for example, a set of three-dimensional coordinate values (referred to as “3D points”) possessed by the original information group. First, the processing unit 13 performs object recognition processing using a set of one or more 3D points. Object recognition processing is processing for identifying a set of 3D points that constitute one object. Next, the processing unit 13 acquires, for example, one or more feature amounts relating to the shape of each object from a set of 3D points of each object. The feature amount is, for example, size (eg, width, height, depth), smoothness, and the like. Next, the processing unit 13 determines the type of the object (an example of the attribute value of the object) using, for example, one or more feature amounts. The object type is an object identifier. Machine learning techniques such as decision trees, deep learning, support vector machines, and decision trees can be used for the process of determining the type of object. Next, the processing unit 13 acquires a set of 3D points (which may also be called object position information) in association with the determined object type information.

In addition, the object recognition process has the following method specifically, for example. First, the first object recognition processing will be explained. The processing unit 13 determines that a set of points that are so close that the distances of the plurality of 3D points satisfy a predetermined condition are points forming the same object, and collects the 3D points for each object.

Further, the processing unit 13 may perform object recognition processing by, for example, a second method, which is a grid map method described below. That is, the processing unit 13 partitions the three-dimensional space into a plurality of BOXes. Then, the processing unit 13 assumes that all 3D points existing in each BOX are 3D points forming the same object. Next, the processing unit 13 acquires feature points (for example, the point closest to the center, the median value of a plurality of points, etc.) from the 3D points of each BOX. Next, BOXes whose feature points are close enough to satisfy a predetermined condition are determined to be BOXes forming the same object. As described above, a set of 3D points forming one object has been detected (object recognition has been completed). A BOX is generally a three-dimensional cube or rectangular parallelepiped.

Also, the processing unit 13 may perform object recognition processing according to the third method, for example, using normal vectors. Such a method is a well-known technique called Region Growing Segmentation. That is, the processing unit 13 acquires, for example, a normal vector calculated from a group of points existing in a certain range around the point of interest (starting point). Then, the processing unit 13 constructs an object by combining neighboring points having similar normal vectors from the starting point as a region.

Then, for example, the processing unit 13 stores two or more pieces of object position information of each object in a buffer (not shown) in association with the moving body position information acquired from one original information group. It is preferable that the processing unit 13 also store the attribute values of each object together in a buffer (not shown).
(2) Second processing example

A second processing example is a process of storing the history of objects recognized in the past and acquiring past objects corresponding to the current object based on changes in the self-position. In the second processing example, the amount of position change (velocity) of the object can be used for object recognition. In addition, in the second processing example, previously recognized attributes can also be used. Note that the self-position is the position of the observation device or the moving body.

In the second processing example, the position (in the reference coordinate system) where each past object can currently exist is estimated from the history of the past position (in the reference coordinate system) of each object and the movement vector. , based on which each object in the past is associated with the current object. A Kalman filter or the like can be used for such position estimation processing. In order to implement this process, it is necessary to store the position of the moving body and the position of each object at least at one or more times in the past.

More specifically, first, the moving object position information acquiring unit 132 performs the above-described processing to determine the amount of movement of the feature point (for example, (x _t −x _t−1 , y _t −y _t−1 , z _t − z _t−1 )) Calculate. Then, the mobile body position information acquiring unit 132 acquires the mobile body position information (eg, (x _t-1 , y _t-1 , z _{t- 1} ))) and the amount of movement, the position information of the mobile object at the current time point (t) (for example, (x _t , y _t , z _t )) is acquired.

Next, the processing unit 13 uses a set of 3D points possessed by one or more pieces of original information at the present time to recognize an object by the above-described object recognition processing. Next, the processing unit 13 acquires, for example, one or more feature amounts relating to the shape of each object from a set of 3D points of each object. Next, the processing unit 13 acquires, for example, the current attribute value of the object (for example, the type of object) from one or more feature amounts.

Next, the processing unit 13 corrects the object position information of the current object using the mobile body position information in order to associate the object at one or more past points in time with the current object. Next, the current position of each object in the past is estimated from the history of the amount of movement in the past. It should be noted that such processing includes processing using a Kalman filter or the like. Next, the attribute value of the current object and the attribute value of the past object are compared to establish a correspondence between the current object and the past object. The current object and the past object to be associated have the same or similar attribute values, and the difference between the coordinate value corresponding to the current position and the coordinate value corresponding to the predicted position is within a predetermined range. It is an object that stays in the difference between Also, when the object at one or more points in the past and the object at the present point in time can be associated with each other, one or more pieces of object position information of the object at the present point in time are acquired.

Then, for example, the processing unit 13 stores one or more pieces of object position information of each object in a buffer (not shown) in association with the moving body position information acquired from the same original information. It is preferable that the processing unit 13 also store the attribute values of each object together in a buffer (not shown).
(3) Third processing example

A third processing example is a method of storing a set of past 3D points, comparing it with a set of 3D points reflecting changes in the self-position, and detecting a set of 3D points divided into moving and non-moving objects. be. In the third processing example, the same effect as the differential processing during fixed-point observation is obtained, so even in the case of moving observation, it is possible to acquire the object point group more robustly. It should be noted that the "moving object" here means "an object whose position (in the reference coordinate system) changes partially or wholly". An object whose position changes partly is, for example, a human being whose only hands are moving. More specifically, first, the moving object position information acquiring unit 132 performs the above-described processing to determine the amount of movement of the feature point (for example, (x _t −x _t−1 , y _t −y _t−1 , z _t − z _t−1 )) Calculate. Then, the mobile body position information acquiring unit 132 acquires the mobile body position information (eg, (x _t-1 , y _t-1 , z _{t- 1} ))) and the amount of movement, the position information of the mobile object at the current time point (t) (for example, (x _t , y _t , z _t )) is acquired.

Next, the processing unit 13 moves the current set of 3D points to synchronize the current set of 3D points with the set of 3D points at one or more past points in time (for example, the previous point in time). Correct using body position information.

Next, the processing unit 13 performs object recognition processing on the current set of 3D points.

Specifically, the processing unit 13 determines whether each current 3D point belongs to a moving object by considering whether it exists at the same position as the past 3D points, object recognition processing. Here, taking into consideration means that, for example, when specifying a set of 3D points forming an object based on the distance between each of the plurality of 3D points, both of the plurality of 3D points exist at the same positions as the past 3D points. It refers to weighting the distance depending on whether or not

Next, the processing unit 13 obtains one or more object position information and/or attribute values of each object obtained from a set of 3D points at a past time (for example, immediately before), and the current 3D point 1 or 2 or more object position information and/or attribute values of each object for a set of , and each object obtained from the set of 3D points at the past time and each object for the set of 3D points at the current time to decide what to do.

Note that an object that was moving or non-moving in the past may now be determined to be non-moving or moving, unlike past determinations. For example, a person who was standing still starts walking, or a moving car is parked. It is also assumed that flag information indicating whether an object is moving or non-moving is a kind of attribute value of the object. Here, the objects may be associated in consideration of the amount of movement of the objects in the past as described in (2).

Then, for example, the processing unit 13 stores one or more pieces of object position information of each object in a buffer (not shown) in association with the moving body position information acquired from the same original information. It is preferable that the processing unit 13 also store the attribute values of each object together in a buffer (not shown). Note that the object position information may be the same as the absolute position information.

For example, the processing unit 13 uses the transmitting means position information and transmitting means direction information paired with the laser identifiers of the two or more pieces of laser information acquired by the original information acquiring unit 131, and the return time of the laser information. , acquiring object position information of an object for each laser information, specifying one or more objects using the acquired two or more object position information, and determining each of the one or more objects and the two or more object position information of each object. correspond.

It is preferable that the processing unit 13 identifies one or more objects at each of two or more points in time, and associates each of the one or more objects with two or more pieces of object position information of each object.

When the original information is a pair of a laser identifier and a return time, the processing unit 13 acquires object position information indicating a relative position within the three-dimensional space with respect to the moving body position information from the original information. For example, the contour of the object is extracted from two or more pieces of object position information (or absolute position information), a feature amount corresponding to the contour is obtained, and the object identifier is obtained using the feature amount. Then, two or more pieces of object position information forming the outline of the object identified by the object identifier and the object identifier are paired and held. It goes without saying that the processing unit 13 may hold object position information inside the object in addition to two or more pieces of object position information forming the outline of the object, paired with the object identifier.

Also, when the original information is a range image, an object in space is recognized from the range image and an object identifier is acquired. A technique for recognizing an object from an image is a known technique. The processing unit 13 can use machine learning technology such as deep learning, pattern matching technology with object information, and the like. Then, the processing unit 13 holds the object identifier and two or more pieces of object position information forming the recognized object as a pair.

The place where the processing unit 13 holds the pair of object identifiers and two or more pieces of object position information may be the storage unit 11 or a buffer (not shown).

The output diagram construction unit 140 uses the moving object position information and two or more absolute position information of one or more objects to create a three-dimensional image that can recognize the relative positional relationship between the moving object and one or more objects. Constructs an output diagram that is a diagram.

Here, the output drawing is a three-dimensional drawing in which the relative positional relationship between the moving object and one or more objects can be recognized. A specific example of the output diagram will be described later. Note that the moving object position information and the absolute position information used by the output map configuration unit 140 are information obtained from the same one or more original information groups. Also, it can be said that two or more pieces of absolute position information of one or more objects are recognition results of one or more objects.

The output diagram constructing unit 140 constructs an output diagram including one or more objects extracted by the object extracting unit 134 at two or more points in time.

The output diagram construction unit 140 generates an output diagram that is a diagram that also includes trajectory information indicating the movement trajectory of the moving object configured by the moving object position information at each of two or more points in time acquired by the moving object position information acquiring unit 132. It is preferred to configure In addition, it is preferable that the output diagram construction unit 140 constructs an output diagram that also includes an object identifier.

The output unit 14 outputs the object identifier acquired by the object recognition unit 139 and the absolute position information of the same object.

Here, output means display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and transmission to another processing device or other program. This is a concept that includes delivery of processing results.

The diagram output unit 141 outputs the output diagram configured by the output diagram configuration unit 140 .

The object identifier output unit 142 outputs the object identifier acquired by the object recognition unit 139 to the position corresponding to the absolute position information of the same object. It should be noted that the position corresponding to the absolute position information is broadly understood as long as the object identifier is output in such a manner as to correspond to the object on the output map.

The storage unit 11, the original information storage unit 111, the learning device storage unit 112, and the correspondence table storage unit 113 are preferably non-volatile recording media, but can also be realized with volatile recording media.

It does not matter how the information is stored in the storage unit 11 or the like. For example, information may be stored in the storage unit 11 or the like via a recording medium, or information transmitted via a communication line or the like may be stored in the storage unit 11 or the like. Alternatively, information input via an input device may be stored in the storage unit 11 or the like.

The reception unit 12 can be realized by a device driver for input means such as a touch panel or a keyboard, control software for a menu screen, or the like.

Processing unit 13, original information acquisition unit 131, mobile position information acquisition unit 132, original information storage unit 133, object extraction unit 134, feature amount acquisition unit 135, absolute position information acquisition unit 136, same object recognition unit 137, feature amount The synthesizing unit 138, the object recognition unit 139, and the output diagram construction unit 140 can usually be implemented by an MPU, memory, or the like. The processing procedure of the processing unit 13 and the like is normally realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The output unit 14, the diagram output unit 141, and the object identifier output unit 142 may or may not include output devices such as displays and speakers. The output unit 14 and the like can be realized by output device driver software, or by output device driver software and an output device.

Next, the operation of the information processing device 1 will be described using the flowchart of FIG.

(Step S201) The reception unit 12 determines whether or not an operation start instruction has been received. If the operation start instruction has been received, the process goes to step S202, and if the operation start instruction has not been received, the process goes to step S208.

(Step S202) The processing unit 13 substitutes 1 for the counter i.

(Step S203) The original information acquisition unit 131 acquires the original information group at the i-th time point. An example of original information acquisition processing for acquiring such an original information group will be described with reference to the flowchart of FIG.

(Step S204) The mobile body position information acquisition unit 132 acquires the mobile body position information at the i-th time point. An example of mobile body position information acquisition processing will be described with reference to the flowchart of FIG. note that,

(Step S205) The original information storage unit 133 associates the original information group acquired in step S203 with the moving body position information acquired in step S204, and stores them in the original information storage unit 111. FIG.

(Step S206) The reception unit 12 determines whether or not an operation end instruction has been received. If the operation end instruction is accepted, the process returns to step S201, and if the operation end instruction is not accepted, the process goes to step S207.

(Step S207) The processing unit 13 increments the counter i by 1. Return to step S203.

(Step S208) The processing unit 13 determines whether or not it is time to recognize the object. If it is the timing for object recognition, the process goes to step S209, and if it is not the time for object recognition, the process goes to step S214. Note that the timing of object recognition is, for example, when an instruction is received from the user, or when a predetermined time has come. However, the timing is irrelevant.

(Step S209) The processing unit 13 substitutes 1 for the counter i.

(Step S<b>210 ) The processing unit 13 determines whether or not the original information group at the i-th time point exists in the original information storage unit 111 . If the original information group at the i-th time point exists, go to step S211; if not, go to step S213.

(Step S211) The object extraction unit 134 extracts one or more objects using the original information group at the i-th time point. Such object extraction processing will be described with reference to the flowchart of FIG.

(Step S212) The processing unit 13 increments the counter i by 1. Return to step S210.

(Step S213) The processing unit 13 performs object recognition processing. Return to step S201. Note that such object recognition processing will be described with reference to the flowchart of FIG.

(Step S214) The accepting unit 12 determines whether or not an output instruction has been accepted. If the output instruction is accepted, the process goes to step S215, and if the output instruction is not accepted, the process returns to step S201.

(Step S215) The output diagram construction unit 140 performs processing for constructing an output diagram. The output diagram construction processing will be described with reference to the flowchart of FIG.

(Step S216) The diagram output unit 141 outputs the output diagram configured in step S215. Return to step S201.

Note that in the flowchart of FIG. 2, the process ends when the power is turned off or when the process ends.

Next, an example of the original information acquisition process in step S203 will be described using the flowchart of FIG.

(Step S301) The original information acquisition unit 131 substitutes 1 for the counter i.

(Step S302) The original information acquisition unit 131 determines whether or not the i-th laser transmitting means exists. If the i-th laser transmission means exists, go to step S303, and if the i-th laser transmission means does not exist, go to step S307.

(Step S303) The original information acquisition unit 131 acquires the laser identifier of the i-th laser transmission means. Note that the laser identifier is normally stored in the storage unit 11 .

(Step S304) The original information acquisition unit 131 acquires the return time of the laser transmitted by the i-th laser transmission means. It should be noted that such processing is a known technology of LIDAR.

(Step S305) The original information acquisition unit 131 configures original information having the laser identifier acquired in step S303 and the return time acquired in step S304. Note that the original information acquisition unit 131 may calculate three-dimensional coordinate values from the laser identifier and the return time, and configure the original information having the three-dimensional coordinate values. In such a case, the original information acquisition unit 131 acquires the distance to the origin from the return time, acquires the angle to the origin corresponding to the laser identifier, and calculates the three-dimensional coordinate value to the origin from the distance and the angle. Such treatments are known in the art. Also, the angle with respect to the origin is stored in the storage unit 11 in association with the laser identifier.

(Step S306) The original information acquisition unit 131 increments the counter i by 1. Return to step S302.

(Step S307) The original information acquisition unit 131 constructs the original information group at this time and temporarily stores it in a buffer (not shown). Return to upper process. Note that the original information group is a set of two or more pieces of original information configured in step S305.

In the flowchart of FIG. 3, the original information group at this time may be, for example, a set of original information for 360 degrees (one round) around the moving object at one time, or a set of original information for 180 degrees (front). It may be a set of original information of minutes. The range (angle) from the observation device of the original information group to be acquired does not matter.

In addition, in the flowchart of FIG. 3, the original information acquisition unit 131 may acquire one or more distance images around the moving object.

Next, an example of the mobile body position information acquisition processing in step S203 will be described using the flowchart of FIG.

(Step S401) The moving body position information acquisition unit 132 acquires from a set of coordinate values in a three-dimensional space possessed by one or more pieces of original information acquired by the original information acquisition unit 131, or from one or more pieces of each piece of original information. One or more feature points are acquired from a set of coordinate values in the three-dimensional space and temporarily stored in a buffer (not shown). Note that the feature points are 3D coordinate values.

(Step S402) The mobile body position information acquisition unit 132 determines whether or not one or more feature points at the immediately preceding point are stored in a buffer (not shown). If one or more feature points are stored, go to step S403, otherwise go to step S408.

(Step S403) The moving body position information acquisition unit 132 reads one or more feature points at the immediately preceding point from a buffer (not shown).

(Step S404) The moving body position information acquiring unit 132 acquires feature points corresponding to one or more feature points among the one or more feature points acquired in step S401. determine the point.

(Step S405) The mobile body position information acquisition unit 132 calculates the movement amount, which is the difference between the two corresponding feature points. Note that the corresponding two feature points are the current feature point acquired in step S401 and the previous feature point. Corresponding means, for example, that the shapes formed by a set of peripheral points are similar and the distance remains within a certain range. Also, the feature points used for calculating the amount of movement are the feature points of the non-moving object. Further, the moving body position information acquiring unit 132 may calculate differences between sets of two or more feature points, and use the largest difference as the amount of movement. Note that a set of feature points is two corresponding feature points.

(Step S406) The moving body position information acquisition unit 132 reads the immediately preceding moving body position information from a buffer (not shown).

(Step S407) The mobile body position information acquisition unit 132 calculates current mobile body position information from the mobile body position information read out in step S406 and the amount of movement acquired in step S405. Return to upper process.

(Step S408) The moving body position information acquisition unit 132 associates the origin with one or more feature points acquired in step S401. Note that the origin is the mobile body position information at the start. Return to upper process.

Next, the object extraction processing in step S211 will be described using the flowchart of FIG.

(Step S501) The object extraction unit 134 substitutes 1 for the counter j.

(Step S502) The object extraction unit 134 determines whether or not the j-th original information exists in the original information group of interest. If the j-th original information exists, go to step S503, and if the j-th original information does not exist, go to step S506.

(Step S503) The object extraction unit 134 acquires absolute position information using the j-th original information and the moving body position information corresponding to the original information group of interest. This absolute position information is the absolute position information corresponding to the j-th original information.

(Step S504) The object extracting unit 134 stores the absolute position information acquired in step S503 in a buffer in association with the original information group of interest, the j-th original information, or the moving body position information. Note that the object extraction unit 134 may store the absolute position information in the storage unit 11 or the like.

(Step S505) The object extraction unit 134 increments the counter j by 1. Return to step S502.

(Step S506) The object extraction unit 134 substitutes 1 for the counter k.

(Step S507) The object extraction unit 134 determines whether the k-th absolute position information exists in the absolute position information accumulated in step S504. If the k-th absolute position information exists, go to step S508, and if the k-th absolute position information does not exist, return to the higher-level processing. Note that the k-th absolute position information is absolute position information that does not constitute the detected object.

(Step S508) The object extraction unit 134 acquires the k-th absolute position information from among the absolute position information accumulated in step S504.

(Step S509) The object extraction unit 134 extracts one or more ((N-1)) absolute position information constituting one object together with the k-th absolute position information from among the absolute position information accumulated in step S504. Get from Then, the object extraction unit 134 acquires an absolute position information group including the k-th absolute position information and one or more acquired absolute position information. Note that the object extraction unit 134 acquires absolute position information indicating a point whose distance from a point indicated by the k-th absolute position information or any other absolute position information constituting one object is within a threshold value or smaller than a threshold value. do.

(Step S510) The object extraction unit 134 detects that any of the original information corresponding to any of the absolute position information constituting the absolute position information group acquired in step S509 is the original information of the end position in the original information group. (that is, it is determined whether any point of the object is the end point of the original information group). If the edge point exists, go to step S511, and if the edge point does not exist, go to step S512.

(Step S511) The object extraction unit 134 adds a flag indicating "portion" in association with the absolute position information group forming one object. The flag indicating "part" is a flag indicating that the object in the original information group is a part of the one object, not the whole.

(Step S512) The object extraction unit 134 adds a flag indicating "whole" in association with the absolute position information group forming one object. Note that the flag indicating "whole" is a flag indicating that the object in the original information group is the entire object.

(Step S513) The feature amount acquisition unit 135 acquires one or more feature amounts using the absolute position information group of one object, and stores them in association with the absolute position information group.

(Step S514) The object extraction unit 134 adds N to the counter k. Return to step S507.

By the processing of the flowchart in FIG. 5, two or more absolute position information groups configuring one or more objects are identified in association with the original information group or the absolute position information group at each point in time. Also, a flag indicating "partial" or "whole" is accumulated in association with each object. Also, one or more feature amounts are accumulated in association with each object.

Next, the object recognition processing in step S213 will be described using the flowchart of FIG.

(Step S601) The processing unit 13 substitutes 1 for the counter i.

(Step S602) The processing unit 13 substitutes 1 for the counter j.

(Step S603) The same object recognition unit 137 determines whether or not the i-th unprocessed object exists at the j-th time. If the i-th unprocessed object exists, go to step S604, and if the i-th unprocessed object does not exist, go to step S612. An unprocessed object is an object that has not been detected in step S604 or step S606.

(Step S604) The identical object recognition unit 137 acquires the absolute position information group and feature amount corresponding to the i-th unprocessed object.

(Step S605) The identical object recognition unit 137 substitutes j+1 for the counter k.

(Step S606) The same object recognition unit 137 determines whether a group of absolute position information identical or similar to the group of absolute position information acquired in step S604 exists in the group of absolute position information of the object at the k-th time point. to judge. If there is an approximate absolute position information group, go to step S607, and if there is no approximate absolute position information group, go to step S609. Two similar absolute position information groups are absolute position information groups specifying the same object. The approximation of two groups of absolute position information means that the two groups of absolute position information are point groups forming the same object. That is, the area overlaps more than a threshold value or more than the threshold value, and the sum of the absolute values of the differences of the corresponding absolute position information constituting each absolute position information group is less than the threshold value or less than the threshold value.

(Step S607) The identical object recognition unit 137 acquires the absolute position information group determined to be approximate in step S606. Also, the feature amount synthesizing unit 138 acquires one or more feature amounts paired with the absolute position information group.

(Step S608) The identical object recognition unit 137 increments the counter k by 1. Return to step S606.

(Step S609) The feature amount synthesizing unit 138 associates the two or more feature amounts obtained from the absolute position information group of the same object, the two or more feature amounts obtained in step S607, and stores them in a buffer (not shown). Temporarily accumulate.

(Step S610) The object recognition unit 139 acquires an object identifier using the two or more feature amounts acquired in step S609, pairs it with one or more absolute position information groups of the object, and stores it. Here, the object recognition unit 139 acquires object identifiers using, for example, a learning device or a correspondence table.

(Step S611) The processing unit 13 increments the counter i by 1. Return to step S603.

(Step S612) The processing unit 13 increments the counter j by 1.

(Step S613) The processing unit 13 determines whether or not the j-th time point exists. If the j-th time point exists, go to step S614, and if the j-th time point does not exist, return to the upper process.

(Step S614) The processing unit 13 substitutes 1 for the counter i. Return to step S603.

Next, the output diagram construction processing in step S215 will be described with reference to the flowchart of FIG.

(Step S701) The output diagram configuration unit 140 acquires a time point identifier included in the accepted output instruction.

(Step S702) The output diagram configuration unit 140 acquires the absolute position information group paired with the time point identifier.

(Step S703) The output diagram construction unit 140 substitutes 1 for the counter i.

(Step S704) The output diagram construction unit 140 determines whether or not there is an i-th object specified by the absolute position information in the absolute position information group acquired in step S702. If it exists, go to step S705; otherwise, return to the upper process.

(Step S705) The output diagram construction unit 140 acquires the absolute position information group corresponding to the i-th object.

(Step S706) The output diagram construction unit 140 draws the i-th object using the absolute position information group acquired in step S705. It is preferable that the output diagram construction unit 140 renders the object by changing the color of the points of the absolute position information constituting the absolute position information group depending on the object. Also, drawing an object means, for example, arranging points of absolute position information forming a group of absolute position information in a three-dimensional space.

(Step S707) The output diagram construction unit 140 acquires the object identifier of the i-th object.

(Step S708) The output diagram construction unit 140 arranges the object identifier obtained in step S707 in the area corresponding to the position indicated by the absolute position information corresponding to the i-th object.

(Step S709) The output diagram construction unit 140 increments the counter i by 1. Return to step S704.

In the flowchart of FIG. 7, the output map configuration unit 140 may configure the output map by arranging each piece of absolute position information forming the absolute position information group acquired in step S702 in a three-dimensional space.

A specific operation of the information processing apparatus 1 according to the present embodiment will be described below. A conceptual diagram of the information processing apparatus 1 is shown in FIG. In FIG. 8, 801 is the information processing apparatus 1 . 802 is a LIDAR and has a number of laser emitting means. 802 is the observation device described above. The information processing device 1 is, for example, a mobile object such as a robot, an automobile, or an AGV (automated guided vehicle).

It is also assumed that the observation device 802 is mounted on a mobile object and moves through the city. Assume that two or more original information groups at each point in time are accumulated in the original information storage unit 111 while the observation device 802 is moving. For example, while the observation device 802 is mounted on a mobile body and moves through the city, two groups of original information at observation timings "T1" and "T2" are generated as shown in FIG. is stored in the original information storage unit 111 . At the observation timing "T1", the entire utility pole object 901 is observed, and at the observation timing "T2", only a portion of the utility pole object 902 is observed.

It is also assumed that the correspondence table shown in FIG. 10 is managed in the correspondence table storage unit 113 . The correspondence table has two or more pieces of correspondence information. Correspondence information has an "object identifier" and a "feature amount". An "object identifier" here is the type of object or the name of the object. "Feature amount" includes size information and shape information here. "Size information" is information about the range of width and the range of height. Also, "shape information" includes surface shape and three-dimensional shape here. Also, the "surface profile" is smoothness here. Further, the "three-dimensional shape" may be the name of the shape or an image showing the shape. Also, the three-dimensional shape represents, for example, 3D model data, but may be the 3D model data itself, and its data structure does not matter.

Assume that the user inputs an object recognition instruction to the information processing apparatus 1 in such a situation. Then, the receiving unit 12 receives an instruction for object recognition. Then, the processing unit 13 determines that it is time to recognize the object.

Next, the object extraction unit 134 extracts the utility pole 901 (whole) from the original information group at the time point "T1" by the object extraction process described above. Further, the object extraction unit 134 extracts the utility pole (portion) 902 from the original information group at the time point "T2" by the object extraction process described above. It is also assumed that the feature amount acquisition unit 135 acquires size information (width: Xa, height: Ya) using the absolute position information group of the utility pole 901 through the above-described processing. Furthermore, it is assumed that the feature amount acquisition unit 135 acquires the surface shape (smoothness: Sa) using the absolute position information group of the utility pole 902 through the above-described processing. The smoothness is calculated, for example, as the size of the error between the surface forming the three-dimensional shape described above and the actual point cloud.

Next, the same object recognition unit 137 recognizes that the utility pole 901 and the utility pole 902 are the same object by part of the object recognition processing described above.

Then, the feature amount synthesizing unit 138 acquires the size information (width: Xa, height: Ya) of the utility pole 901 and the surface shape (smoothness: Sa) of the utility pole 902 . It is assumed that "Sa" is equal to or less than " _S2 ".

Next, the object recognition unit 139 searches the correspondence table of FIG. 10 for correspondence information that matches the size information (width: Xa, height: Ya) and surface shape (smoothness: Sa). That is, the object recognition unit 139 detects the record of "ID=2". Next, the object recognition unit 139 acquires the object identifier “telephone pole” of the record of “ID=2”. Then, the object recognition unit 139 stores the absolute position information group of the object in association with the object identifier “telephone pole”.

As described above, according to the present embodiment, an object can be easily recognized using two or more original information groups, which are a set of three-dimensional coordinates of a specific portion of an object existing around a moving object.

FIG. 11 shows an example of an output diagram in this embodiment. It is a diagram constructed by an output diagram construction unit 140 and output by a diagram output unit 141. FIG. In FIG. 11, the object identifier is not arranged, but it is preferred that the object identifier is arranged. Also, the self-position in FIG. 11 is the position indicated by the mobile body position information at that time.

Furthermore, the processing in this embodiment may be realized by software. Then, this software may be distributed by software download or the like. Also, this software may be recorded on a recording medium such as a CD-ROM and distributed. Note that this also applies to other embodiments in this specification. The software that implements the information processing apparatus according to the present embodiment is the following program. In other words, this program is, for example, information that is the basis for specifying the position in the three-dimensional space of an object that exists around a moving object, and two or more elements that are a set of three-dimensional coordinates of a specific part of the object. A computer capable of accessing an original information storage unit in which a group of original information, which is information, is stored in association with mobile body position information indicating the position of the mobile body at each of two or more points in time, at each of said two or more points in time an object extracting unit for extracting one or more objects using a group of associated original information; and one or more extracted by the object extracting unit using the group of original information associated with each of the two or more points in time. and the object extraction unit using the original information group associated with each of the two or more points of time and the moving body position information of each of the two or more points of time. an absolute position information acquisition unit that acquires absolute position information about the absolute position of each of the one or more objects extracted by the absolute position information acquisition unit, and the absolute position of each of the one or more objects at each of the two or more points in time acquired by the absolute position information acquisition unit using the information, a same object recognition unit for matching the same object at two or more different points in time, acquiring two or more feature amounts of the same object correlated by the same object recognition unit; and obtaining an object identifier that identifies the same object using the feature amount set obtained by the feature amount synthesizing unit. and an output unit that outputs an object identifier acquired by the object recognition unit and the absolute position information of the same object.

FIG. 12 is an external view of a computer system 300 that implements the information processing apparatus 1 by executing the program according to this embodiment.

Each device of this embodiment can be realized by computer hardware and a computer program executed thereon. In FIG. 12, computer system 300 comprises computer 301 including disk drive 305 , keyboard 302 , mouse 303 and display 304 . The entire system including the keyboard 302, mouse 303, and display 304 may be called a computer.

FIG. 13 is a diagram showing an example of the internal configuration of the computer system 300. As shown in FIG. In FIG. 13, the computer 301 includes an MPU 3013, a ROM 3015 for storing programs such as a boot-up program, and a RAM 3016 connected to the MPU 3013 and the like for temporarily storing application program instructions and providing a temporary storage space. , a hard disk 3017 for storing application programs, system programs, and data, and a bus 3014 for interconnecting the MPU 3013, ROM 3015, and the like.

A program that causes the computer system 300 to execute the functions of the information processing apparatus 1 may be stored in a disk 3101 such as a CD-ROM, inserted into the disk drive 3012 , and transferred to the hard disk 3017 . Alternatively, the program may be transmitted to computer 301 via a network and stored on hard disk 3017 . Programs are loaded into RAM 3016 during execution. Note that the program may be loaded directly from the hard disk 3016 or from the network. Also, the program may be read into the computer system 300 via another removable recording medium (eg, DVD, memory card, etc.) instead of the hard disk 3016 .

The program does not necessarily include an operating system (OS) or a third-party program that causes the information processing apparatus 1 to execute the functions of the computer details 901 . A program may contain only those portions of instructions that call the appropriate functions or modules in a controlled manner to produce the desired result. How the computer system 300 operates is well known and will not be described in detail. Note that the above is an example, and the hardware configuration of the computer that implements the information processing apparatus 1 does not matter.

Also, the number of computers that execute the above programs may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

Further, in each of the above embodiments, each process may be implemented by centralized processing by a single device, or may be implemented by distributed processing by a plurality of devices.

It goes without saying that the present invention is not limited to the above-described embodiments, and that various modifications are possible and are also included within the scope of the present invention.

As described above, the information processing apparatus according to the present invention can recognize an object using two or more original information groups that are a set of three-dimensional coordinates of a specific portion of an object existing around a moving object. It has an effect that it can be easily performed, and is useful as an information processing device or the like.

1 information processing device 11 storage unit 12 reception unit 13 processing unit 14 output unit 111 original information storage unit 112 learning device storage unit 113 correspondence table storage unit 131 original information acquisition unit 132 moving body position information acquisition unit 133 original information storage unit 134 object Extraction unit 135 Feature amount acquisition unit 136 Absolute position information acquisition unit 137 Same object recognition unit 138 Feature amount synthesis unit 139 Object recognition unit 140 Output map configuration unit 141 Diagram output unit 142 Object identifier output unit

Claims (10)

an original information group consisting of two or more pieces of original information that are information that is the basis for specifying the position in the three-dimensional space of an object that exists around a moving object and that is a set of three-dimensional coordinates of a specific part of the object; an original information storage unit stored in association with two or more moving body position information indicating the position of the moving body at each time point;
an object extraction unit that extracts one or more objects using the original information group associated with each of the two or more time points;
a feature amount acquisition unit that acquires a feature amount of each of the one or more objects extracted by the object extraction unit using the group of original information associated with the two or more points in time;
a same object recognition unit that takes correspondence of the same object at two or more different points in time;
a feature amount synthesizing unit that acquires two or more feature amounts of the same object that the same object recognition unit has taken correspondence with, and acquires a feature amount set of the same object using the two or more feature amounts;
an object recognition unit that acquires an object identifier that identifies the same object using the feature amount set acquired by the feature amount synthesizing unit;
an output unit that outputs the object identifier acquired by the object recognition unit ,
The feature quantity acquisition unit is
A group of original information at a first point in time, which is an original information group at a second point in time, and which is an original information group at a second point in time, using an original information group that can identify the entire one object. obtaining a partial feature amount of the one object using a group of original information that can identify a part of the object;
The feature quantity synthesizing unit
The same object recognition unit acquires the overall feature amount and the partial feature amount of the same object for which correspondence is obtained, and acquires the feature amount set of the same object using the overall feature amount and the partial feature amount. , information processing equipment. 2. The information processing apparatus according to claim 1 , wherein the overall feature amount is size information about the size of the object or the aspect ratio of the object, and the partial feature amount is shape information about the shape of the object. One or more absolute positions of the one or more objects extracted by the object extracting unit using a group of original information associated with two or more points in time and moving body position information at two or more points in time. an absolute position information acquisition unit that acquires absolute position information;
FIG. 11 is a three-dimensional diagram in which a relative positional relationship between the moving body and the one or more objects can be recognized using the moving body position information and two or more absolute position information of the one or more objects. and an output diagram configuration unit that configures the output diagram,
The output unit
a diagram output unit that outputs the output diagram;
3. The information processing apparatus according to claim 1 , further comprising an object identifier output unit that outputs the object identifier acquired by the object recognition unit to a position corresponding to the absolute position information of the same object. an original information group consisting of two or more pieces of original information that are information that is the basis for specifying the position in the three-dimensional space of an object that exists around a moving object and that is a set of three-dimensional coordinates of a specific part of the object; an original information storage unit stored in association with two or more moving body position information indicating the position of the moving body at each time point;
an object extraction unit that extracts one or more objects using the original information group associated with each of the two or more time points;
a feature amount acquisition unit that acquires a feature amount of each of the one or more objects extracted by the object extraction unit using the group of original information associated with the two or more points in time;
a same object recognition unit that takes correspondence of the same object at two or more different points in time;
a feature amount synthesizing unit that acquires two or more feature amounts of the same object that the same object recognition unit has taken correspondence with, and acquires a feature amount set of the same object using the two or more feature amounts;
an object recognition unit that acquires an object identifier that identifies the same object using the feature amount set acquired by the feature amount synthesis unit;
One or more absolute positions of the one or more objects extracted by the object extracting unit using a group of original information associated with two or more points in time and moving body position information at two or more points in time. an absolute position information acquisition unit that acquires absolute position information;
FIG. 10 is a three-dimensional diagram in which a relative positional relationship between the moving body and the one or more objects can be recognized using the moving body position information and two or more absolute position information of the one or more objects. an output diagram configuration unit that configures an output diagram;
a diagram output unit that outputs the output diagram;
and an object identifier output unit that outputs the object identifier acquired by the object recognition unit to a position corresponding to the absolute position information of the same object. further comprising a learning device storage unit that stores a learning device acquired by learning processing by a machine learning algorithm using two or more sets of information having an object feature set and an object identifier,
The object recognition unit
The information processing apparatus according to any one of claims 1 to 4, wherein the feature amount set acquired by the feature amount synthesizing unit is applied to the learning device, and an object identifier is acquired by a machine learning algorithm. The object recognition unit
6. The information processing apparatus according to any one of claims 1 to 5, wherein an object identifier for identifying one or more types of objects of a utility pole, a traffic light, or a post on a road is obtained. an original information group consisting of two or more pieces of original information that are information that is the basis for specifying the position in the three-dimensional space of an object that exists around a moving object and that is a set of three-dimensional coordinates of a specific part of the object; An original information storage unit stored in association with moving body position information indicating the position of the moving body at each of two or more points in time; an object extraction unit; a feature amount acquisition unit; a same object recognition unit; An information processing method implemented by a unit, an object recognition unit, and an output unit,
an object extraction step in which the object extraction unit extracts one or more objects using the original information group associated with the two or more points in time;
a feature amount acquisition step in which the feature amount acquisition unit acquires the feature amount of each of the one or more objects extracted in the object extraction step using the group of original information associated with the two or more points in time;
a same object recognition step in which the same object recognition unit takes correspondence with the same object at two or more different points in time;
The feature amount synthesizing unit acquires two or more feature amounts of the same object whose correspondence is taken in the same object recognition step, and uses the two or more feature amounts to generate a feature amount set of the same object. a feature synthesis step to be obtained;
an object recognition step in which the object recognition unit acquires an object identifier that identifies the same object using the feature amount set acquired in the feature amount synthesizing step;
an output step in which the output unit outputs the object identifier acquired in the object recognition step ;
In the feature amount acquisition step,
A group of original information at a first point in time, which is an original information group at a second point in time, and which is an original information group at a second point in time, using an original information group that can identify the entire one object. obtaining a partial feature amount of the one object using a group of original information that can identify a part of the object;
In the feature amount synthesizing step,
The same object recognition unit acquires the overall feature amount and the partial feature amount of the same object for which correspondence is obtained, and acquires the feature amount set of the same object using the overall feature amount and the partial feature amount. , information processing methods. an original information group consisting of two or more pieces of original information that are information that is the basis for specifying the position in the three-dimensional space of an object that exists around a moving object and that is a set of three-dimensional coordinates of a specific portion of the object; An original information storage unit stored in association with moving body position information indicating the position of the moving body at each of two or more points in time; an object extraction unit; a feature acquisition unit; a same object recognition unit; An information processing method implemented by a unit, an object recognition unit, an absolute position information acquisition unit, an output diagram configuration unit, a diagram output unit, and an object identifier output unit,
an object extraction step in which the object extraction unit extracts one or more objects using the original information group associated with the two or more points in time;
a feature amount acquisition step in which the feature amount acquisition unit acquires the feature amount of each of the one or more objects extracted in the object extraction step using the group of original information associated with the two or more points in time;
a same object recognition step in which the same object recognition unit takes correspondence with the same object at two or more different points in time;
The feature amount synthesizing unit acquires two or more feature amounts of the same object whose correspondence is taken in the same object recognition step, and uses the two or more feature amounts to generate a feature amount set of the same object. a feature synthesis step to be obtained;
an object recognition step in which the object recognition unit acquires an object identifier that identifies the same object using the feature amount set acquired in the feature amount synthesizing step;
The absolute position information acquiring unit uses a group of original information associated with two or more points in time and moving body position information at two or more points in time to identify the one or more objects extracted by the object extraction unit. an absolute position information acquisition step of acquiring one or more absolute position information relating to the absolute position;
The output diagram construction unit can recognize a relative positional relationship between the moving body and the one or more objects using the moving body position information and two or more absolute position information of the one or more objects. an output diagram construction step for constructing an output diagram that is a three-dimensional diagram;
a diagram output step in which the diagram output unit outputs the output diagram;
and an object identifier output step in which the object identifier output unit outputs the object identifier acquired in the object recognition step to a position corresponding to the absolute position information of the same object. an original information group consisting of two or more pieces of original information that are information that is the basis for specifying the position in the three-dimensional space of an object that exists around a moving object and that is a set of three-dimensional coordinates of a specific part of the object; a computer capable of accessing an original information storage unit stored in association with two or more pieces of mobile body position information indicating the position of the mobile body at each time point;
an object extraction unit that extracts one or more objects using the original information group associated with each of the two or more time points;
a feature amount acquisition unit that acquires a feature amount of each of the one or more objects extracted by the object extraction unit using the group of original information associated with the two or more points in time;
a same object recognition unit that takes correspondence of the same object at two or more different points in time;
a feature amount synthesizing unit that acquires two or more feature amounts of the same object that the same object recognition unit has taken correspondence with, and acquires a feature amount set of the same object using the two or more feature amounts;
an object recognition unit that acquires an object identifier that identifies the same object using the feature amount set acquired by the feature amount synthesizing unit;
A program for functioning as an output unit that outputs the object identifier acquired by the object recognition unit ,
The feature quantity acquisition unit is
A group of original information at a first point in time, which is an original information group at a second point in time, and which is an original information group at a second point in time, using an original information group that can identify the entire one object. obtaining a partial feature amount of the one object using a group of original information that can identify a part of the object;
The feature quantity synthesizing unit
The same object recognition unit acquires the overall feature amount and the partial feature amount of the same object for which correspondence is obtained, and acquires the feature amount set of the same object using the overall feature amount and the partial feature amount. A program for making the computer function as a thing . an original information group consisting of two or more pieces of original information that are information that is the basis for specifying the position in the three-dimensional space of an object that exists around a moving object and that is a set of three-dimensional coordinates of a specific part of the object; a computer capable of accessing an original information storage unit stored in association with two or more pieces of mobile body position information indicating the position of the mobile body at each time point;
an object extraction unit that extracts one or more objects using the original information group associated with each of the two or more time points;
a feature amount acquisition unit that acquires a feature amount of each of the one or more objects extracted by the object extraction unit using the group of original information associated with the two or more points in time;
a same object recognition unit that takes correspondence of the same object at two or more different points in time;
a feature amount synthesizing unit that acquires two or more feature amounts of the same object that the same object recognition unit has taken correspondence with, and acquires a feature amount set of the same object using the two or more feature amounts;
an object recognition unit that acquires an object identifier that identifies the same object using the feature amount set acquired by the feature amount synthesizing unit;
One or more absolute positions of the one or more objects extracted by the object extracting unit using a group of original information associated with two or more points in time and moving body position information at two or more points in time. an absolute position information acquisition unit that acquires absolute position information;
FIG. 11 is a three-dimensional diagram in which a relative positional relationship between the moving body and the one or more objects can be recognized using the moving body position information and two or more absolute position information of the one or more objects. an output diagram configuration unit that configures an output diagram;
a diagram output unit that outputs the output diagram;
A program for functioning as an object identifier output unit that outputs the object identifier acquired by the object recognition unit to a position corresponding to the absolute position information of the same object.

Images