JP7153264B2 - Image analysis system, image analysis method and image analysis program - Google Patents

Description

The present disclosure relates to an image analysis system, an image analysis method, and an image analysis program that perform attribute determination by analyzing image data captured by an imaging device.

2. Description of the Related Art Techniques for determining attributes of elements included in an image by analyzing an image obtained by imaging a target area are known. Here, an attribute refers to a property or feature to which an element included in an image belongs, and determination of an attribute refers to determination of an attribute to which an element belongs from a plurality of predefined attributes. Such attributes include, for example, vegetation such as plants and animals, man-made objects, water, sky, roads and soil.

For example, Patent Literature 1 discloses a damage area automatic extraction system that automatically extracts a damage area of a natural disaster or man-made disaster from captured image data by using such an attribute determination technique. In this system, a damaged area is identified by attribute determination based on the brightness of pixels forming read image data.

Japanese Patent No. 3924569

In Patent Document 1, attribute determination is performed on single image data. Therefore, depending on the state of the target image data, there is a possibility that sufficient attribute determination cannot be made. For example, depending on the lighting conditions at the time of shooting, if halation areas occur in the image data, or if low-contrast areas occur due to insufficient illuminance, it may be difficult to determine attributes in these areas. be. In addition, in the image data captured from a predetermined angle of the target area, some areas become blind spots due to the terrain of the target area and the presence of obstacles (trees, buildings, etc.). It can get difficult.

At least one embodiment of the present invention has been made in view of the above circumstances, and an object thereof is to provide an image analysis system, an image analysis method, and an image analysis program capable of accurately determining attributes over a wide range of a target area. and

(1) In order to solve the above problems, the image analysis system according to at least one embodiment of the present invention
an image data acquisition unit that acquires a plurality of image data obtained by imaging a target area from different angles;
an attribute determination unit that determines an attribute for each pixel of each of the plurality of image data;
a shape information creation unit that creates shape information in the target area by associating each pixel included in the plurality of image data with each other;
an attribute distribution calculation unit that calculates an attribute distribution in the target area by summing up determination results by the attribute determination unit for each point included in the shape information;
Prepare.

According to the configuration (1) above, shape information is created by associating corresponding pixels in a plurality of image data obtained by imaging the target area from different angles. By aggregating the attribute determination results for each image data for each point included in the shape information, even if there is an area where attribute determination is difficult with a single image data, the area can be interpolated. , attribute determination can be performed over a wide range. In addition, by aggregating the attribute determination results for a plurality of image data, it is possible to obtain a highly accurate determination result compared to attribute determination based on single image data.

(2) In some embodiments, in the configuration of (1) above,
The attribute distribution calculation unit calculates the attribute distribution by counting the number of determinations for each type of the attribute for each point of the shape information and adopting the attribute of the type with the maximum number of determinations.

According to the above configuration (2), when summarizing the attribute determination results in each image data for each point included in the shape information, the attribute with the largest number of times of determination at each point is adopted. As a result, a highly reliable attribute distribution can be obtained based on the determination result of the attribute with the highest determination frequency.

(3) In some embodiments, in the configuration of (1) above,
The attribute determination and classification unit obtains a score for each attribute type,
The attribute distribution calculation unit calculates the total value of the scores for each type of the attribute for each point of the shape information, and adopts the attribute of the type having the maximum total value, thereby calculating the attribute distribution. calculate.

With configuration (3) above, attribute determination is performed by obtaining a score for each attribute type (for example, a probability value for each attribute type) for each pixel included in image data. Then, at the time of aggregation, for each point included in the shape information, the total value of the scores obtained from each image data is calculated, and finally the attribute distribution is obtained by adopting the type of attribute with the maximum total value. .

(4) In some embodiments, in the configuration of (2) or (3) above,
A distance calculation unit that calculates a distance from the imaging point for each pixel of each of the plurality of image data,
The attribute distribution calculator sets a weighting factor according to the distance for each point included in the shape information.

According to the configuration (3) above, when summing the attribute determination results for each point of the shape information when calculating the attribute distribution, a weighting coefficient corresponding to the distance between the imaging point and each pixel in each image data is considered. The weighting factor is variably set according to the distance, considering that the determination results for pixels closer to the imaging point are more reliable than the determination results for pixels farther from the imaging point, for example. . A more reliable attribute distribution can be obtained by aggregating in consideration of such weighting coefficients.

(5) In some embodiments, in any one configuration of (1) to (4) above,
a peculiar point identifying unit that identifies a peculiar point in the target area based on the attribute distribution;
Prepare.

With configuration (5) above, it is possible to easily specify the presence or absence and range of a singular point in the target area based on the attribute distribution calculated by the above configuration.

(6) In some embodiments, in the configuration of (5) above,
An output device capable of superimposing and displaying the attribute distribution on a digital map is provided.

According to the above configuration (6), the attribute distribution calculated by the above configuration is displayed superimposed on the digital map, so that the operator can easily grasp the attribute distribution situation.

(7) In some embodiments, in the configuration of (6) above,
The output device displays the singular point by distinguishing it from the surrounding area.

With configuration (7) above, it is possible for the operator to easily recognize the existence and range of the peculiar point in the target area.

(8) In some embodiments, in any one configuration of (1) to (7) above,
The shape information is three-dimensional information created based on the plurality of image data.

According to the configuration (8) above, by associating each pixel included in a plurality of image data, the uneven terrain in the target area can be grasped as three-dimensional information.

(9) In some embodiments, in any one configuration of (1) to (8) above,
The attribute distribution calculation unit calculates the attribute distribution in consideration of occlusion obtained based on the shape information.

With configuration (9) above, occlusion (hiding) is obtained from the uneven topography in the target area based on the shape information. By taking such occlusion into account, it is possible to eliminate attribute determinations that are clearly erroneous, and to improve the accuracy of attribute distribution.

(10) In some embodiments, in any one configuration of (1) to (9) above,
a moving body equipped with an imaging device capable of imaging the plurality of images;
a movement control unit for controlling the moving body;
an imaging control unit for controlling the imaging device;
Prepare.

With configuration (10) above, the imaging operation of the imaging device is controlled by the imaging controller while the position of the moving body relative to the target area is changed by the movement controller. This makes it possible to acquire a plurality of image data from different angles with respect to the target area.

(11) In some embodiments, in the configuration of (10) above,
The imaging control unit controls the imaging device so that imaging is performed under a plurality of different imaging conditions at each imaging point,
The image data acquisition unit selectively acquires image data that satisfies a predetermined quality standard from the plurality of image data captured by the imaging device.

With configuration (11) above, by performing imaging under a plurality of different imaging conditions at each imaging point, it is possible to effectively improve the possibility of obtaining high-quality imaging data. The image data acquisition unit selectively acquires image data that satisfies a predetermined quality standard from a plurality of captured image data, thereby further improving the attribute determination accuracy and shortening the processing time.

(12) In some embodiments, in the configuration of (11) above,
The plurality of imaging conditions are set such that at least one imaging parameter including the imaging position/angle of the imaging device, exposure time, focal length, aperture value, or environmental condition is different.

According to the configuration (12) above, at each imaging point, imaging is performed under a plurality of imaging conditions in which at least one of these imaging parameters is different, thereby further improving the attribute determination accuracy and shortening the processing time. be done.

(13) In some embodiments, in the configuration of (12) above,
The plurality of imaging conditions are set such that the difference between the imaging parameters is equal to or greater than a predetermined value.

According to the above configuration (13), in a plurality of imaging conditions, the difference in the imaging parameters included in each imaging condition is set to be equal to or greater than a predetermined value, so that imaging data can be acquired under a wider range of imaging conditions. It is possible to more effectively improve the acquisition probability of high-quality imaging data at each imaging point.

(14) In order to solve the above problems, the image analysis method according to at least one embodiment of the present invention includes:
Acquiring a plurality of image data obtained by imaging the target area from different angles;
determining an attribute for each pixel of each of the plurality of image data;
creating shape information in the target area by associating each pixel included in the plurality of image data with each other;
a step of calculating an attribute distribution in the target area by aggregating the attribute determination results for each point included in the shape information;
Prepare.

According to the method (14) above, shape information is created by associating corresponding pixels in a plurality of image data obtained by imaging the target area from different angles. By aggregating the attribute determination results for each image data for each point included in the shape information, even if there is an area where attribute determination is difficult with a single image data, the area can be interpolated. , attribute determination can be performed over a wide range. In addition, by aggregating the attribute determination results for a plurality of image data, it is possible to obtain a highly accurate determination result compared to attribute determination based on single image data.

(15) In order to solve the above problems, an image analysis program according to at least one embodiment of the present invention
Acquiring a plurality of image data obtained by imaging the target area from different angles;
determining an attribute for each pixel of each of the plurality of image data;
creating shape information in the target area by associating each pixel included in the plurality of image data with each other;
a step of calculating an attribute distribution in the target area by aggregating the attribute determination results for each point included in the shape information;
run on the computer.

The program of (15) above constitutes at least a part of the above-described image analysis system by being executed by a computer, and can preferably implement the above-described image analysis method.

According to at least one embodiment of the present invention, it is possible to provide an image analysis system, an image analysis method, and an image analysis program capable of accurately determining attributes over a wide range of target areas.

1 is a schematic diagram showing the overall configuration of an image analysis system according to at least one embodiment of the present invention; FIG. 2 is a block diagram showing the hardware configuration of the main server of FIG. 1; FIG. 3 is a block diagram functionally showing the internal configuration of a main server according to at least one embodiment of the present invention; FIG. 4 is a flow chart showing steps of an image analysis method according to at least one embodiment of the present invention. 5 is an example of a plurality of image data acquired in step S100 of FIG. 4; It is an example of the attribute determination result corresponding to each pixel in step S101 of FIG. 5 is a schematic diagram conceptually showing calculation of attribute distribution in step S103 of FIG. 4. FIG. 5 is a diagram showing the attribute distribution calculated in step S103 of FIG. 4 together with a comparative example; FIG. 3 is a block diagram functionally showing the internal configuration of a main server according to at least one embodiment of the present invention; FIG. 4 is a flow chart showing each step of an imaging data acquisition method according to at least one embodiment of the present invention.

Several embodiments of the present invention will now be described with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc. of the components described as embodiments or shown in the drawings are not intended to limit the scope of the present invention, and are merely illustrative examples. do not have.

FIG. 1 is a schematic diagram showing the overall configuration of an image analysis system 1 according to at least one embodiment of the invention.
The image analysis system 1 is a system for identifying damaged areas in a target area 2 by image analysis, and includes a mobile body 6 equipped with an imaging device 4 for capturing an image of the target area 2 . The moving body 6 is configured to be movable on the target area 2 so as to be able to change the relative positional relationship of the mounted imaging device 4 with respect to the target area 2 . In this embodiment, as an example of the moving body 6, a UAV (Unmanned Aerial Vehicle) capable of moving by unmanned flight over the target area 2 by remote control is shown.

The imaging device 4 mounted on the moving body 6 is a camera for capturing image data including the target area 2, and is, for example, a CMOS (Complementary Metal Oxide Semiconductor Image Sensor) or a CCD (Charge Coupled Device Image Sensor). Including image sensor. The imaging device 4 is mounted in a fixed posture with respect to the moving body 6 and is configured to be able to capture images of the target area 2 from different angles as the moving body 6 moves over the target area 2 .
Note that the imaging device 4 may be mounted on the moving body 6 such that its attitude is variable.

The image analysis system 1 includes a main server 10 arranged in a base station geographically separated from a mobile body 6 on which an imaging device 4 is mounted. The main server 10 is composed of an arithmetic processing device such as a computer capable of communicating with the imaging device 4 and the mobile body 6 via the communication network 7 .

2 is a block diagram showing the hardware configuration of the main server 10 of FIG. 1. As shown in FIG. The main server 10 includes an arithmetic device 12 , a storage device 14 , a drive device 16 , an input device 18 and an output device 20 .

The computing device 12 is a processor such as a CPU (Central Processing Unit). The computing device 12 is configured to be able to implement the image analysis method according to at least one embodiment of the present invention by executing the computer program 22 stored in the storage device 14 . The storage device 14 may be a non-volatile memory such as ROM (Read Only Memory) or a volatile memory such as RAM (Random Access Memory).

The computer program 22 stored in the storage device 14 is an example of an image analysis program according to at least one embodiment of the invention, and may be installed from the recording medium 24 via the drive device 16 . The recording medium 24 may be a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk, or an electrical recording medium such as a ROM or flash memory. A semiconductor memory that records information in the memory may also be used.

The input device 18 is an interface for inputting various kinds of information necessary for the operation of the main server 10 from the outside. Particularly in this embodiment, the input device 18 includes a receiving section 18 a for receiving various information transmitted from the imaging device 4 and the mobile object 6 at a remote location via the communication network 7 . The input device 18 may also include input means (for example, computer keyboard, mouse, touch panel, etc.) for inputting data by operator's operation.

The output device 20 includes a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OELD), and displays display data generated by the arithmetic device 12 . The display data may be image data or character data. The output device 20 may include at least one of a printing device and a speaker device that outputs audio data.

The output device 20 also includes a transmission unit 20a for transmitting various control signals to the imaging device 4 and the mobile object 6 at remote locations via the communication network 7. FIG. Thereby, the main server 10 is configured to be able to remotely control the imaging device 4 and the moving object 6 .

<First embodiment>
FIG. 3 is a block diagram functionally showing the internal configuration of the main server 10 according to at least one embodiment of the present invention. The main server 10 includes a controller 26 for remotely controlling the imaging device 4 and the mobile body 6 via the communication network 7, and an image analysis device 28 for analyzing images acquired via the communication network 7. include.

The controller 26 is a control device for remotely controlling the imaging device 4 and the moving object 6 by transmitting and receiving control signals to the imaging device 4 and the moving object 6 via the communication network 7 (not shown in FIG. 3). It includes a movement control unit 30 for controlling the moving body 6 and an imaging control unit 32 for controlling the imaging device 4 .

The movement control unit 30 controls the movement state (for example, speed, tilt angle, etc.) of the moving body 6 by transmitting and receiving control signals to the moving body 6 via the communication network 7 (not shown in FIG. 3). Therefore, the position of the moving object 6 in the sky above the target area 2 can be managed. The imaging control unit 32 controls the imaging operation of the imaging device 4 by transmitting and receiving control signals to and from the imaging device 4 via the communication network 7 (not shown in FIG. 3), thereby capturing an image under predetermined imaging conditions. Data creation is configured to occur.

The movement control unit 30 and the imaging control unit 32 that constitute the controller 26 are configured to be synchronously controllable with each other. As a result, image data is created with respect to the target area 2 from different angles by capturing images at predetermined timings with the imaging device 4 while the moving body 6 is moving over the target area 2 .

The image analysis device 28 includes an image data acquisition section 36 , an attribute determination section 38 , a shape information creation section 40 , an attribute distribution calculation section 42 , and a singular point identification section 44 .

The image data acquisition unit 36 acquires image data captured by the imaging device 4 . In the imaging device 4 , image data is created by performing imaging under predetermined imaging conditions in accordance with the control signal from the imaging control section 32 . The image data created by the imaging device 4 is transmitted to the main server 10 via the communication network 7 and acquired by the image data acquisition section 36 .

The attribute determining section 38 determines the attribute of each pixel of the image data acquired by the image data acquiring section 36 . Here, the attributes of a pixel refer to properties or characteristics to which the pixel belongs. Determining an attribute means determining an attribute to which a pixel belongs from an attribute library that defines a plurality of attributes in advance. In this embodiment, "vegetation", "road", "soil", and "unknown (attribute cannot be determined)" are exemplified as attribute types, but other types may be included.

Attribute determination by the attribute determining unit 38 is performed by the following process using, for example, a random forest algorithm. First, the attribute determination unit 38 takes in the image data acquired by the image data acquisition unit 36 and recognizes the image data on a pixel-by-pixel basis. Then, the attribute determination unit 38 specifies a feature amount for each pixel recognized from the image data. Various parameters such as brightness, lightness, hue, and texture can be used as pixel feature amounts. Specification of such a feature amount can be performed using a known feature amount calculation method such as HOG (Histograms of Oriented Gradients), for example. The attribute determination unit 38 has an attribute library that defines a plurality of attributes in advance, and performs attribute determination by comparing each pixel recognized from the image data with the attribute library.

In the following description, a case where attribute determination is performed in units of pixels will be described, but attribute determination may be performed in units of a plurality of pixels. In this case, for example, a patch image made up of a plurality of pixels is created from image data, and attribute determination is performed by comparing the patch image with an attribute library.

The shape information creation unit 40 creates shape information by associating each pixel included in the plurality of images acquired by the image data acquisition unit 36 with each other. The shape information is, for example, the three-dimensional shape of the target area 2 to be imaged, and is calculated by, for example, SfM (Structure from Motion). In SfM, based on a plurality of two-dimensional image data, the imaging point (camera position) of each image data and the three-dimensional shape of the imaging target (target area 2) are arithmetically calculated as point cloud data. .

The attribute distribution calculation unit 42 calculates the attribute distribution in the target area 2 by totalizing the attribute determination results of the attribute determination unit 38 for each point included in the shape information. The attribute determining unit 38 determines the attribute of each pixel in each image as described above. Since such attribute determination is performed for each of the plurality of image data acquired by the image data acquisition unit 36, each point of the shape information created by the shape information creation unit 40 is assigned a plurality of attribute determination results. will be The attribute distribution calculator 42 can specify the most probable attribute at each point by summing up the attribute determination results based on each image data at each point of the shape information.

Note that any statistical method can be used for aggregating the attribute determination results described above. Here, as an example, the attribute distribution calculation unit 42 counts the number of determinations for each type of attribute for each point of the shape information, and finally adopts the attribute of the type with the maximum number of determinations. That is, at each point of the shape information, the attribute of the type with the largest count of the number of determinations is specified. By performing such attribute determination for each point of the shape information, the attribute distribution in the target area 2 is calculated. The distribution of the attribute determination results specified by each point of the shape information in this manner has higher accuracy than the attribute determination result based on single image data.

The peculiar point identification unit 44 identifies peculiar points based on the attribute distribution calculated by the attribute distribution calculation unit 42 . Identification of such a peculiar point is performed based on whether or not the ratio of pixels for which a specific type of attribute determination is made is equal to or greater than a threshold value for an arbitrary area of the attribute distribution. For example, when the ratio of pixels for which the attribute determination of "soil" is made in a predetermined area is equal to or greater than a threshold, the area is eroded by soil, and the affected area is specified as a peculiar point.

Incidentally, the peculiar point may be a point having an attribute different from the originally assumed attribute. In the present embodiment, a case is exemplified in which a spot that originally has the attribute of "road" is identified as a peculiar spot when the attribute of "soil" is determined. In this embodiment, attributes are classified by "road", "soil", "vegetation", and "other". Landslides) and ground subsidence are specified, if it is "vegetation", then landslides are specified, and if there are "buildings", then singular points are specified by collapsed houses.

Next, an image analysis method performed by the image analysis system 1 having the above configuration will be described. FIG. 4 is a flow chart showing steps of an image analysis method according to at least one embodiment of the present invention.

First, the image data acquisition unit 36 acquires a plurality of image data from the imaging device 4 mounted on the mobile body 6 (step S100). The plurality of image data acquired by the image data acquisition unit 36 are created by imaging the target area 2 from different angles by controlling the imaging device 4 and the moving body 6 by the controller 26 . In other words, in step S100, while the controller 26 controls the moving object 6 to move along a predetermined path over the target area 2, the imaging device 4 captures images at predetermined timings, thereby capturing images of the target area from different angles. 2 is acquired.

Here, FIG. 5 is an example of a plurality of image data acquired in step S100 of FIG. FIG. 5 shows a plurality of image data created by imaging a target area 2 in which a single road 2b exists in a vegetation area 2a in which trees exist from different angles.

Subsequently, the attribute determination unit 38 determines the attribute of each pixel of each of the plurality of image data acquired by the image data acquisition unit 36 (step S101). Attribute determination by the attribute determining unit 38 is performed independently for each image acquired in step S100. Here, FIG. 6 is an example of the attribute determination result corresponding to each image in step S101 of FIG. In FIG. 6, the pixels whose attribute determination is "vegetation", "soil", "road" and "unknown" are indicated by hatching with different patterns.

Subsequently, the shape information creation unit 40 creates shape information in the target area 2 by associating each pixel included in the plurality of image data acquired by the image data acquisition unit 36 (step S102). That is, the shape information is constructed by identifying the coordinates in the three-dimensional space for each pixel of a plurality of data and by associating the corresponding pixels with each other. Such shape information is created as three-dimensional point cloud data by SfM, for example, as described above.

Subsequently, the attribute distribution calculation unit 42 calculates the attribute distribution in the target area 2 by totalizing the attribute determination results for each point included in the shape information (step S103). FIG. 7 is a schematic diagram conceptually showing calculation of the attribute distribution in step S103 of FIG. FIGS. 7A to 7D show the attribute determination results corresponding to each image data in step S101 (here, the three images on the top of FIG. 6 and the image on the second left from the top). Attribute determination results for one image are representatively shown in FIGS. 7(a) to (d)). Further, FIG. 7(e) shows the shape information created in step S102. Here, the plurality of attribute determination results shown in FIGS. 7A to 7E are adjusted in advance by being enlarged/reduced/rotated so that the display areas are shared with each other. A point and each point of the shape information are associated with each other on a one-to-one basis. The attribute distribution calculation unit 42 aggregates each of the attribute determination results for each point that is associated with each other in this way, thereby finally specifying the attribute and calculating the attribute distribution (see FIG. 7(f)). Ask for

Further, in the attribute distribution calculation method described above, the attribute determination unit 38 specifies which of the attributes of each pixel is “vegetation”, “soil”, “road” or “unknown” included in the attribute library, Although the results are totaled, the attribute distribution may be calculated by calculating the score for each type of attribute by the attribute determination unit 38 and totaling the score values. In this case, the attribute determination unit 38, for example, for each pixel included in each image data, as a score value such as "vegetation: XX%, soil: XX%, road XX%, unknown XX%", Attribute determination is made. Then, the attribute distribution calculation unit 42 calculates the total score value for each type of attribute for each point of the shape information, and adopts the score with the maximum total value as the attribute determination result. As a result, the attribute distribution can be calculated with higher accuracy because the score value can be added to the attributes other than the attribute with the highest score value.

When performing aggregation based on score values as described above, if the score difference between the attribute with the largest total value and the attribute with the next largest total value is small, accurate attribute determination is difficult. Very likely. Therefore, when the score difference is less than a predetermined threshold, the attribute distribution calculation unit 42 may calculate the attribute distribution with the determination result regarding the pixel set to "unknown".

Further, the attribute distribution calculation unit 42 obtains occlusion (hiding) at the imaging point of each image data based on the shape information, thereby excluding attribute determination results clearly identified as errors from the attribute determination results by the attribute determination unit 38. Attribute distribution may be calculated by summing up. The shape information specifies a three-dimensional structure including uneven topography and obstacles in the target area 2 . Therefore, by considering the three-dimensional structure, the attribute distribution calculation unit 42 determines attributes that cannot be determined on the optical axis of the imaging device 4 from the positional relationship between the imaging point of each image data and each pixel. If so, the determination result is treated as an error and excluded from the aggregation target. Since the attribute determination unit 38 performs attribute determination based on the two-dimensional image, intermediate information between the imaging point and the target area 2 is not included. It becomes possible to consider intermediate information. In this manner, the attribute distribution calculation unit 42 uses the occlusion obtained from the shape information, thereby excluding attribute determinations that are clearly erroneous, and can further improve the calculation accuracy of the attribute distribution.
Note that occlusion is a phenomenon in which an object in the foreground hides an object in the background and cannot be seen because there is a front-to-back relationship in addition to up/down and left/right in a three-dimensional space.

Also, the shape information creating section 40 may take in ready-made information such as a digital map from the outside as the topography information. Such a digital map may include preset attribute information in addition to terrain information. In this case, the attribute distribution calculation unit 42 may set the weighting coefficients for aggregation based on whether or not the attribute information included in the digital map and the determination result by the attribute determination unit 38 match. Specifically, when the determination result by the attribute determining unit 38 matches the attribute information included in the digital map, the reliability may be considered to be high, and a large weighting factor may be set. As a result, the attribute determination results at each point of the terrain information can be weighted according to the degree of matching with the digital map, and the attribute distribution can be calculated with higher accuracy.

FIG. 8 is a diagram showing the attribute distribution calculated in step S103 of FIG. 4 together with a comparative example. FIG. 8A shows the attribute distribution calculated based on the attribute determination results of the multiple image data in step S103 of FIG. 4, and FIG. 8B shows the attribute distribution included in the multiple image data acquired in step S100. It is a comparative example showing an attribute distribution (one of the attribute determination results calculated in step S101) calculated based on single image data.

As is clear from a comparison of FIGS. 8A and 8B, in the attribute distribution calculated in this embodiment, regions with unknown attributes have disappeared, and the accuracy is lower than in the comparative example. Good results have been obtained. In this embodiment, attribute determination results obtained from a plurality of image data captured at different angles are aggregated for each point of the shape information. This is because it is possible to perform attribute determination in an interpolative manner, for example, even in areas where attribute determination is difficult due to occurrence of halation, and areas that are blind spots due to trees or the like from a specific imaging point. This enables accurate attribute determination over a wide range of the target area 2 .

Subsequently, the peculiar point identifying unit 44 identifies peculiar points in the target area 2 based on the attribute distribution calculated by the attribute distribution calculating unit 42 (step S104). A peculiar point is an area having characteristics related to a specific type of attribute with respect to the surrounding area. be done. Since such a disaster site is identified by erosion by soil, the peculiar point identification unit 44 selects an area where the percentage of pixels determined to have the attribute "soil" in the attribute distribution is equal to or greater than a threshold value as a peculiar point. Identify.

In FIG. 8, the peculiar points specified from the attribute distribution are displayed surrounded by dashed lines, and an enlarged view thereof is shown. At the peculiar points identified by the peculiar point identification unit 44 in this way, the ratio of the predetermined specific attribute "soil" is high, and the erosion area of the road by the soil is identified. As is clear from a comparison of FIGS. 8(a) and 8(b), even in the peculiar point specified by the peculiar point specifying unit 44, the attribute is determined to be "soil" in the present embodiment compared to the comparative example. The ratio of the pixels that are displayed has increased, and the state of soil erosion on the road is captured more clearly.

Subsequently, the attribute distribution calculated by the attribute distribution calculation unit 42 and the peculiar points identified by the peculiar point identification unit 44 are superimposed on the digital map corresponding to the target area 2 prepared in advance. The image data for output is processed (step S105), and output and displayed on the output device 20 (step S106). By visually recognizing the display screen of the output device 20, the operator can easily identify the presence or absence and range of the peculiar point in the target area 2. FIG.

As described above, according to the first embodiment, shape information is created by associating corresponding pixels in a plurality of image data obtained by imaging a target area from different angles. By aggregating the attribute determination results for each image data for each point included in the shape information, even if there is an area where attribute determination is difficult with a single image data, the area can be interpolated. , attribute determination can be performed over a wide range. In addition, by aggregating the attribute determination results for a plurality of image data, it is possible to obtain a highly accurate determination result compared to attribute determination based on single image data. Then, based on the attribute distribution obtained in this way, it is possible to easily identify the presence or absence and range of the peculiar point in the target area.

<Second embodiment>
FIG. 9 is a block diagram functionally showing the internal configuration of the main server 10 according to at least one embodiment of the present invention. The main server 10 of this embodiment differs from the above-described embodiments in that it further includes a distance calculator 46 .

In addition, in the present embodiment, common reference numerals are used for configurations common to the above-described other embodiments, and duplicate descriptions are appropriately omitted.

The distance calculation unit 46 calculates the distance from the imaging point for each pixel of each of the plurality of image data acquired by the image data acquisition unit 36 . Such distance calculation for each pixel may be performed based on the shape information created by the shape information creation unit. Since the shape information specifies the coordinates of each point and the imaging point that constitute the target area 2 in the three-dimensional space, each distance can be geometrically calculated based on the coordinates of the imaging point and each pixel. can be done.

The attribute distribution calculator 42 sets a weighting factor according to the distance from the imaging point to each pixel in each image calculated by the distance calculator 46 . Such a weighting coefficient is set to increase as the distance from the imaging point to the pixel decreases, and is multiplied when the attribute distribution calculation unit 42 aggregates the attribute determination results for each point of the shape information. As a result, aggregation can be performed with an emphasis on attribute determination results for pixels that are close in distance from the imaging point. In other words, a pixel closer to the imaging point has higher imaging accuracy than a pixel farther from the imaging point. A pixel far from the imaging point has lower imaging accuracy than a pixel close to the imaging point, so the reliability of the attribute determination result is also low.). In this embodiment, by setting a weighting factor according to the distance from such an imaging point to each pixel, a more reliable attribute distribution can be calculated.

<Third Embodiment>
The following embodiments are characterized by a method of capturing a plurality of image data acquired by the image data acquiring unit 36 by controlling the imaging device 4 and the moving body 6 by the controller 26 . FIG. 10 is a flow chart showing each step of a method for acquiring imaging data according to at least one embodiment of the present invention. Note that the structure of the image analysis system 1 in this embodiment may be the same as in the above-described embodiments.

In general, imaging by the imaging device 4 should be performed under lighting conditions that facilitate image processing for the object to be imaged. The lighting conditions of the imaging device 4 change from moment to moment depending on the positional relationship with the light source such as the sun. Therefore, in the above-described first and second embodiments, the imaging device 4 is basically adjusted so that the brightness of the entire image is constant.

On the other hand, the target area 2 includes various elements including roads, soil, and vegetation. If the imaging conditions are adjusted based on vegetation with low brightness, roads and soil with relatively high brightness will be imaged with a slight halation. Therefore, when an image is captured under certain imaging conditions, halation or the like may occur in the captured data, and an area in which it is difficult to determine the attribute of each pixel may occur.

Therefore, in the present embodiment, when an image is captured by the imaging device 4 while the mobile body 6 is moving over the target area 2, the imaging control unit 32 performs imaging under a plurality of imaging conditions at each imaging point. The device 4 is controlled (step S200). In other words, multiple images are captured under different imaging conditions at each imaging point. A plurality of imaging conditions are set such that at least one of imaging parameters such as imaging position/angle, exposure time, focal length, aperture value, or environmental conditions is different. As a result, even when the lighting conditions change from moment to moment, the acquisition probability of high-quality imaging data at each imaging point can be effectively improved.

Note that the plurality of imaging conditions in step S200 may be set such that the difference between the above-described imaging parameters is equal to or greater than a predetermined value. In other words, a plurality of imaging conditions may be set such that the imaging parameters that define each imaging condition are significantly different. As a result, imaging data can be acquired under a wider range of imaging conditions, so that the probability of acquiring high-quality imaging data at each imaging point can be more effectively improved.

Subsequently, the image data acquisition unit 36 acquires a plurality of image data created by the imaging device 4 (step S201). Here, the plurality of image data acquired by the image data acquisition unit 36 includes image data captured under different imaging conditions at different imaging points of the target area 2 .

Subsequently, the image data acquisition unit 36 selectively acquires image data that satisfies a predetermined quality standard from the plurality of image data captured by the imaging device 4 by analyzing the acquired image data. That is, by excluding low-quality image data from the image data acquired by the image data acquiring unit 36, image data to be used for subsequent image analysis is selected (step S202). Selection of the image data in step S202 is performed depending on whether or not the image data has a quality standard that enables appropriate attribute determination. This makes it possible to eliminate image data unsuitable for attribute determination due to occurrence of halation or the like from among a plurality of image data, and select image data suitable for attribute determination.

The attribute determination unit 38 and the shape information creation unit 40 perform attribute determination and shape information creation based on the image data selected by the image data acquisition unit 36 . That is, in the third embodiment, the plurality of image data acquired by the method of FIG. 10 are analyzed in the same manner as in the above-described embodiment by performing the same processing as in step S101 and subsequent steps of FIG. Calculations and identification of singular points are performed.

As described above, in the third embodiment, attribute determination and shape information creation are performed based on a plurality of high-quality image data, so attribute distribution can be calculated with higher accuracy. In addition, when imaging data is acquired under a plurality of imaging conditions at each imaging point, the amount of imaging data increases and the amount of processing data tends to become enormous. By sorting by using the above method, it is possible to effectively reduce the processing load and the processing time while ensuring good analysis accuracy. That is, by performing imaging under a plurality of different imaging conditions at each imaging point, it is possible to effectively improve the possibility of obtaining high-quality imaging data. The image data acquisition unit 36 selectively acquires image data that satisfies a predetermined quality standard from a plurality of captured image data, thereby further improving the attribute determination accuracy and shortening the processing time.

As described above, according to the above embodiments, it is possible to provide an image analysis system, an image analysis method, and an image analysis program capable of accurately determining attributes over a wide range of target areas.

At least one embodiment of the present invention can be used for an image analysis system, an image analysis method, and an image analysis program capable of image analysis by attribute determination of elements included in an image.

1 Image analysis system 2 Target area 4 Imaging device 6 Mobile object 7 Communication network 10 Main server 12 Arithmetic device 14 Storage device 16 Drive device 18 Input device 20 Output device 22 Computer program (image analysis program)
24 recording medium 26 controller 28 image analysis device 30 movement control unit 32 imaging control unit 36 image data acquisition unit 38 attribute determination unit 40 shape information creation unit 42 attribute distribution calculation unit 44 peculiar point identification unit 46 distance calculation unit

Claims (17)

an image data acquisition unit that acquires a plurality of image data obtained by imaging a target area from different angles;
an attribute determination unit that determines an attribute for each pixel of each of the plurality of image data;
a shape information creation unit that creates shape information in the target area by associating each pixel included in the plurality of image data with each other;
an attribute distribution calculation unit that calculates an attribute distribution in the target area by summing up determination results by the attribute determination unit for each point included in the shape information;
a peculiar point identifying unit that identifies a peculiar point in the target area based on the attribute distribution;
An image analysis system. The attribute distribution calculation unit counts the number of determinations for each type of the attribute for each point of the shape information, and calculates the attribute distribution by adopting the attribute of the type with the maximum number of determinations. The image analysis system according to claim 1. The attribute determination unit obtains a score for each attribute type,
The attribute distribution calculation unit calculates the total value of the scores for each type of the attribute for each point of the shape information, and adopts the attribute of the type having the maximum total value, thereby calculating the attribute distribution. 2. The image analysis system of claim 1, wherein the image analysis system calculates A distance calculation unit that calculates a distance from the imaging point for each pixel of each of the plurality of image data,
4. The image analysis system according to claim 2, wherein said attribute distribution calculator sets a weighting factor according to said distance for each point included in said shape information. 2. The image analysis system according to claim 1 , further comprising an output device capable of displaying said attribute distribution superimposed on a digital map. 6. The image analysis system according to claim 5 , wherein said output device displays said peculiar point by distinguishing it from surrounding areas. 7. The image analysis system according to any one of claims 1 to 6 , wherein said shape information is three-dimensional information created based on said plurality of image data. The image analysis system according to any one of claims 1 to 7 , wherein said attribute distribution calculation unit calculates said attribute distribution in consideration of occlusion obtained based on said shape information. a moving body equipped with an imaging device capable of imaging the plurality of images;
a movement control unit for controlling the moving body;
an imaging control unit for controlling the imaging device;
The image analysis system according to any one of claims 1 to 8 , comprising: The imaging control unit controls the imaging device so that imaging is performed under a plurality of different imaging conditions at each imaging point,
10. The image analysis system according to claim 9 , wherein said image data acquisition unit selectively acquires image data satisfying a predetermined quality standard from said plurality of image data captured by said imaging device. 11. The image analysis according to claim 10 , wherein the plurality of imaging conditions are set such that at least one imaging parameter including the imaging position/angle, exposure time, focal length, aperture value, or environmental conditions of the imaging device is set differently. system. 12. The image analysis system according to claim 11 , wherein said plurality of imaging conditions are set such that the difference between said imaging parameters is equal to or greater than a predetermined value. an image data acquisition unit that acquires a plurality of image data obtained by imaging a target area from different angles;
an attribute determination unit that determines an attribute for each pixel of each of the plurality of image data;
a shape information creation unit that creates shape information in the target area by associating each pixel included in the plurality of image data with each other;
an attribute distribution calculation unit that calculates an attribute distribution in the target area by summing up determination results by the attribute determination unit for each point included in the shape information;
with
The attribute distribution calculation unit counts the number of determinations for each type of the attribute for each point of the shape information, and calculates the attribute distribution by adopting the attribute of the type with the maximum number of determinations. Image analysis system. an image data acquisition unit that acquires a plurality of image data obtained by imaging a target area from different angles;
an attribute determination unit that determines an attribute for each pixel of each of the plurality of image data;
a shape information creation unit that creates shape information in the target area by associating each pixel included in the plurality of image data with each other;
an attribute distribution calculation unit that calculates an attribute distribution in the target area by summing up determination results by the attribute determination unit for each point included in the shape information;
with
The attribute determination unit obtains a score for each attribute type,
The attribute distribution calculation unit calculates the total value of the scores for each type of the attribute for each point of the shape information, and adopts the attribute of the type having the maximum total value, thereby calculating the attribute distribution. Image analysis system to calculate. an image data acquisition unit that acquires a plurality of image data obtained by imaging a target area from different angles;
an attribute determination unit that determines an attribute for each pixel of each of the plurality of image data;
a shape information creation unit that creates shape information in the target area by associating each pixel included in the plurality of image data with each other;
an attribute distribution calculation unit that calculates an attribute distribution in the target area by summing up determination results by the attribute determination unit for each point included in the shape information;
a moving body equipped with an imaging device capable of imaging the plurality of images;
a movement control unit for controlling the moving body;
an imaging control unit for controlling the imaging device;
with
The imaging control unit controls the imaging device so that imaging is performed under a plurality of different imaging conditions at each imaging point,
The image data acquisition unit selectively acquires image data that satisfies a predetermined quality standard from the plurality of image data captured by the imaging device,
The plurality of imaging conditions are set so that at least one imaging parameter including the imaging position/angle of the imaging device, exposure time, focal length, aperture value, or environmental conditions is different,
The image analysis system, wherein the plurality of imaging conditions are such that a difference between the imaging parameters is set to be equal to or greater than a predetermined value. Acquiring a plurality of image data obtained by imaging the target area from different angles;
determining an attribute for each pixel of each of the plurality of image data;
creating shape information in the target area by associating each pixel included in the plurality of image data with each other;
a step of calculating an attribute distribution in the target area by aggregating the attribute determination results for each point included in the shape information;
Identifying a singular point in the target area based on the attribute distribution;
An image analysis method comprising: Acquiring a plurality of image data obtained by imaging the target area from different angles;
determining an attribute for each pixel of each of the plurality of image data;
creating shape information in the target area by associating each pixel included in the plurality of image data with each other;
a step of calculating an attribute distribution in the target area by aggregating the attribute determination results for each point included in the shape information;
Identifying a singular point in the target area based on the attribute distribution;
An image analysis program that causes a computer to execute

Images