JP7359380B2 - Detection parameter generation device, detection parameter generation method, detection parameter generation program, object detection device, object detection method, and object detection program - Google Patents

Description

The present invention relates to a detection parameter generation device, a detection parameter generation method, a detection parameter generation program, an object detection device, an object detection method, and an object detection program, and more specifically, to a detection parameter generation device, a detection parameter generation method, a detection parameter generation program, and an object detection program. The present invention relates to a detection parameter generation device, a detection method, and a detection program that generate detection parameters for an object, and an object detection device, a detection method, and a detection program that detect an object using the generated detection parameters.

Conventionally, structure adaptive deep learning methods have been known (Non-Patent Documents 1 to 3, Patent Document 1). According to this method, when constructing a deep belief network (DBN) by stacking restricted Boltzmann machines (RBM) in multiple stages, the optimal number of hidden neurons and hidden layers are automatically determined. Thereby, it is possible to construct a neural network with an optimal structure based on the learning data. A structure-adaptive DBN constructed using this structure-adaptive deep learning method has shown higher classification accuracy than existing convolutional neural networks (CNNs) in multiple benchmark tests (Non-Patent Document 4).

Note that a fine tuning method is known as a method for improving classification accuracy (for example, Non-Patent Document 5). In this method, the weights of the network are modified according to the frequency of input/output patterns of each layer of the network after learning so that incorrectly classified cases are classified correctly.

The present inventors have proposed an algorithm that can detect the presence or absence of cancer with high accuracy using a structure-adaptive DBN trained using a public database of chest X-ray images (ChestX-Ray8: CXR8). (Non-patent Document 6).

In this algorithm, first, an image containing an object to be detected is input, and the image is divided along a plurality of regions to obtain a plurality of divided images. Then, the divided images are given to the learned structure-adaptive DBN to obtain the values output from the output layer. The value output from each neuron is normalized by a softmax function and indicates the probability of object detection.

Next, it is determined whether a value (probability) output from a neuron corresponding to a certain object is larger than a threshold (first detection parameter) set for the object. Then, if the output value is larger than the first detection parameter, a deformed image is created based on the divided image, the deformed image is given to the learned structure adaptive DBN, and the value output from the output layer is get. It is determined whether the output value (probability) is larger than another threshold (second detection parameter) set for the object. Then, if the output value is larger than the second detection parameter, it is finally determined that the object has been detected. Note that the first and second detection parameters have values greater than 0 and less than 1, and the smaller the value, the easier the object is to be detected.

JP2019-74946A

S. Kamada and T. Ichimura, An Adaptive Learning Method of Restricted Boltzmann Machine by Neuron Generation and Annihilation Algorithm, Proc. of 2016 IEEE International Conference on Systems, Man, and Cybernetics (IEEE SMC 2016), pp.1273-1278 (2016 ) S.Kamada and T.Ichimura, A Structural Learning Method of Restricted Boltzmann Machine by Neuron Generation and Annihilation Algorithm, Neural Information Processing, Lecture Notes in Computer Science (LNCS, vol.9950), pp.372-380 (2016) S.Kamada and T.Ichimura, An Adaptive Learning Method of Deep Belief Network by Layer Generation Algorithm, Proc. of 2016 IEEE Region 10 Conference (TENCON), pp.2971-2974 (2016) S. Kamada, T. Ichimura, Akira Hara, and Kenneth J. Mackin, Adaptive Structure Learning Method of Deep Belief Network using Neuron Generation-Annihilation and Layer Generation, Neural Computing and Applications, doi.org/10.1007/s00521-018-3622 -y, pp.1-15 (2018) S.Kamada and T.Ichimura, Fine Tuning of Adaptive Learning of Deep Belief Network for Misclassification and its Knowledge Acquisition, International Journal Computational Intelligence Studies, Vol.6, No.4, pp.333-348 (2017) Takumi Ichimura, Makoto Kamata, “Chest disease location detection system using structure-adaptive deep belief network using ChestX-ray8”, Proceedings of the Society of Instrument and Control Engineers, Systems and Information Division Academic Conference 2018 (SSI2018) (2018)

It has been confirmed that high detection accuracy can be obtained for chest X-ray images by using the object detection algorithm described above.

However, further research by the present inventors has revealed that it may be difficult to obtain high detection accuracy depending on the type of image. For example, in the case of images related to architectural equipment drawings, high detection accuracy may not be obtained.

Building equipment drawings include various types of symbols that indicate lighting, outlets, toilets, etc. Furthermore, each symbol has different characteristics such as frequency and rarity. Here, the frequency refers to the degree to which an object (symbol) appears in a certain image (drawing), and the rarity refers to the degree to which the object is rare in a plurality of images. Rarity is high when an object only appears in a specific drawing.

In conventional object detection algorithms, the first and second detection parameters are given as common values to each object. Therefore, with conventional algorithms, it has been difficult to detect various symbols included in drawings with high accuracy. High detection accuracy can sometimes be obtained by searching for detection parameters through trial and error, but this is extremely time-consuming and inefficient.

The present invention has been made based on the above-mentioned technical recognition, and its purpose is to easily generate detection parameters that make it possible to detect multiple types of objects with different characteristics with high accuracy. To provide a parameter generation device, a detection parameter generation method, and a detection parameter generation program, and an object detection device, an object detection method, and an object that can detect multiple types of objects with different characteristics with high accuracy using the generated detection parameters. The purpose is to provide a detection program.

The detection parameter generation device according to the present invention includes:
A detection parameter generation device that generates detection parameters for detecting an object from an image using a structure adaptive DBN, comprising:
a frequency index calculation unit that calculates a first index indicating the frequency of a certain object in one image of a dataset having a plurality of images;
a rarity index calculation unit that calculates a second index indicating the rarity of the object in the data set;
a first detection parameter for determining whether the object is present in a divided image obtained by dividing the detection image based on the product of the first index and the second index; a detection parameter updating unit that updates a second detection parameter for determining whether the object is present in the deformed image created based on the divided image;
It is characterized by having the following.

Further, in the detection parameter generation device,
The detection parameter updating unit may decrease the first detection parameter and the second detection parameter as the product of the first index and the second index increases.

Further, in the detection parameter generation device,
The image may be an image related to a building equipment drawing, and the object may be a symbol of the building equipment drawing.

The object detection device according to the present invention includes:
an image input section for inputting a detection image;
an image dividing unit that divides the detection image into a plurality of divided images;
a first probability acquisition unit that applies the divided image to an input layer of a structure adaptive DBN and obtains a first probability for a certain object output to an output layer of the structure adaptive DBN;
a deformed image creation unit that creates a deformed image based on the divided image if the first probability is greater than the first detection parameter generated by the detection parameter generation device;
a second probability acquisition unit that applies the deformed image to the input layer of the structure-adaptive DBN to obtain a second probability for the object output to the output layer of the structure-adaptive DBN;
a determination unit that determines that the object has been detected if the second probability is greater than the second detection parameter generated by the detection parameter generation device;
It is characterized by having the following.

According to the present invention, it is possible to easily generate detection parameters that enable highly accurate detection of multiple types of objects with different characteristics.

Further, according to the present invention, multiple types of objects with different characteristics can be detected with high accuracy using the detection parameters.

FIG. 2 is a diagram schematically showing an overall processing flow related to object detection according to the embodiment. FIG. 2 is a diagram showing an overview of training data and test data used to construct a structure-adaptive DBN. FIG. 3 is a diagram showing an example of symbols classified by category used in constructing a structure-adaptive DBN. FIG. 2 is a diagram schematically showing an example of a constructed structure-adaptive DBN. FIG. 3 is a diagram showing the classification accuracy of the constructed structure-adaptive DBN. 1 is a diagram showing a schematic configuration of a detection parameter generation device according to an embodiment. It is a figure showing an example of a drawing database concerning an embodiment. 3 is a flowchart illustrating a detection parameter generation method according to an embodiment. 1 is a diagram showing a schematic configuration of an object detection device according to an embodiment. It is a figure which shows an example of the image of a building equipment drawing. FIG. 3 is a diagram for explaining a method of creating a deformed image. 3 is a flowchart illustrating an object detection method according to an embodiment. It is a figure which shows the detection accuracy of the symbol by the conventional (manual adjustment) and embodiment (automatic adjustment). FIG. 6 is a diagram illustrating an example of a case where a symbol cannot be detected correctly.

Embodiments according to the present invention will be described below with reference to the drawings.

<Overall processing flow>
With reference to FIG. 1, an overall processing flow related to object detection using a structure-adaptive DBN according to this embodiment will be described.

Note that in this embodiment, the image including the object is an image related to a building equipment drawing including various symbols. However, the present invention is not limited to this, and can be applied to line drawings other than architectural equipment drawings, as well as various other images containing objects to be detected.

In step S1, a structure adaptive DBN is constructed. In this step, a structure-adaptive DBN is constructed by a known method developed by the present inventors (non-patent documents 1 to 3, etc.). Images of symbols related to a plurality of classification targets are used as training data (learning data) and test data (evaluation data). Data for each pixel of the symbol image is input to each neuron in the input layer of the neural network. Note that fine tuning may be performed to further improve classification accuracy.

FIG. 2 shows, by category, the number of training data used in step S1 and the number of test data for evaluating the structure-adaptive DBN. In addition, in order to increase the learning data, data augmentation was performed to increase the number of data. Specifically, the number of data was increased approximately 12 times by performing operations such as horizontal flip, vertical flip, 90 degree rotation, 190 degree rotation, vertical movement, enlargement, and reduction on the original symbol image. The numbers shown in FIG. 2 are the numbers of data after data augmentation.

FIG. 3 shows examples of symbols for each category. The symbol M1 is an example of a symbol image belonging to the category "sanitary utensils." Symbol M2 is an example of a symbol image belonging to the category "electric light." Symbol M3 is an example of a symbol image belonging to the category "outlet". Symbol M4 is an example of a symbol image belonging to the category "emergency lights/guidance lights." Symbol M5 is an example of a symbol image belonging to the category "intercom."

FIG. 4 schematically shows an example of the constructed structure-adaptive DBN. The output layer has neurons N1, N2, N3, N4, and N5 corresponding to the number of categories. The output values of the neurons N1 to N5 are normalized by a softmax function or the like so that the sum becomes 1. The output values of neurons N1, N2, N3, N4, and N5 indicate the probability of the associated object. For example, when the output value of neuron N1 is 0.8, the probability that the symbol image input to the input layer is in the category "sanitary utensils" is 80%.

After constructing a structure-adaptive DBN using training data, the classification accuracy is evaluated. FIG. 5 shows the evaluation results of the structure-adaptive DBN. Here, the classification accuracy for training data, test data, and test data after fine-tuning is shown for each category. Note that the numerical value in parentheses in the classification accuracy of test data indicates the number of data that could not be classified.

As shown in FIG. 5, the correct answer rate for the training data was 100.0% in all categories. On the other hand, the test data was over 95.6%. Furthermore, when an attempt was made to improve the classification accuracy using a fine tuning method, a classification accuracy of 100.0% was obtained for all categories.

In step S2, detection parameters (a first detection parameter and a second detection parameter) for detecting symbols on building equipment drawings are generated. Here, the first detection parameter (T1 _i ) determines whether the i-th object (object i) exists in the divided image obtained by dividing the image for object detection (detection image). This is the threshold for The second detection parameter (T2 _i ) is a threshold value for determining whether object i exists in the deformed image created based on the divided images. Generation of detection parameters will be explained in detail later with reference to FIGS. 6 to 8.

In step S3, an object is detected from the detection image using the detection parameters generated in step S2. In step S3, the first and second detection parameters generated in step S2 are used as thresholds for determining whether an object exists in the image. Object detection will be described in detail later with reference to FIGS. 9 to 14.

<Detection parameter generation device>
The detection parameter generation device 1 according to this embodiment will be described with reference to FIG. 6. As described below, the detection parameter generation device 1 is configured to generate (adjust) detection parameters for detecting objects from images using the structure-adaptive DBN constructed in step S1.

As shown in FIG. 6, the detection parameter generation device 1 includes a control section 10, a communication section 20, an operation input section 30, a display section 40, and a storage section 50.

The control unit 10 controls the operation of the detection parameter generation device 1, and is constituted by a processor such as a CPU (central processing unit) as hardware. In this embodiment, the control unit 10 is realized by a processor in the detection parameter generation device 1 executing a predetermined program. Note that at least a portion of the control unit 10 may be configured by hardware such as an ASIC. Details of the control unit 10 will be explained in detail later.

The communication unit 20 transmits and receives information between the detection parameter generation device 1 and an external information processing device (not shown). Note that the communication by the communication unit 20 may be wired or wireless, and the communication protocol is not limited.

The operation input unit 30 is an interface for a user to input information to the detection parameter generation device 1, and is, for example, a keyboard, a mouse, a touch panel, a button, or the like. The operation input unit 30 receives initial values of detection parameters, detection parameter generation instructions, etc. from the user.

The display unit 40 is an interface that outputs various information (such as generated detection parameter values) to the user. The display unit 40 is, for example, a display (liquid crystal display, organic EL display, etc.) that displays images.

The storage unit 50 is a storage device composed of a hard disk, a semiconductor memory, or the like. This storage unit 50 stores a drawing database 51. Note that the storage unit 50 may store programs and various data necessary for information processing by the control unit 10.

The drawing database 51 is a database that stores the number of appearances (number of drawings) of each symbol in each drawing, which is extracted from a data set having images related to a plurality of building equipment drawings. The drawing database 51 is created by a person viewing a drawing image. Note that the number of parts in the drawing database 51 does not necessarily have to be a precise value, as long as the frequency and rarity trends of each part are correct.

FIG. 7 shows an example of the drawing database 51. The drawing database 51 stores the number of each part (symbol of the building equipment drawing) for each of the J building equipment drawings. Note that I is the total number of types of parts. That is, the drawing database 51 stores the number of parts i (1≦i≦I) drawn in drawing j (1≦j≦J). The rightmost column stores the total number of parts for each drawing. Note that this column is not essential, and may be calculated by the control unit 10.
Next, the control section 10 will be explained in detail.

As shown in FIG. 6, the control unit 10 includes a data input unit 11, a frequency index calculation unit 12, a rarity index calculation unit 13, and a detection parameter update unit 14. Note that each functional unit included in the control unit 10 is provided in a distributed manner in a plurality of information processing devices that are communicatively connected, and the functions of the control unit 10 are realized by cooperation of these plurality of information processing devices. good.

The data input unit 11 acquires the initial value of the first detection parameter and the initial value of the second detection parameter. It is preferable that the initial value of the first detection parameter (T1 _i,1 ) and the initial value of the second detection parameter (T2 _i,1 ) have a relationship of T1 _i,1 ≦T2 _i,1 . Note that as the initial value, a common value that does not depend on the type of object may be acquired, or a value for each type of object may be acquired.

The frequency index calculation unit 12 calculates a first index indicating the frequency of a certain object (object i) in one image of the data set. In this embodiment, the frequency index calculation unit 12 refers to the drawing database 51 and calculates the first index x _i,j using equation (1). The first index x _i,j indicates the proportion of object i to all objects in image j.

Here, x _i,j is the first index, n _i,j is the number of objects i in image j, and I is the total number of object types in the data set.

For example, if an image j related to a certain building equipment drawing includes 100 parts and the number of parts i is 0, the first index x _i,j is 0. On the other hand, if image j includes 100 parts, all of which are part i, the first index x _i,j is 1. In this way, the first index x _i,j takes a value of 0 or more and 1 or less depending on the frequency of object i in image j.

Note that the frequency index calculation unit 12 uses the value stored in the right column of the drawing database 51 in FIG. 7 as the denominator on the right side of equation (1). If there is no column in the right column, the frequency index calculation unit 12 may calculate the sum of parts 1, 2, . . . , part I.

The rarity index calculation unit 13 calculates a second index indicating the rarity of object i in the data set (J images). The second index y _i is determined by a function that returns a larger value as the number of images containing object i in the data set is smaller. In this embodiment, the rarity index calculation unit 13 refers to the drawing database 51 and calculates the second index using equation (2).

Here, y _i is the second index, J is the total number of images in the data set, and A _i is the number of images containing object i.

For example, if the total number of images related to architectural equipment drawings is 10, and only one of the images includes part i, the second index y _i is 1. On the other hand, if the total number of drawings is 10 and all drawings include part i, the second index y _i will be 0. In this way, the second index y _i takes a value greater than or equal to 0 and less than or equal to log|J|, depending on the rarity of the object i in the plurality of images included in the data set.

The detection parameter updating unit 14 updates the first detection parameter (T1 _i ) and the second detection parameter (T2 _i ) based on the product of the first index and the second index.

More specifically, the detection parameter updating unit 14 decreases the first detection parameter and the second detection parameter as the product of the first index and the second index increases. In this embodiment, the detection parameter update unit 14 updates the first detection parameter using equation (3).

Here, T1 _i,j+1 : the first detection parameter of object i after update, T1 _i,j : the first detection parameter of object i before update, z _i,j are the first index and 2 with the index (=x _i,j ×y _i ).

Similarly, the detection parameter updating unit 14 updates the second detection parameter using equation (4).

Here, T2 _i,j+1 : second detection parameter after update of object i, T2 _i,j : second detection parameter before update of object i, z _i,j are the first index and 2 with the index (=x _i,j ×y _i ).

According to equations (3) and (4), the more frequent and rare object i is (that is, the larger z _i,j is), the smaller the updated values of the first and second detection parameters will be. Therefore, it becomes easier to detect the object. Generally, if the detection parameter is made small, there is a risk of false detection, but in this embodiment, detection accuracy is improved by adjusting the detection parameter based on the product of the first index and the second index. are doing. The actual detection results will be explained later with reference to FIG. 13.

Note that equations (3) and (4) are just examples. For example, as shown in equation (5), z _i,j may be calculated using the function f. The function f is an increasing function, such as an exponential function, a logarithmic function, or a polynomial function.

In addition, z _i,j is obtained as the product of the values obtained by converting the first index x _i,j and the second index y _i using the function f and the function g, respectively, as shown in equation (6). It's okay. The functions f and g are increasing functions, such as exponential functions, logarithmic functions, and polynomial functions. In this application, the expression "(based on) the product of the first index and the second index" includes the product of the value based on the first index and the value based on the second index.

As described above, the detection parameter generation device 1 according to the present embodiment calculates a first index indicating frequency and a second index indicating rarity for each object. Then, first and second detection parameters for each object are generated based on the product of the first index and the second index. This allows the detection parameters to be adjusted for each object according to the frequency and rarity of the object, making it possible to detect multiple types of objects with different characteristics with high accuracy without the need for trial and error. can be generated.

Further, according to the present embodiment, since detection parameters for each object are automatically adjusted, it is possible to achieve a significant increase in efficiency compared to the conventional manual method.

Note that when updating the detection parameters, adjustment parameters that can be manually set may be included. This allows the detection parameters to be adjusted based on human senses. For example, the first detection parameter and the second detection parameter are updated using equation (7) and equation (8), respectively. In these formulas, the adjustment parameters α ₁ and α ₂ are usually 1, but if the detection accuracy does not improve as expected due to the data having a different trend from before, these manual adjustment parameters can be set to 1. Detection accuracy can be improved by increasing or decreasing from .

Furthermore, the first and second detection parameters may be updated in accordance with the update of the data set. That is, if a new drawing is acquired after detection parameters for J building equipment drawings have been generated, the drawing database 51 may be updated and the first and second detection parameters may be updated.

<How to generate detection parameters>
Next, with reference to FIG. 8, a method for generating detection parameters according to this embodiment will be described.

In step S11, the data input unit 11 obtains initial values (T1 _i,1 , T2 _i,1 ) of the first detection parameter and the second detection parameter for each object. Note that the initial value acquired in this step may be a common value that does not depend on the type of object. Further, the initial value may be input by the user via the operation input unit 30, may be stored in advance in the storage unit 50, or may be acquired from an external device via the communication unit 20. .

In step S12, the control unit 10 initializes two variables i and j. In this embodiment, variable i and variable j are set to 1. The variable i indicates the object number, and the variable j indicates the image number. Note that this step S12 may be performed before step S11.

In step S13, the control unit 10 determines whether the variable j is less than or equal to the number J of images in the data set. If the variable j is less than or equal to the number of images J (S13: Yes), the process advances to step S14.

On the other hand, if the variable j is larger than the number of images J (S13: No), the detection parameter generation process is ended. At this time, if the first detection parameter is less than 0, the finally generated value of the first detection parameter is set to a predetermined minimum value (for example, 0). Further, when the first detection parameter is larger than 1, the value of the first detection parameter finally generated is a predetermined maximum value (for example, 1). The same applies to the second detection parameter.

In step S14, the control unit 10 determines whether the variable i is less than or equal to the number of object types I. If the variable i is less than or equal to the number of types I (S14: Yes), the process advances to step S15. On the other hand, if the variable i is larger than the number of types I (S14: No), the variable j is increased by one (step S19), and then the process returns to step S13.

In step S15, the frequency index calculation unit 12 calculates a first index x _i,j indicating the frequency of object i in image j. In this embodiment, the frequency index calculation unit 12 refers to the drawing database of FIG. 7 and calculates the first index x _i,j using the above-mentioned equation (1).

In step S16, the rarity index calculation unit 13 calculates a second index y _i indicating the rarity of the object i in the data set (J images). In this embodiment, the rarity index calculation unit 13 refers to the drawing database shown in FIG. 7 and calculates the second index y _i using the above-mentioned formula (2). Note that the order of execution of step S15 and step S16 may be reversed.

In step S17, the detection parameter updating unit 14 updates the first detection parameter T1 i and the second detection parameter _{T2 i} based on the product of the first index x _i,j and the second index y _i . do. In this embodiment, the detection parameter update unit 14 updates the first detection parameter T1 _i using the above-mentioned equation (3), and updates the second detection parameter T2 _i using the above-mentioned equation (4). do.

After step S17, the control unit 10 increases the variable i by one (step S18), and then returns to step S14.

According to the above detection parameter generation method, the detection parameters are adjusted for each object according to the frequency and rarity of the object, and the detection parameters make it possible to detect multiple types of objects with different characteristics with high accuracy. can be easily generated.

<Object detection device>
Next, with reference to FIG. 9, an object detection device 100 that detects an object included in an image (detection image) using the first and second detection parameters generated by the detection parameter generation device 1 will be described. do.

The object detection device 100 according to this embodiment includes a control section 110, a communication section 120, an operation input section 130, a display section 140, and a storage section 150, as shown in FIG.

The control unit 110 controls the operation of the object detection device 100, and is constituted by a processor such as a CPU (central processing unit) as hardware. In this embodiment, the control unit 110 is realized by a processor within the object detection device 100 executing a predetermined program. Note that at least a portion of the control unit 110 may be configured by hardware such as an ASIC. Details of the control unit 110 will be explained in detail later.

The communication unit 120 transmits and receives information between the object detection device 100 and an external information processing device (not shown). For example, the communication unit 120 receives detection parameters from the detection parameter generation device 1 . Note that the communication by the communication unit 120 may be wired or wireless, and the communication protocol is not limited.

The operation input unit 130 is an interface for a user to input information to the object detection device 100, and is, for example, a keyboard, a mouse, a touch panel, a button, or the like. The operation input unit 130 receives designations of detection images, object detection instructions, and the like from the user.

The display unit 140 is an interface that outputs various information (for example, object detection results, etc.) to the user. The display unit 140 is, for example, a display (liquid crystal display, organic EL display, etc.) that displays images.

The storage unit 150 is a storage device composed of a hard disk, a semiconductor memory, or the like. This storage unit 150 stores DBN data 151 and a detection parameter database 152. Note that the storage unit 150 may store programs and data necessary for information processing by the control unit 110.

The DBN data 151 is data related to the structure-adaptive DBN constructed in step S1 (parameter values such as the shape of the neural network (number of layers, number of neurons in each phase), weights between neurons, bias of neurons, etc.).

The detection parameter database 152 stores a first detection parameter T1 _i and a second detection parameter T2 _i of an object i (1≦i≦I), which are generated by the detection parameter generation device 1.

Next, the control unit 110 will be explained in detail.

As shown in FIG. 9, the control unit 110 includes an image input unit 111, an image division unit 112, a probability acquisition unit (first probability acquisition unit) 113, a deformed image creation unit 114, and a probability acquisition unit (first probability acquisition unit). 2 (probability acquisition unit) 115, and a determination unit 116. Note that each functional unit included in the control unit 110 is provided in a distributed manner in a plurality of information processing devices that are communicatively connected, and the functions of the control unit 110 are realized by cooperation of these plurality of information processing devices. good.

The image input unit 111 inputs a detection image. FIG. 10 shows an example of a detection image (a building equipment drawing of an apartment building). Note that the detection image may be stored in advance in the storage unit 150, may be obtained from an external device via the communication unit 120, or may be obtained from an information recording medium such as a CD-ROM or an SD card. You can.

The image dividing unit 112 divides the detection image into a plurality of divided images. For example, the image dividing unit 112 divides the input detection image along a plurality of rectangular regions.

Note that the division of the detection image is not limited to rectangular shapes, and for example, a Voronoi region may be used to generate divided images that are more suitable for object detection than a rectangular shape. In this case, the image dividing unit 112 generates a plurality of Voronoi regions based on the discrete Voronoi diagram, and divides the detection image into a plurality of divided images using the plurality of generated Voronoi regions.

The probability acquisition unit 113 provides the divided images to the input layer of the structure-adaptive DBN, and obtains the probability (first probability) for a certain object output to the output layer of the structure-adaptive DBN. In this embodiment, the probability acquisition unit 113 sequentially supplies the plurality of divided images created by the image division unit 112 to the input layer of the structure adaptive DBN, and outputs the value of the output layer for each divided image (i.e., for each object). probability).

If the probability acquired by the probability acquisition unit 113 (probability of object i) is larger than the first detection parameter (T1 _i ), the deformed image creation unit 114 creates a plurality of deformed images based on the divided images.

Here, a method for creating a deformed image will be described with reference to FIG. Note that FIG. 11 shows only the outline of the deformed image.

First, the deformed image creation unit 114 extracts a predetermined size from the center point of the divided image. In the example of FIG. 11, an image T of 50 pixels vertically and 50 pixels horizontally is extracted from the center point of the divided image. Then, the modified image creation unit 114 creates a plurality of images by changing the size of the image T. In the example of FIG. 11, the vertical and horizontal sizes of the image T are increased by 10 pixels up to 100 pixels to create a total of 35 images. The plurality of deformed images created by the deformed image creation unit 114 include image T and images obtained by changing the size of image T (36 images in this example).

The probability acquisition unit 115 supplies the deformed image created by the deformed image creation unit 114 to the input layer of the structurally adaptive DBN constructed in step S1, and obtains a certain object outputted to the output layer of the structurally adaptive DBN. Obtain the probability (second probability) for . In this embodiment, the probability acquisition unit 115 sequentially supplies the plurality of deformed images created by the deformed image creation unit 114 to the input layer of the structure adaptive DBN, and outputs the value of the output layer for each deformed image (i.e., Obtain the probability for each object.

The determination unit 116 determines that the object i has been detected if the probability for the object i acquired by the probability acquisition unit 115 is greater than the second detection parameter T2 _i for the object. In this embodiment, if the probability of at least one deformed image among the plurality of deformed images created by the deformed image creation unit 114 for object i is greater than the second detection parameter T2 _i , it is determined that object i has been detected. judge.

As described above, the object detection device 100 according to the present embodiment performs the first-stage determination using the first detection parameter on the divided images obtained by dividing the detection image, and then performs the first-stage determination on the deformed images created from the divided images. A second-stage determination is performed using the detection parameter No. 2. As a result, according to the present embodiment, it is possible to detect a plurality of types of objects with different characteristics from the detection image with high accuracy.

<How to detect objects>
Next, an object detection method according to this embodiment will be described with reference to FIG. 12.

In step S21, the image input unit 111 inputs a detection image.

In step S22, the image dividing unit 112 divides the detection image input in step S21 into a plurality of divided images. In this step, the detection image is divided into N divided images.

In step S23, the control unit 110 initializes two variables i and n. The variable i indicates the object number, and the variable n indicates the divided image number. In this embodiment, variable i and variable n are set to 1. Note that this step S23 may be performed before step S21 or step S22.

In step S24, the control unit 110 determines whether the variable n is less than or equal to the total number N of divided images. If the variable n is less than or equal to the total number N (S24: Yes), the process advances to step S25. On the other hand, if the variable n is larger than the total number N (S24: No), the object detection process ends.

In step S25, the probability acquisition unit 113 provides the n-th divided image to the input layer of the structure-adaptive DBN constructed in step S1, and acquires the probability of each object output to the output layer.

In step S26, the control unit 110 determines whether the variable i is less than or equal to the number of object types I. If the variable i is less than or equal to the number of types I (S26: Yes), the process advances to step S27. On the other hand, if the variable i is larger than the number of types I (S26: No), the variable n is increased by one (step S33), and the process returns to step S24.

In step S27, the control unit 110 determines whether the probability of object i is greater than the first detection parameter T1 _i . If the probability of object i is greater than the first detection parameter T1 _i (S27: Yes), the process advances to step S28. On the other hand, if the probability of object i is less than or equal to the first detection parameter T1 _i (S27: No), the variable i is increased by one (step S32), and then the process returns to step S26.

In step S28, the deformed image creation unit 114 creates a plurality of deformed images based on the n-th divided image. In this step, a plurality of deformed images are created from the divided images as described with reference to FIG.

In step S29, the probability acquisition unit 115 sequentially supplies the plurality of deformed images created in step S28 to the input layer of the structure-adaptive DBN constructed in step S1, and the value of the output layer for each deformed image (i.e. , probability for each object).

In step S30, the determination unit 116 determines whether the probability for the object i acquired in step S29 is greater than the second detection parameter T2 _i for each of the plurality of deformed images created in step S28. Then, if the probability of at least one of the plurality of deformed images for object i is greater than the second detection parameter T2 _i (S30: Yes), it is determined that object i has been detected (step S31). Thereafter, after incrementing the variable i by one (step S32), the process returns to step S26.

On the other hand, if the probability for object i is less than or equal to the second detection parameter T2 _i for all of the plurality of deformed images created in step S28 (S30: No), the determination unit 116 determines that object i is not detected. After making a determination and incrementing the variable i by one (step S32), the process returns to step S26.

As described above, in the object detection method according to the present embodiment, the first stage determination is performed using the first detection parameter for the divided image (step S27), and then the first stage determination is performed for the deformed image created from the divided image. A second-stage determination is performed using the second detection parameter (step S30). A two-stage determination process is performed using the first and second detection parameters generated by the detection parameter generation device 1. As a result, according to the present embodiment, it is possible to detect a plurality of types of objects with different characteristics from the detection image with high accuracy.

FIG. 13 compares the object detection accuracy by the object detection apparatus and method according to this embodiment with the detection accuracy by a conventional object detection algorithm (manual adjustment). The detection accuracy here is the proportion of correctly detected parts out of the total number of parts included in the data set (J building equipment drawings). Note that the values of T1 and T2 in FIG. 13 indicate the initial values of the detection parameters. Here, even in the case of automatic adjustment, the initial value is a common value for each component.

As shown in FIG. 13, when the detection parameters were manually adjusted, a maximum detection accuracy of 98.5% was obtained, but there were variations in the detection accuracy. This is considered to be due to the fact that uniform detection parameters are used without considering the characteristics of each component. On the other hand, in the case of this embodiment (automatic adjustment), the detection parameters are automatically adjusted to the optimal values for each component without going through trial and error like manual adjustment. The highest detection accuracy was obtained in all cases of adjustment 2).

Here, a case will be described in which a symbol cannot be detected correctly. FIG. 14 shows a part of the building equipment drawing when the symbol cannot be detected. FIG. 14 includes symbols S31, S32, S33, and S34 indicating electrical outlets. Symbols S32 and S34 could not be detected because images of other appliances were superimposed on the outlet symbols. On the other hand, symbols S31 and S34 were correctly detected although there were other lines near the outlet symbol.

As described above, according to the present embodiment, various types of symbols with different characteristics included in building equipment drawings can be detected efficiently with high accuracy. This makes it possible to greatly improve the efficiency of accurately understanding architectural equipment drawings. As a result, it will be possible to contribute to the diversification of buildings, work style reform, and labor shortage issues.

The embodiments of the present invention have been described above. Although the detection parameter generation device 1 and the object detection device 100 are described as separate devices in the above description, they may be configured as an integrated device.

Based on the above description, those skilled in the art may be able to envision additional effects and various modifications of the present invention, but aspects of the present invention are not limited to the embodiments described above. Various additions, changes, and partial deletions are possible without departing from the conceptual idea and gist of the present invention derived from the content defined in the claims and equivalents thereof.

1 Detection parameter generation device 10 Control unit 11 Data input unit 12 Frequency index calculation unit 13 Rarity index calculation unit 14 Detection parameter update unit 20 Communication unit 30 Operation input unit 40 Display unit 50 Storage unit 51 Drawing database 100 Object detection device 110 Control section 111 Image input section 112 Image division section 113 Probability acquisition section 114 Deformed image creation section 115 Probability acquisition section 116 Judgment section 120 Communication section 130 Operation input section 140 Display section 150 Storage section 151 DBN data 152 Detection parameter database M1, M2, M3, M4, M5 symbol N1, N2, N3, N4, N5 (output layer) neuron

Claims (12)

A detection parameter generation device that generates detection parameters for detecting an object from an image using a structure adaptive DBN, comprising:
a frequency index calculation unit that calculates a first index indicating the frequency of a certain object in one image of a dataset having a plurality of images;
a rarity index calculation unit that calculates a second index indicating the rarity of the object in the data set;
a first detection parameter for determining whether the object is present in a divided image obtained by dividing the detection image based on the product of the first index and the second index; a detection parameter updating unit that updates a second detection parameter for determining whether the object is present in the deformed image created based on the divided image;
A detection parameter generation device comprising: 2. The detection parameter updating unit reduces the first detection parameter and the second detection parameter as the product of the first index and the second index increases. The detection parameter generation device described. 3. The detection parameter generation device according to claim 1, wherein the frequency index calculation unit calculates the first index using equation (1).

Here, x _i,j is the first index, n _i,j is the number of objects in image j, and I is the total number of object types in the data set. 4. The detection parameter generation device according to claim 1, wherein the rarity index calculation unit calculates the second index using equation (2).

Here, y _i is the second index, J is the total number of images in the data set, and A _i is the number of images including the object. 5. The detection parameter generation device according to claim 1, wherein the detection parameter updating unit updates the first detection parameter using equation (3).

Here, T1 _i,j+1 : the first detection parameter after the update of the object, T1 _i,j : the first detection parameter before the update of the object, α ₁ : adjustment parameter, z _i,j : It is the product of the first index and the second index. 6. The detection parameter generation device according to claim 1, wherein the detection parameter updating unit updates the second detection parameter using equation (4).

Here, T2 _i,j+1 : second detection parameter after update of the object, T2 _i,j : second detection parameter before update of the object, α ₂ : adjustment parameter, z _i,j : It is the product of the first index and the second index. 7. The detection parameter generation device according to claim 1, wherein the image is an image related to a building equipment drawing, and the object is a symbol of the building equipment drawing. an image input section for inputting a detection image;
an image dividing unit that divides the detection image into a plurality of divided images;
a first probability acquisition unit that applies the divided image to an input layer of a structure adaptive DBN and obtains a first probability for a certain object output to an output layer of the structure adaptive DBN;
If the first probability is larger than the first detection parameter generated by the detection parameter generation device according to claim 1, a deformed image creation unit that creates a deformed image based on the divided image;
a second probability acquisition unit that applies the deformed image to the input layer of the structure-adaptive DBN to obtain a second probability for the object output to the output layer of the structure-adaptive DBN;
a determination unit that determines that the object has been detected if the second probability is greater than the second detection parameter generated by the detection parameter generation device according to claim 1;
An object detection device comprising: A detection parameter generation method for generating detection parameters for detecting an object from an image using a structure adaptive DBN, the method comprising:
determining a first indicator of the frequency of an object in one of the images in the dataset;
determining a second indicator of the rarity of the object in the dataset;
a first detection parameter for determining whether the object is present in a divided image obtained by dividing the detection image based on the product of the first index and the second index; updating a second detection parameter for determining whether the object is present in the deformed image created based on the divided image;
A detection parameter generation method comprising: inputting a detection image;
dividing the detection image into a plurality of divided images;
providing the divided image to an input layer of a structure-adaptive DBN to obtain a first probability for a certain object output to an output layer of the structure-adaptive DBN;
If the first probability is greater than the first detection parameter generated by the detection parameter generation method according to claim 9, creating a deformed image based on the divided image;
providing the deformed image to an input layer of the structure-adaptive DBN to obtain a second probability for the object output to an output layer of the structure-adaptive DBN;
If the second probability is greater than the second detection parameter generated by the detection parameter generation method according to claim 9, determining that the object has been detected;
An object detection method comprising: A detection parameter generation program that generates detection parameters for detecting an object from an image using a structure adaptive DBN, comprising:
determining a first indicator of the frequency of an object in one of the images in the dataset;
determining a second indicator of the rarity of the object in the dataset;
a first detection parameter for determining whether the object is present in a divided image obtained by dividing the detection image based on the product of the first index and the second index; updating a second detection parameter for determining whether the object is present in the deformed image created based on the divided image;
A detection parameter generation program that causes a computer to execute. inputting a detection image;
dividing the detection image into a plurality of divided images;
providing the divided image to an input layer of a structure-adaptive DBN to obtain a first probability for a certain object output to an output layer of the structure-adaptive DBN;
If the first probability is greater than the first detection parameter generated by the detection parameter generation program according to claim 11, creating a deformed image based on the divided image;
providing the deformed image to an input layer of the structure-adaptive DBN to obtain a second probability for the object output to an output layer of the structure-adaptive DBN;
If the second probability is greater than the second detection parameter generated by the detection parameter generation program according to claim 11, determining that the object has been detected;
An object detection program that causes a computer to run

Images